AN APPLICATION OF THE MULTETRAiT-MULTIMETHOD MATRM TO THE STUDY OF THE N-ACHIEVEMENT CONSTRUCT Thesis gov ”10 Degree of DH. D. MICHEGAN STATE UNIVERSITY Louis J. Hofmann 1965 '”‘ ‘ 'I’Y' , l 3 _.\ ‘ r' ’ ‘ . \ Lo thhfi§”” vfilv a) University TH llfl!“l“fl[l|flfll\ll 1n1luluuuuyuujmw 1013 This is to certify that the thesis entitled AN APPLICATION OF THE MULTITRAlT-MULTIMETHOD MATRIX TO THE STUDY OF THE N-ACHIEVEMENT CONSTRUCT presented by Louis J. Hofmann has been accepted towards fulfillment of the requirements for Ph.D. d __ egree in Services a Educational Psychology Counseling, Personnel A: I ,, z." 1* prelim/7‘1 (n/’ ¢?f3;1€_...-fa;f"§¢u up“ a A 1'. LIL-7m] Major professor Date FEbruary 25. 1966 O-l69 ABSTRACT AN APPLICATION OF THE MULTITRAIT-MULTIMETHOD MATRIX TO THE STUDY OF THE N-ACHIEVEMENT CONSTRUCT~ by Louis J. Hofmann This study was an investigation of the construct validity of certain’nrachievement scales. Nine scales were included in the analysis. Five of the scales were measures of Zrachievement, while four others were measures cfifl-affili- ation. The five scales examined in the study were:.1) the McClelland and Atkinson TAT, 2) the French Test of Insight, 3) the Edwards Personal Preference Schedule, 4)la Self- Report measure of l-achievement and z-affiliation, and 5) the Michigan State M-Scales. The construct validity of these scales was assessed by: 1) comparing the intercorrelations of the test scores within a multitrait-multimethod matrix such as that proposed by Campbell and Fiske, 2) examining the correlations of the nrachievement and z-affiliation scores with intelligence and achievement scores, and 3) a factor analysis of the test scores. The test scores were obtained on a sample of eleventh and twelfth grade male high school students. The multitrait-multimethod matrix analysis yielded evidence for the convergent validity of the z-achievement scales, while the zyaffiliation scales did not meet this Louis J. Hofmann this requirement. The intercorrelations of the‘zrachieve- ment scales were generally positive and significantly dif— ferent from zero. The projective methods were less adequate than the objective methods of measurement. The correlations of the Edwards Personal Preference Inventory, the Self-Report Inventory, and the M-Scales with Lorge-Thorndike Intelligence scores were positive (r = .286, .167, and .388, respectively), while the Test of Insight and the TAT were related to a lesser degree (r = -.217 and .089, respectively). The correlations of the motivation scores with academic achievement were similar. The EPPS, the Self-Report Inventory and the M-Scales were positively related (r = .360, .301 and .356), while the T01 and TAT correlations were lower (r = .200 and -.O98). The factor analysis produced results consistent with the Multitrait-Multimethod Analysis. The five.lrachievement scales had their highest loadings on the same factor as predicted. TheJK-affiliation scales had their highest loadings on a second factor. The failure of projective methods of motivation assessment to‘demonstrate construct validity in this analysis were attributed to the unreliability of these methods along with response sets unique to the production of thematic material. AN APPLICATION OF THE MULTITRAIT-MULTIMETHOD MATRIX TO THE STUDY OF THE N-ACHIEVEMENT CONSTRUCT By . AS Louis J; Hofmann A THESIS Submitted to Michigan State University in partial fulfillment of the requirements for the degree of DOCTOR OF PHILOSOPHY Department of Counseling, Personnel Services, and Educational Psychology 1965 ACKNOWLEDGEMENTS The technical assistance and moral support of many persons was necessary in the completion of this thesis. Dr. William Farquhar and Dr. David Krathwohl were especially supportive. I wish also to thank the graduate students at Michigan State University for their helpful comments. Finally, thanks are also due to my wife Judith for her forbearance during the time required for the comple- tion of the thesis. ii TABLE OF CONTENTS Page ACKNOWLEDGMNTS O O I O O O O I O . I O O O O O O 0 ii LIST OF TABLES O 0 O 0 o o o 0 I o o o o o o 9 o e 0 V LIST OF APPENDICES . . . . . . . . . . . . . . . . . Vi Chapter I INTRODUCTION TO THE PROBLEM . . . . . . . . 1 Need for the Study . . . . . . . . . . . . 4 Clarification of Constructs . . . . . . 4 Problem of Validation of Tests . . . . . 10 New Empirical Evidence . . . . . . . . . 13 Summary of Need for Study . . . . . . . 14 The Purpose of the Study . . . . . . . . . 14 Theory . . . . . . . . . . 15 The Hypotheses of the Study . . . . . . . 21 Overview of the Study . . . . . . . . . . 23 II REVIEW OF THE LITERATURE . . . o . . o . . . 24 Types of Validity . . . . . . . . . . . . 24 Content Validity . . . . . . . . . . . . 25 Predictive Validity . . . . . . . . . . 26 concurrent Validity o o o o o o o o o o 26 Construct Validity . . . . . . . . . . . 27 Trait Validity . . . . . . . . . . . . 33 Nomological Validity . . . . . . . . . 33 Discriminant Validity . . . . . . 34 Summary of Construct Validity . . . . 35 Examples of Construct Validation Studies in the Psychological Literature . . ... . 36 Reliability of Achievement Motivation _, InStrImlents o o o o o o o o o o 41 Summary of Reliability Studies . . . . . 45 Validity Studies of Achievement Motiva- tion Tests . . . . . . . . . . . . . . . 46 Interrelationships Among Tests of . Achievement Motivation . . . . . . . . 46 Achievement Motivation and Intelli- gence................. 48 Achievement Motivation and Academic Performance 0 o o o o o o o o o o o o o 49 Summary of Ach Motivation - Academic Performance Studies . . . . . . . . . 53 Validity Studies of the M-Scales . . . . 53 iii TABLE OF CONTENTS CON'T. Chapter Page III DESIGN OF THE STUDY . . . . . . . . o o o c U! 0‘ sample 6 o o o o o o o o o o o o o o o o o 56 Instrumentation o o o o o o o o o o o o o 56 The M-Scales o o o o o o o o o o o o 57 The Thematic Apperception Test (TAT) . . 57 The French Test of Insight (TOI) . . . . 58 The Edward's Personal Preference Schedule (EPPS) . . . . . . . . 58 Statistical Hypotheses . . . . . . . . . . 60 Statement of Hypotheses . . . . . . . . 61 Analysis Procedures . . . . . . . . . . 64 IV ANALYSIS OF RESULTS ,.....u............. . . 66 Convergent and Discriminant Validity . . . 66 Discussion . . . . . . 72 The Predictive Quality of the Motivar- _ tional Scales . . . . . . . . . . . . . . 75 Factor Analysis of Test Scores . . . . . . 84 V SUMMARY AND CONCLUSIONS . . . . . . . . . . 85 Major Findings . . . . . . . . . . . 85 Discussion of the Major Findings . . . . . 88 Validating and Invalidating Evidence . . . 89 Experimental Procedures Possibly Dis- torting the Results of the Study . . . . 92 Recommendations and Implications of the StUdy o o o o o o o o o o o 9 Ok. 0 o o o 95 BIBLImRAPHY O O O O O D O O O O O C O O O O O O O O 101 APPENDICES I O O O O I O O O O O O O O O O O O O O O 106 iv Table 2.1 2.2 4.2 4.3 4.4 4.5 4.6 4.7 4.8 4.9 4.10 LIST OF TABLES A MODIFIED MULTI-TRAIT--MULTI-METHOD MATRIX USING DATA FROM HILLS' STUDY OF ACHIEVE- MENT NEEDS AND ASPIRATION LEVELS . . . . . . SUMMARY OF STUDIES USING ACHIEVEMENT MOTIVA- TION SCORES TO PREDICT ACADEMIC PERFORM-a ANCE O O O O O O I O O O O O O O O O O O O 0 MEANS, STANDARD DEVIATIONS, AND INTERCOR- RELATIONS OF THE N-ACHIEVEMENT AND~ _ N-AFFILIATION TEST SCORES FOR 112 MALE HIGH SCHOOL STUDENTS . . . . . . . . . . . . . . INTERCORRELATIONS OF THE TAT, EPPS, TOT, SELF-REPORT INVENTORY, AND M-SCALE “ N-ACHIEVEMENT SCALES FOR 112 MALE HIGH SCHOOL STUDENTS . . . . . . . . . . . . . . INTERCORRELATIONS OF THE TAT, TOI, EPPS AND SELF-REPORT INVENTORY N-AFFILIATION TEST SCORES FOR 112 MALE HIGH SCHOOL STUDENTS . . COMPARISONS OF N-ACH VALIDITY COEFFICIENTS AND HETEROTRAITLHETEROMETHOD COEFFICIENTS-— THE HYPOTHESES AND OBTAINED COEFFICIENTS . . THE COMPARISON OF VALIDITY N-ACH COEFFICIENTS AND HETEROTRAIT-MONOMETHOD COEFFICIENTS SHOWING HYPOTHESES AND OBTAINED VALUES . . . SIGNIFICANCE TESTS AND CORRELATIONS BETWEEN ACHIEVEMENT MOTIVATION MEASURES AND 1963 AND 1964 GRADE POINT AVERAGES FOR 54 MALE STUDENTS OF SAMPLE C . . . . . . . . . . . . CORRELATIONS AND SIGNIFICANCE OF CORRELA- TIONS OF MOTIVATION SCALES AND ACADEMIC ACHIEVEMENT FOR 54 HIGH SCHOOL STUDENTS . . COMPARISONS OF CORRELATIONS OF ACHIEVEMENT MOTIVATION AND INTELLIGENCE AND ACHIEVE- MENT SCORES O O O O O Q O O I O O O O I O O CORRELATIONS BETWEEN AFFILIATION MOTIVATION SCORES AND GRADE POINT AVERAGE AND LORGE- THORNDIKE SCORES FOR FOUR AFFILIATION MOTIVATION TESTS . . . . . . . . . . . . . . ROTATED FACTOR LOADINGS OF FIVE ACHIEVEMENT MOTIVATION AND FOUR.AFFILIATION MOTIVATION SCALESNé-‘lIZooooooooooooooo . Page 40 50 67 69 69 71 72 76 78 80 82 84 Appendix LIST OF APPENDICES Page THE MATRIX OF FACTOR LOADINGS PRODUCED BY A PRINCIPAL AXIS ANAEYSIS OF FIVE N-ACHIEVEMENT AND FOUR N-AFFILIATION TEST SCORES FOR 112 STUDENTS . . . . . . 106 SELF REPORT INVENTORY ......... . . . . . 107 vi CHAPTER I INTRODUCTION TO THE PROBLEM Farquhar,1in his study of motivational factors underlying school achievement, has shown that the M-scales possess what the Technical Recommendations for Psycholog- ical Tests and Diggnostic Techniques2 would call predic- tive validity. Other studies using the M-scalesB’4 have tended to show concurrent, and "factor validity," but as yet no study has been designed to test what ostensibly was the theoretical background for the construction of the test. 1W.W. Farquhar, Motivational Factors Related to Academic Achievement, United States Office of Education Cooperative Research Project No. 846, 1963. - American Psychological Association, Committee on Psychological Tests, Technical.Recommendations for Psycho- logical Tests and Diagnostic Techni ues, Washington, D.C.: APA, 1954. 3David A. Payne, "The Concurrent and Predictive Validity of an Objective Measure of Academic Self-Concept," Educationalfiand Psychological Measugement, 1962.’ 4Marion D. Thorpe, "The Factored Dimensions of an Objective Inventory of Academic Motivation Based on Eleventh Grade Male Over- and Underyachievers." Unpub- lished doctoral dissertation, Midhigan State University, 1961. Many other studies of achievement motivation Gn-ach)* such as the series conducted by Atkinson,1 present evidence for validity of‘flaach tests which ranges from what the Technical Recommendations would call concurrent to what is called construct. Cronbach and Meehl2 and Campbell3 have tended to regard construct validity as perhaps the most important part of test validation efforts, but as yet, the evidence aboutiz-ach scales would not indicate that they all possess this desirable quality. Campbell has indicated that the APA Technical Recommendations regarding content, concurrent, and predic- tiyg validity are not sufficient to cover what is generally considered to be exhaustive evidence of test validity is needed because certain interpretations are made of the test scores. As he states: 1John W. Atkinson, An Introduction to Motivation (D. Van Nostrand and Co., Inc., 1964). 2Lee J. Cronbach and Paul E. Meehl, "Construct Validity in Psychological Tests," PsychologicalfiBulletin, Vol. 52, No. 4, July 1955, 281-302. 3D.T. Campbell, "Recommendations for APA Test Stan- dards Regarding Construct Trait or Discriminant Validity," American Psychologist, Vol. 15, No. 8, August 1960, 546- 553. *The words achievement motivation, and Need-achievement (n-ach) will be used interchangeably throughout this thesis. Affiliation motivation and Z-aff will be simi- larly used. "In the labels given tests, in statements of intent and descriptive material, many ex- plicit and implicit claims are made," and "Test constructors and users as we have known them have generally been prone to reifying and hypostatizing, prone to assume that their tests were tapping dispositional syndromes with other symptoms than those utilized in the test."1 He believes that the implications of such hyposta- tizing should be checked, and for this reason the construct validation of tests is necessary. He distinguishes between two types of construct validity. The first is called trait validity. He says: "It is applicable at that level of devel- opment still typical of most test development efforts, in which "theory," if any, goes no farther than indicating a hypothetical syn- drome, trait, or personality dimension." The second could be called nomological validity and would "represent . . . the possibility of vali- dating tests by using the scores from a test as interpretations of a certain term in a formal theoretical network, and through this, to generate predictions which would be val- idating if confirmed when interpreted as still other operations and scores."3 For example, if the 'fl—ach scores are shown to be correlated with teachers' ratings of "academic achieve- ment needs," then trait validity is shown.* If the Reach 11bid., p. 546. 21bid., p. 547. 31bid., p. 547. *Trait-validity is demonstrated for both measures, thelnpach scale and teachers' ratings. As Campbell states "valida- tion is symmetrical and equalitarian. The presumptive val- ‘I A; 9"? A; Raf-k Panfn 4 n 4 unmaa BAA k aaaaaaaaaa O- " n CAR scores are interpreted as a measure of Drive (D) in the Hullian theory of learning, and correct predictions are made regarding performance in learning situations, then nomological validity would be demonstrated. It is clear, however, that nomological validity may be too great a requirement at this time for all n-ach measures, and that about all which could be expected is evidence of trait validity. Need for the Study Clarification of Constructs It is generally held that the development of theory, the testing of the theory, and the redevelopment of theory is an efficient approach to science. An opposing view held by some others is that science is essentially a des- cription of the relationships among events and that many empirical laws must be found before an overall integration of some of these empirical laws is attempted. Most practicing researchers probably operate be- tween these two modes of attack and develop their theories_ from empirical facts which are not exhaustive of the phenomena in question, and revamp their theories as new research investigates other phenomena encompassed within the constructed theory. The question of constructs (concepts) in science has been of interest to many phi1080phers of science, but there is no general agreement as to how one gives meaning to these constructs. One method has been to define opera- tionally the construct and limit the "meaning" of the construct to the specified set of operations. A strict adherence to this rule would indicate that the "meaning" of intelligence for any investigator is the score on a particular IQ test. Hempel1 specifies that other proper- ties of the concept may be spoken of which may not be directly related to the operations of the operational definition, and the empirical relationships between the concept (in this case some measure or score) and other concepts. For example, the concept of Drive (D) in the Hullian framework may be operationally defined by the number of hours of food deprivation, or the voltage of current on a grid, and these measures might be related to the speed or latency of rat's journey down a runway. Once developed, however, the concept of D may have other properties imputed to it. It could be also thought of as a general tendency to react to cues directly related to the deprivation, and to be unreactive to all others, e.g., to "notice" cues for food when a deprivation schedule for food has been set up, and not to "notice" cues for water, if a deprivation of 1C.G. Hempel, "Fundamentals of Concept Formation in Empirical Science," International Encyclopedia of Unified Science, Vol. II, No. 7, Chicago: University of Chicago Press, 1952. water has not been part of the experiment. These other properties of D would require further experimentation in order to give D a wider meaning than just a tissue deficit related to food, or a general activation related to nox- ious stimulation of the footpads by electricity. These considerations lead to the question of con- cepts and constructs in psychological theory. The con- struct of need or motive has had a long history in psychol— ogy. It has been usually used in the wider sense (not operationally defined) in many theories of personality. Some theorists like Murray1 provide an operational defini- tion of need (scores yielded by a special scoring on the Thematic Apperception Test), other theorists use the terms need and motive in a more loose sense and provide only a verbal network of relationships between needs. There are other issues involved in the meaning of the concepts of need or motive which may have to be re- solved, but these are for the most part the province of the theorists who use the term in divergent senses. It would appear, however, that the two terms, need and motive, are used in much the same way by most researchers. Because the concept of achievement need, and/or achievement motive has recently been the focal point of 1H.A. Murray, Explorations in Personality (New York Oxford Press, 1938). much theoretical and empirical work,1’2’3 it is importantthat the concept and its measurement be subjected to further empirical test. In Farquhar's research, the concept of achievement motivation has been related to a criterion of academic achievement, while McClelland and Atkinson have related their broader concept to recall of tasks, self ratings, academic achievement, production records and economic behavior of societies among other things. As new rela- tionships are found, the concept takes on a broader meaning. It is at this point that the question of construct validity comes in. The question becomes whether measures of7L-ach differentially fit the predicted theoretical re- lationships between need scores and achievement behavior. A second question is whether all the tests of achievement measure an underlying trait, and are thus different from other motivation measures. Various objections to and clarifications of con- struct validity have appeared. The issues discussed re— garding construct validity are generally those of termin- ology and phiIOSOphy of science rather than exhortations 1Farquhar, op. cit. 2D. McClelland, J. Atkinson, et al., The Achieve- ment Motive (New York: Appleton-Century-Crofts, 1953). 3A.L. Edwards, Pergongi Preference Schedule Manual (The Psychological Corporation, 1954). to eliminate requirements subsumed under the topic of con- struct validity. Some writers would abandon the notion of construct validity,1’2 while others would add or change the specific requirements for construct validation. However, a number of attributes of construct validity remain: a. In tests that presumably measure a trait, syn- drome or construct, evidence for the presence of these attributes cannot be gathered from the test alone, but must come from relation- ships with other tests and non-test behavior. b. Test scores should acquire meaning from psy- chological theory (Loevinger:3;Cronbach and Meehlfla),although the level of theory develop- ment may vary tremendously. Some of the state- ments in the theory should lead to predicted relationships among observables. ,c. Construct validation is both convergent and divergent, that is, relationships between test scores and other variables should be high in some cases and low or non-existent in others. d. Constructs may vary from being very close to observation to ones highly removed from obser- vation (intervening variables or hypothetical constructs). 1Robert L. Ebel, "Must All Tests Be Valid?," The American Psychologist, 1961, 640-647. 2Harold Bechtoldt, "Construct Validity: A Critique," American Psychologist, Vol. 14, No. 10, October, 1959, 619‘6290 3Jane Loevinger, "Psychological Tests as Instruments of Psydhological Theory," Psychologicai Reports, 3, 1957, 635-6940 4Cronbach and Meehl, op. cit. Some of the writers consider construct validity as the Sipe qua non of validity while others stress predic- piyp and content validity as sufficient. The single issue pervading most of the discussion is that of in- ferences which we make from test scores. This is evident in the following examples: (a) Cronbach and Meehl's1 dis- cussion of the nomological net, in which tests scores are used as variables scores which are used as terms in the theoretical net; (b) Campbell's2 notion of trait and nomo- logical validity in which test scores are seen as measures of a trait possessed in varying degrees by different indi- viduals; (c) Ebel's3 substitution of meaningfulness for validity in which test score relationships to other measures is an important aspect of meaningfulness; and (d) Loevinger's4 notion of "structural and external com- ponent" in which relations of items to themselves and to total scores and non-test behavior are considered evidence of construct validity. Ibid. 2D.T. Campbell and D.W. Fiske, "Convergent and Dis- criminant Validation by the Multitrait-Multimethod Matrix," Psychological Bulletin, Vol. 56, No. 2, 1959, 81-105. 3Ebel, op. cit. 4Loevinger, op. cit. 10 Problemggf Validation of Testp Whenever a new test is used in a research study, a usual "validational" procedure encountered is the correla- tion of this new test with a whole host of other tests or other "criterion" variables. Occasionally, the terms "construct validity," "predictive validity," are employed as justification for the employment of this new test. Intelligence tests are validated by other intelli- gence tests, scholastic aptitude tests are validated by achievement in high schools and colleges, personality tests are validated by efficient discrimination of groups of persons presumably variant on these personality traits, and so forth. Validation techniques are almost as numerous as the number of tests and are potentially more numerous. What then constitutes adequate validation of psy- chological tests? Every test constructor is aware that his test should possess the qualities of validity and reliability, reli- ability being presumed to be a necessary predecessor of validity. There are types of reliability as well as validity and the propriety of each is determined within the framework of the use of the test. Once reliability of the test is presumed adequate, the efforts of the test constructor are focused on validity 11 considerations. Since the APA Technical Recommendations1 were published, a neat classification of validity has been available to test constructors. Four types of "val- idity" were established with the necessity of each depen- dent upon the type of test and the varieties of inferences which were to be made from each test score. In general, test scores are not seen as important in their own right when construct validity is being con- sidered. Test scores usually represent fundamental enti- ties which are measured in some fashion by a particular test. When content, concurrent and predictive validity are assessed, the test score may be the datum of impor- tance rather than any entity or trait or syndrome. The problem of this study is basically that of establishing what evidence is available regarding the con- struct validity of measures of achievement motivation. Two aspects of the problem could be isolated. First, whether the "construct" of achievement motivation is a valid one, and second, whether measures of achievement motivation have "construct" validity. On the face of it, these questions would appear to be similar; however, different methodologies would be employed to answer the separate aspects. 1American Psychological Association, Committee on Psychological Tests, op. cit. 12 In the first case a theoretical network could be erected in which the term "achievement motive" was used as a term in the axioms or theorems of the theory. If this theory generated empirical relations which were con- firmed, the construct of "achievement motivation" would also be validated. This conceivably could occur even when separate or distinct measures or tests of achievement motivation were used as operational definitions in the research. However, one measure of "achievement motivation" could be more efficient than others in the various predic- tions made by the theory. In this case, one measure possesses greater construct validity than the others and the question of the validity of the construct is assessed concomitantly with the assessment of the construct val- idity of the measure of achievement motivation. The case, however, is not nearly as neat when single tests are considered when they are based on less developed theory or on the hunches and intuitions of the test con- structor. When one measures introversion,}L-ach, self concept, ego, nurturance or any of a whole host of traits, which have more or less of a place in theoretical or nomo- logical nets, the problem is more difficult. Perhaps a number of independent measures relate and are called "achievement motivation" measures. Do these relationships signify the construct validity of the test or the validity of the construct (achievement motivation)? It would appear 13 that evidence for the validity of a theory is also evidence for the constructs embedded in the theory and is mutually confirming of the construct validity of the tests used as measures of these constructs. It is difficult to imagine cases in which "construct validation" of tests in the sense of Campbell and Cronbach and Meehl could proceed without explicit theoretical or empirical statements and as such, construct validation of tests probably coincides with validation of constructs. (For opposing view, see Kausler and Trappl.) This study does not attempt to solve the philoso- phical problems attendant to the use of concepts such as "construct validity." It does, however, extract some minimal relevant requirements for construct validity and apply these to measures of achievement motivation. New Empirical Evidence There are at least three other vacuums that research on achievement motivation could attempt to fill. The first vacuum is caused by the fact that most research on achievement motivation has focused on college students. The attenuation of empirical relationships . 1D.H. Kausler and E.P. Trapp, "Methodological Con- SIderations in the Construct Validation of Drive Oriented Scales," Psychological. Bulletip, 56, 1959, 152-157. 14 exhibited by the studies reviewed in Chapter II is poten- tially related to the relative 7L-ach homogeneity of the samples studied. There is also the second vacuum, caused by the small number of variables used in the research. A slightly larger scale study will provide a more exten- sive empirical study of the relationships among measures of achievement motivations. Finally, the explicit job of differential predic- tion is a void in the liturature. Few studies have checked the differential association of achievement motivation to the variables of aptitude and achievement. Summary of Need for Study The need of this study is assumed to proceed from: a. Lack of clarity of the concept of achieve- ment motivation. b. Lack of validity information concerning Reach measures. c. The lack of evidence concerning the empiri- cal relationship of achievement motivation to other variables, especially at the high school level. The Purpose of the Study The first and main purpose of the study is to pro- vide evidence about convergent and discriminant validity of various achievement motivation scales. The purpose will be therefore to pro- vide evidence for the trait validity of achieve— ment motivation scales. 15 2. A second purpose is to extend the information available regarding the predictive validity of achievement motivation scales. Academic achieve— ment behavior, along with aptitude test responses, will be considered as criterion measures. 3. A third purpose is to explicitly submit motiva- tional scales to the job of prediction. Motiva- tional measures will be compared concerning their ability to predict achievement behavior. Theory The theory which underlies this research study is the need theory of Murray1 and its refinements and redevelop- 3;4;5 ments by McClelland, et al.,2 Atkinson and Farquhar.6 1M . urray, op. Cit. chClelland and Atkinson, et al., 0 . cit. 3Atkinson, op. cit., An Introduction . 4John W. Atkinson (ed.), Motives in Fantasy Action ssd Society, D. Van Nostrand Co., Inc., 1958. 5John W. Atkinson, et al., "The Achievement Motive, Goal Setting and Probability Preferences," Journal of :éDnormal and Social Psychology, pp, No. 1, 1960, 27-36. Farquhar, o . cit. 16 Murray conceived of needs as constructs which stood for forces in the brain region which organize apperception and actions to transform unsatisfying situations in a certain direction. Murray believed that these needs could be aroused from within by internal visceral processes, or by the immediate situation. These needs were manifested by typical behaviors, avoidance and selection of Certain cathected objects, characteristic affect, manifestation of satisfaction with the achievement of a certain effect, and dissatisfaction caused by the failure to achieve these effects. The Thematic Apperception Test was designed to assess these underlying needs. Murray and others devised systems for scoring the content of these stories. Early studies of the TAT demonstrated that fear and hunger af— fected the imagery produced on the TAT. These studies led to the conception of the relationship between motivation and projective responses. McClelland and Atkinson1 developed group methods of administering the TAT and standardized the questions which guided the production of responses by the subject. Analysis of stories given under food deprivation gave sup- port to the belief that thematic apperceptive content was susceptible to motivational influence. Studies of the TAT 1McClelland and Atkinson, et al., Op. Cit. 17 under various conditions -- relaxed, neutral, success, failure, and others -- demonstrated that achievement imagery was related to the type of induced motivational condition. It was found that stories following achievement orienta- tion contained references to (a) performing a task in relation to a standard of excellence; (b) performance of a task socially defined as a unique accomplishment; and (c) pursuit of a long-term goal, the characters of the stories being concerned with "success". These studies provided the foundation for a series of studies which ultimately lead to the formalized conception outlined by Atkinson.1 Although most of McClelland's and Atkinson's investigations focused on the achievement motive, the affiliation motive has also been shown to be related to behavior. Performance under "non-ego involved" condi- tions was positively correlated with affiliation motive scores, and was unrelated to 'n-ach level. These modifications and elaborations of Murray's original need theory lead to tests used in this study. The TAT method and its scoring is outlined in Atkinson.2 The lAtkinson, o . cit., An Intgoduction . . . 2Atkinson, o . cit., Motives in Fantasy . . . . 18 Test of Insight (T01) is described by French.“2 The Edwards Personal Preference Schedule,3 which is scored for affiliation and achievement need, has also a similar theoretical rationale. Farquhar4 conceptualized academic motivation as being a bi-polar dimensional construct which considered ipp motivation as the polar opposites of McClelland's three characteristics of achievement motivation. Thus, (a) meet- ing a minimal standard of excellence, (b) performing tasks which are considered common accomplishment and (c) having short-term goals, became the characteristics describing persons with low academic motivation. Academic motivation became merely a special case of achievement motivation in an academic setting. Atkinson's recent statements5 place the original concept of achievement motivation into a more specific form. He states that the tendency to achieve success (TS) 1Elizabeth G. French and Irene Chadwick. "Some Characteristics of Affiliation Motivation," Journal of Ab- normal and Social Psycholo , Vol. 52, No. 3, 1956, 296-300. 2Elizabeth G. French, "Some Characteristics of Achievement Motivation," Journal of Experimental Psychology, Vol. 50, No. 4, 1955, 232-236. 3Edwards, pp cit. 4Farquhar, o . cit. 5Atkinson, o . cit., An Introduction . . . 19 ("overtly expressed in the direction, magnitude, and per- sistence of achievement and oriented performance") is a product of the joint action of the motive to achieve suc- cess (MS), the strength of the expectancy or probability of success (PS), and the incentive value of success (IS). The equation T5 = M5 X PS X IS describes the inter- relationships of these variables. MS is conceived to be a relatively stable and general characteristic of the person while PS and IS are specifically determined by the experi- ence a person has had in similar performance situations. Atkinson also posits a general motive to avoid failure, this motive being measured by the Test Anxiety Questionnaire and the Manifest Anxiety Scale. The theory underlying these uses of the tests is different than that underlying the development of these anxiety questionnaires. Some relevant implications for this study emerge from Atkinson's theorizing. He states1 that TS should be greatest when the PS is at .50, and should be lower if PS is very high or low. He also states when the apparent dif- ficulty of the tasks is held constant for a group, then the TS should be greater when Ms is strong than when MS is weak. This difference again should be most pronounced when PS is near .50. 20 Also, when there is no incentive to achieve, there will be no basis to predict the performance of persons dif- fering on Ms' These implications are important for this study because some of the experimental comparisons evaluate the relative relationship of achievement motivation scores and affiliation motive scores to academic performance. A positive correlation between achievement motivation and academic performance is reasonable because a. Some tasks in school are likely to have near .50 difficulty -- that is, not all tasks are extremely easy or impossibly difficult, so Ms CR-ach) is likely to be operative in academic behavior. b. PS (perceived probability pf success) is pro- bably unrelated to achievgment motivation (Ms) so analysis of individual probability (PS) for each subject is not necessary. c. Most schools have incentives for academic per- formance; e.g., grades and academic honors. These incentives may, however, be extremely idiosyncratic to each school, but it would seem to be reasonable to assume that many academic tasks have incentive value for most students. If it can be assumed that there is no interaction between PS and IS and MS in the samples studied, then a straightforward prediction about performance in indicated. 21 The theory underlying affiliation motivation is very similar to that of achievement motivation. No formal schema has been provided for it, but some hypotheses regarding its action in performance situation were demonstrated by French and Chadwick.1 Its place in the study is mainly one of control. Other traits could have been chosen, except that three pre- viously investigated instruments yield scores on this variable. The main technique of the present study, the multi- trait-multimethod matrix, was outlined by Campbell and Fiske.2 Its main purpose is to demonstrate that, given a valid test, correlations between certain variables should be high, and correlations between other variables should be low. This matrix led to the major hypotheses of the study, although the variables chosen for inclusion in the study were chosen by the theory outlined above. The Hypopggses of the Study 1. Independent measures of the’Z-ach trait will corre— late positively, and significantly different from zero . 1French and Chadwick, o . cit. 2Campbell and Fiske, op. cip, 22 The correlation of two independent measures of the ‘n—ach trait, will be larger than any correlation of these trait measures and any other trait measured by the same or different methods. Independent measures of the same trait will be more highly correlated than either of these independent measures will be correlated with other traits measured by the same method. The correlations between the self-report achievement motivation leach scores and academic achievement scores will be lower than the other correlations between achievement motivation scores and academic achievement scores. The correlations between achievement motivation scores and academic achievement scores will be higher than the correlation between affiliation motivation scores and academic achievement scores with method being the same. The correlation between achievement motivation scores and intelligence scores will be lower than the cor- relation between achievement motivation scores and academic achievement scores with methods controlled. The correlation between affiliation motivation trait measures and intelligence will be lower than the correlation of affiliation motivation and academic achievement. 23 8. The factor loadings of the'nrachievement tests will be concentrated on a first order factor, with lower loadings on a second factor, a factor which will have high loadings of'nraffiliation tests. Overview of the Study An analysis of the problem of construct validity is made in the first chapter. The problem of construct vali— dity as it relates to’l-achievement measures is similarly treated in the first chapter. The second chapter contains a review of the litera- ture dealing with construct validity, and also contains a review of theory and empirical characteristics of tests of achievement motivation. The design and the analysis of the study are pre- sented in chapters three and four. The fifth chapter con- tains the summary and conclusions of the study. CHAPTER II REVIEW OF THE LITERATURE The literature deemed relevant to this study would appear to separate into four areas. The first section is a review of some of the historical background of the con- struct validity criterion; a second is some examples of putative research of test construct validity; a third sec- tion contains reviews of studies of the reliability of achievement motivation scales; and a fourth area reviews validity studies of achievement motivation scales. It is not intended to be an exhaustive overview of research of achievement motivation, but is designed to place construct validity in a historical perspective, and show results of representative studies using measures of achievement moti— vation, especially as they relate to academic achievement behavior. Types of Validity In the history of psychological testing, only re- cently have exhortations been made by people concerned with test construction that test publishers should show evidence of construct validity when claims of test validity are made. 24 25 One of the earliest expressions of this concern was the APA Manual, "Technical Recommendations for Psychological Tests and Diagnostic Techniques".1 Contained in this publication is one of the first elaborations of the term, construct validity. Its definition of construct validity is best understood when compared to its definitions of the other types of validity. Content Validity The manual states "Content validity is evaluated by showing how well the content of the test samples the class of situations or subject matter about which conclusions are to be drawn." (p. 13) In general, decisions about content validity are made by judging the correspondence between the test behavior and the behavior which the test presumably samples. Decisions about many educational tests are more easily made than are de— cisions regarding unstructured personality tests (such as the Rorschach or TAT), and in the latter case, one makes some assumptions about the relationships between the two sets of behaviors (e.g., in the TAT the stories told in the pictures are assumed to be derived from unconscious needs, but some relationship has to be postulated between verbal reports and underlying dynamics). 1American Psychological Association, Committee on Psychological Tests, op. cit. 26 In work samples, arithmetic and vocabulary tests, content validity is more easily evaluated since the test behavior is in some cases replicas of the situations to which the conclusions are made. Predictive Validity "Predictive validity is evaluated by show- ing how well predictions made from the test are confirmed by evidence gathered at some subse- quent time. The most common means of checking predictive validity is correlating test scores. with a subsequent criterion measure." (p. 13) When "validity" coefficients are usually reported they are of the predictive validity sort, or the concurrent validity sort which will be explained below. Concurrent Validity "Concurrent validity is evaluated by show- ing how well test scores correspond to measures of concurrent criterion performance or status. Studies which determine whether a test dis— criminates between presently identifiable groups are concerned with concurrent validity." (p. 14)2 The predictive validity notion and the first aspect of the concurrent validity notion are practically identical. The decision between whether one or the other is being exhibited would depend on whether the two behaviors are seen as synonymous or not. Presumably, when the test per- formance precedes criterion performance, predictive validity Ibid., p. 13. Ibid., p. 14. 27 is shown, and concurrent validity is shown when both measures are taken at the same time. The second aspect of concurrent validity seems to be slightly different from the first -- namely, the discrim- ination between presently identifiable groups, or the cor- respondence between current status and test performance. This raises an interesting question regarding the usual conception of concurrent validity. Since inclusion in iden- tifiably different groups is presumably based on prior behavior (schizophrenics and manic depressives exhibited their symptomatology before diagnosis and inclusion), and the discrimination is ostensibly about presently constituted groups, it, nonetheless, really has "predicted" behavior prior to the test behavior. Another example might be the "prediction" of cumulative grade point averages, which in some cases are constituted by behaviors prior to the test- ing period. The Technicai Recommendations have no category for such "predictions" since "predictions after the fact" are usually an anathema to the scientific investigator. Another category, however, could be evaluated which could be called "postdictive validity," which includes aspects and considerations such as the above. Construct Validity The Technigai Recommendations state "Construct validity is evaluated by inves- tigating what psychological qualities a test 28 measures, i.e., by demonstrating that certain explanatory constructs account to some degree for performance on the test. To examine con- struct validity requires both logical and empirical attack. Essentially, in studies of construct validity, we are evaluating the theory underlying the test. The validation procedure involves two steps. First, the in- vestigator inquires: From this theory, what predictions would we make regarding the varia- tion of scores from person to person or occasion to occasion? Second, he gathers data to con- firm these predictions." (p. 14)1 Following closely the statement of the Technical Recommendations, Cronbach and Meehl attempted to clarify the term construct validity. A construct, they said, "is some postulated attribute of people, assumed to be reflected in test performance. In test validation the attribute about which we make statements in interpreting the test is a con- struct."2 According to Cronbach and Meehl, these attributes have occasionally inadequate criteria and their meaning is inherent in the network of associations or propositions in which the construct occurs. Validation of the construct or construct validation occurs only when the statements in the network lead to predicted relationships among observable events. They go on to state that construct validity is examined under many types of evidence including: Ibid. 2Cronbach and Meehl, op. cit. 29 a. content validity b. inter-item correlations c. inter-test correlations d. test-criterion correlation e. stability f. stability under experimental intervention Excessively high correlations and stability may con- stitute confirming or negative (disaffirming) evidence. When predicted relationships do not occur, the fault is either in the network of laws or in the construct. Cron- bach and Meehl also conclude that construct validity cannot generally be expressed in the form of a single coefficient. Variance attributable to the construct can however be roughly estimated. They see construct validation as not essentially different from the scientific enterprise of theory development and confirmation. The essence of Cronbach and Meehl's definition is its requirement that the construct be systematically in— vestigated in theories, and predicted relationships are the evidence for the acceptance of a construct. Jessor and Hammond1 made similar recommendations using manifest anxiety as a measure of D (drive) in Hull's mathematical learning theory. 1R. Jessor and K.R. Hammond, "Construct Validity and‘ the Taylor Anxiety Scale," Psychological Bulletin, Vol. 54, 1957, pp. 161-170. 30 At about the same time, Loevinger1 attempted to show how test construction itself is a procedure which has important relationships to construct validation. She dis- tinguishes three types of validity: a. substantive b. structural c. external Her substantive category is closely akin to the content validity and external validity encompasses much of what is included within predictive and concurrent categories of the technical recommendations. Of construct validity, she says, "The basic concept is that of the construct validity of the test, the degree to which it measures some trait which really exists in some sense.. Construct validity can only be estab— lished by convergence of several lines of evidence. Evidence for construct validity can be broken down into evidence that the test measures something systematic, and evidence for the particular interpretation of what it measures." Loevinger concurs with the requirements of the Technical Recommendations and seems to add some unique facets. Her structural validity category includes evidence similar to the Technical Recommendations and adds factoral structure and homogeneity of items as new evidence. The content and reliability of the test are also stressed more 1Loevinger, op. cit., p. 685. Ibid. 31 than in the Technigal Recgmmendations or in Cronbach and Meehl.1 Although she is highly interested in the nature of items and their interrelationships, she says, "External correlations provide the court of last appeal."2 Loevinger's3 requirements for validity are sweeping and extensive. She raises an issue which deserves further treatment, i.e., whether evidence for the validity of a test is evidence for the validity of the construct. This issue is similarly raised by Kausler and Trapp insofar as manifest anxiety and "D" are concerned. They maintain that both processes cannot coexist. In their example, the use of the Manifest Anxiety Scale (MAS) was criticized as a measure of D, and that validation of the construct D is more adequate if experimentally induced D states were used as a measure of D. A fallacy here is that experimentally induced D states are no more of a final criterion for D than is the Manifest Anxiety Scale. In the second experiment using ex- perimentally induced drive states as a measure of D, they are providing another line of evidence for the validity of the construct D, and for the measure (experimentally induced states). In the experiment, using MAS as a measure of D, both the construct D and the MAS were being validated. The 1Cronbach and Meehl, op. cit. 2Loevinger, o . cit., p. 675. 31bid., p. 659. 32 issue raised by Kausler and Trapp is really whether experi- mentally induced positive drive states (or some other opera- tional definition of D) or MAS stand in a more preeminent position as a measure of D. Recently other criticisms and clarifications of con- struct validity have emerged. Bechtoldt1 criticized the concept of construct validity because the category allowed private interpretations of test scores and behavior and lead to unnecessary confusion attendant to the concept. He pro- posed a return to an "operational methodology" which re- garded test scores as Operational definitions. He states: "What is being questioned is the tendency to consider as part of public empirical science hunches involving 'vague' ill defined variables and relations between such vari- ables, these hunches being derived primarily from the observed performance." He goes on to criticize the circularity inherent in construct elaboration which uses the construct to explain performance after deriving the construct from observed per— formance. Campbell3 answered many of Bechtoldt's objections and proposed new distinctions. He states that the pre- dictive and concurrent aspects might be better called "practical validity." These occur primarily when there lBechtoldt, op. cit. 2Ibid. 3Campbell, op. cit. 33 are "criteria" against which the tests might be judged. Campbell1 points out, however, that "not all psychological tests have been designed solely to predict performance against extant institutional decisions, situations. There are relatively few settings which produce such criteria." The scientist is however more interested in single factored traits "for which society produces no correspondingly pure criteria." He pointed out that "Cronbach and Meehl, Jessor and Hammond, have tended to tie construct validity to tests developed and validated in the context of explicit theorem tical structures or 'nomological nets.'"2 Such developed theory is usually lacking in test validation efforts. He would therefore divide construct validity into two types. Trait Validity "is applicable at that level of development still typical of most test develop- ment efforts in which 'theory,' if any, goes no further than indicating a hypothetical syn- drome, trait, or personality dimension."3 Nomological Validity "would represent the very important and novel emphasis of Cronbach and Meehl on the possibility of validating tests 11bid., p. 547. 2Ibid., p. 627. 31bid., p. 547. 34 by using the scores from the tests as interpre- tations of a certain term in a formal theoretical network and through this to generate predictions which would be validating if confirmed when in- terpreted as still other operations and scores."1 Somewhat earlier Campbell and Fiske2 advocated the use of the multitrait-multimethod matrix as a method of test validation. Their method provides evidence for convergent and discriminant validity. (Both incidentally are also pgedigtive, concurrent, practical and trait validity pre- viously described.) Discriminantvelidity is demonstrated when correlations between independent methods measuring different traits are consistent with prior knowledge concerning the relationship between these different traits. The relationships may be positive or negative, but should in any case be quite low. Ebel3 was also concerned with the notion of validity. He points out correctly that validity has a good deal of confusion attendant to its use in the literature. This leads to a suggestion that a word "meaningfulness" be substituted for validity and the requirements for meaningfulness are outlined. It is not clear to this reviewer how these Ibid. 2Campbell and Fiske, op. cit. 3Ebe1, op. cit. 35 requirements differ from those under the separate rubric of content, concurrent, predictive and construct validity. Summary pf Construct Validity In some ways, validity now means very little more than what Cureton1 proposed earlier. New terms have sup- planted and been superimposed on older terms, but operational requirements for validity are substantially similar to those of previous years. Correlations of test scores with other scores constitute the main method of test validation with logical and statistical analysis of test content serving a subsidiary role. The theoretical efforts preceding the computation of the correlations, the types of scores which are correlated, and the interpretation of the coefficients are the main differentiators of the different types of validity. Recently, other validational techniques have come increasingly into vogue. Factor analysis is one prominent example. Guttman's facet analysis is another. For the most part, a factor analysis of a test or a battery of tests gives similar information as the MT-MM Matrix, although prior factor structure is not usually hypothesized as are the interrelationships of the MM-MT Matrix. Guttman's facet analysis probably functions both as a test generator 1Edward Cureton, "Validity," in E.F. Lindquist, Educasionai Measurement, ACE, Washington, 1951, 621-692. 36 and construct validator. Its heavy emphasis on the use of theory to generate tests would seem to be highly desirable. Even with the general use of factor techniques, there have been few, if any, rationalizations of this method into the hypothethxrdeductive methods of science, and it remains an exploratory (inductive) method of concept development. Guilford and Eysench (see Cronbach and Meehl, reference 16) present factor analysis as a method of test and construct validation. And Cronbach and Meehl also support the use of factor analysis in construct validity although it does not fit their more restrictive requirements. In general, these methods are used in early stages of construct development. Examples of Construct Validation Studies in the Psychological Literature ‘ A few studies selected from the psychological liter- ature were reviewed in order to determine the variety and scope of investigations which purport to analyze the con- struct validity of tests, and/or the validity of constructs. A study of Phares and Adams1 examined the validity of the EPPS heterosexuality scale. Extreme groups of males on the Heterosexuality Scales ranked "sexual" and "nonsexual" pictures which had been previously equated esthetically. 1E.J. Phares and C. Adams, "The Construct Validity of the Edwards PPS Heterosexuality Scale," Journal of Con— sultingyPsycholo , id, 1961, 341-344. 37 The high scorers on the Heterosexuality Scale ranked sexual pictures higher than the low scorers. A high group, when exposed to sexual communication, learned and retained the material more easily than a low group. The second hypoth- eses received statistical support however only after elimi- nating subjects. Bernardin and Jessor1 used two EPPS scales to define "dependent" and "independent" subjects. The scales used were the Autonomy and Deference scales. Percentile separa— tions on the two scales of 30 points with 70th and 50th percentiles as cut-off points was the mode of classifica- tion. Three hypotheses were examined. The hypothesis that dependent persons would perform a finger maze mask less efficiently under negative verbal reinforcement, and the hypothesis that dependent persons will seek more help in a problematical situation (when assistance is available on request) were confirmed. However, a third hypothesis that dependent groups would conform more often to group judgments than would independent groups was not supported. Zuckerman, pp_di.,2 factor analyzed various measure— ments and ratings of dependency for 72 student nurses. They 1A. C. Bernardin and R. Jessor, "A Construct Valida- tion of the Edward's Personal Preference Schedule with Respect to Dependency, Jougnai of Consuiping Psychology, 1;, 1957, 63-67 0 2M. Zuckerman, et al., "Concurrent and Construct Validity of Direct and Indirect Measuring Dependency," Journal 0: Consulting Psycholo , Vol 2 No.4, 1961, 316-323 0 38 found low relationships between projective measures of de- pendency and self and peer ratings. However, the EPPS and a sentence completion test were positively related to self and peer ratings. The factor analysis was judged by the authors to be the relevant analysis of "construct validity." The first two studies indicate that the construct validity of tests is demonstrated when high and low scorers on a particular trait behave appropriately to what is known about the trait. The theory is the trait description. The second study1 used two scale scores to define a third trait called dependency. The confirmation cu? hyptheses regarding dependency presumably lent.evidence for the construct validity of the original measures (autonomy and deference). The third study2 assessed construct validity with factor analysis, with the exact validity evidence being unspecified. Apparently, then, construct validation procedures do not have a common base, and perhaps all of the usual validity evidence is also construct validity evidence (a position similar to that of LoevingerB). 1Bernardin and Jessor, op. cit. 2 Zuckerman, et al., 0 . cit. 3Loevinger, o . cit. 39 Two studies which explore the validity problems similarly to that of the MT-MM matrix are those of Dicken1 and Hillsz. The first study assesses the validity of the California Psychological Inventory. Two methods (the CPI and Staff Ratings) were used to measure five traits (domin- ance, responsibility, impulsiveness, intellectual competence, and rigidity). The convergent criterion was met satisfactor- ily by 4 of 5 CPI variables. However, only 2 CPI variables met the discriminant criterion, i.e., certain CPI variables correlated too highly with other variables. Hills' study while not purporting to examine convergent and discriminant validity, does provide the raw material for such a matrix. See Table 1. He attempted to modify the general notion of McClelland regardingvt-ach into four kinds of achievement motivation. Specially selected pictures and questionnaires were designed to tap these four kinds of achievement motiva- tion. There were economic, social, academic, and profes- sional. These methods are labelled I and II respectively. Method III are two other sets of pictures previously used to measure achievement motivation. 1Charles F. Dicken, "Convergent and Discriminant Validity of the California Psychological Inventory," Educa— tional and Psychological Measurement, id, 3, 1963, 449-459. 2John R. Hills, "Needs for Achievement, Aspirations, and College Criteria," Journal of Educational Psychology, 3i, 1, 1958, 156-161. 4O mm. om. mo.- om. so. No. Na. oo. no. oo. oo. cs. Ho. oo. on. mm. mm. an. mo. em. ca. No.- so.- so.- so. ma. mo. co. oH.- so. as. as. as. mm. mm. oo. No. on. om. mm. as. No.- No. oo. so.- No.- ma. on. oo.- a w No No mm N< Ho Ho Ho HHH oozes: HH oomemz H noses: m HHH cocoa: ouduuam a H , coasts E E ilHHHi mqm>mq ZOHH¢mHmm< Qz< mnmuz Hzmzu>mHmU< mo MQDHm .mqum 26mm auumhoum m nuHB mmpmuu anopmo< mo coauoapmum o£H= .wuadouwm .z kunom .ono .mo .Heohooez .uHo .mo .umzdoumm cam NuaonESHM .mmwlamm .oaeem WONG) .ooe-ame .omoa .mm .socosoaooo Amoco: no Hocusoo cooeuos< =.couoehco ooohooom Saamucmz can HmEhoz Ga o>auoz ucoEo>oHno< 639: .mEumnoon .m can ampuoh .m.Hd .uau .mo .xoonwoouw can uoEHosuuoz .mmflozm .uao .mo .mqonmoum can newsflass N .mmmldmd .wapaomH mo. u u c How owoeaoo owe ooouuom om. ”on. u u e was memos Hossooem oo. u h = Baa owoaaoo HmH “enthuse: em. u h e eoq z Houmwaumo>aH QUZHHOE Hzmzm>mHmU¢ OZHmD mmHQDHm m0 >m<223m N.N mam. cit. 3 . Jordan and DeCharms, op. c1t. 4Lawrence W. Littig and Constantine A. Yeracaris, "Academic Achievement Correlates of Achievement and Affilia- tion Motivation," Journal of P3 cholo , £5, No. 1, 1963, 115-119. 5 Shaw, 0 . cit. to the achiever-underachiever classification. The TAT and the EPPS were in the predicted direction, but the differ- ences were not statistically significantly different. Two other studies reported negative relationships between achievement motivation and achievement behavior. Cole, §£_§i.,1 divided their studies into over- and under-achiever samples using students who fell 1 S.D. above or below the regression line. This produced 8 over- and 13 under- achievers. A Wilcoxen matched pairs test produced a sta- tistic with a .06 probability, indicating differences higher in achievement motivation (TAT) in the underachieving sample. They attributed the differences between McClelland's and their findings to be the non-life-like situation in the laboratory studies. Somewhat earlier, Broverman, sp_di.,2 found that high "strivers" produced significantly more achievement fan- tasy themes than did low strivers. The size of their sample was rather small (74: 34) for the confidence of their pro- posals. They contended that their study supports the "alter- native channels" interpretation of the expression of 1D. Cole, et al., "The Relation of Achievement Imagery Scores to Academic Performance," Journal of Abnormal and Social Psychology, Vol. 65, No. 3, 1962, 208-211. 2Donald M. Broverman, Edward J. Jordan and Leslie Phillips, "Achievement Motivation in Fantasy and Behavior," Journal of Abnormal and Social Psychology, d9, No. 3, 1960, 374-378. 53 achievement motivation, which states that achievement fantasy may serve as a substitute channel for the expression of achievement motivation when this expression is blocked. Summary of Ach Motivation - Academic Performance Studies In summary, achievement motivation seems to be quite unrelated to the academic performance criterion. Two criti- cisms of the studies appear to be in order. For the most part the samples studied were very small. 72's below 100 are more likely than not. Secondly, most of the studies have employed homogeneous groups of subjects. Nearly all of the studies reviewed were college samples and some within narrow professional areas. Had studies been conducted which had less restriction in the range of ability and achievement, greater relationships may have been obtained. Validity Studies of the M-Scales Four studies were conducted using the M-Scales to predict academic achievement. Green} in a study of 104 male euxi 129 female selected high school students,found rather high correlations between the subtests of the M-Scales and Grade Point Average. The M-Scale-aptitude test score correlation was also extremely high. 1Robert Lee Green, "The Predictive Efficiency and Fac- tored Dimensions of the Michigan State M-Scales for Eleventh Grade Negro Students, An Exploratory Study," Thesis for the Degree of Ph.D., Michigan State University, 1962. 54 Kipfmueller1 also found high correlations between the sub-tests of the M-Scales and Grade Point Average for a sample of 200 randomly selected parochial high school students. The correlations of the M-Scales to an aptitude measure were substantial, but generally lower than the M- Scale-GPA correlation. For males, however, the correlations between HIT and PTCS and GPA were lower than HTI, PTCS and an aptitude score. Johnson2 found lower correlations gen- erally between all subtests and GPA and aptitude. HTI and PTCS also were more highly correlated with aptitude than with GPA. In published data by Farquhar? similar correla- tions between HTI and PTCS and both aptitude and GPA were obtained. Total M-Scale scores were found to correlate highly with GPA by Farquhar (r = .40, 48 females; r = .56, r = .49 males), but the correlations between M-Scales total scores and APT are similarly quite high (r = .30, r = .43 females; 1Mark K. Kipfmueller, "The Predictability and Fac- tored Dimensions of the M-Scales for Eleventh Grade Paro- chial School Students," unpublished doctoral dissertation, Michigan State University, 1963. 2Van C. Johnson, "An Assessment of the Motivation Factor in the Estimation of Academic Achievement of Eleventh Grade Indian Students and the Factored Dimensions of the M-Scales. An Exploratory Study. An unpublished doctoral dissertation, Michigan State University, 1963. 3Farquhar, o . cit. r = .45, r = .50 males). So, in at least one of the compari- sons, the correlation of motivation scales and aptitude is higher than the correlation of motivation scales and achieve— ment behavior, a situation clearly opposite to the theory surrounding the three trait measures. The reason for these high mutual relationships pro- bably stems from the method of test development, in which discrimination between "over- and under-achievers" was the criterion for item selection. Had over- and under-achievers been matChed on aptitude such high correlations with aptitude would probably not have emerged. Unpublished data collected under the direction of Farquhar indicated that in another sample (2.: 179 males, 189 females) the correlation between M-Scales and an apti- tude score was r = .25, r = .45 for males and females res- pectively, while the correlation between M-Scales and GPA was r = .52 and r = .54 for these same groups; this rela- tionship fitting more closely the theory regarding achieve- ment motivation, aptitudes and achievement behavior. CHAPTER III DESIGN OF THE STUDY Sample The theoretical population of individuals from which the sample was selected was 11th and 12th grade high school male students. After contacting a number of schools (five) three agreed to participate in the study. The following samples resulted from these contacts. Sample A: 22 11th grade male students from a medium sized town. Sample B: 30 11th grade male students from a rural consolidated school district. Sample C: 60 students (all 11th and 12th grade males) from a small town. The major portion of the research problem did not seem to indicate that representatives of sample was a par- ticular requisite. What was necessary was a population with sufficient heterogeneity so that the trait measures would have sufficient variance. Instrumentation One standardized commercial test was used (EPPS) while the others, although being used for research studies, were not commercial. The M-Scales, TAT, TOI and a specially constructed self-report inventory were also administered. 56 57 The M-Scales -- The battery is composed of four subtests. They are: The Generalized Situational Choice Inventory. A forced choice instrument which elicites choices between situational events which were designed to compare the two polar dimensions of academic motivation. The Preferred Job Charscteristics Scais. This is also a forced choice instrument with types of jobs being the stimulus material. The Word Rating List. This scale is composed of a series of adjectives to which the subject responds as to their perception of teachers' beliefs about them. The range of applicability is from "always" to "never." The Human Trait Inventory. This scale is made of a series of statements describing various human behaviors. The subject again rates these behaviors on a four point scale as to their applicability to him (or her). The Thematic Apperception Test (TAT) -- Six TAT-type pictures were used in the present study. These pictures were made from slides purchased from the American Documentation Institute and are pictures of Atkinson's list.1 The verbal A‘— 1Atkinson (ed.), op. cit., Motives in Fantasy . . . 58 Number 4 Cub reporter scene 25 Boy watching large airliner 26 Foreman and worker standing near machine 33 Boy smiling at desk at home 88 A group of youngmen seated, with one young man standing 97 A chemist in a white coat productions made to the pictures are assumed to contain ex- pressions of need for achievement and need-affiliation. They are scored according to Atkinson's1 scoring manual. The French Test of Insight (TOL) -- The T01 is a series of ten statements about human behavior. The subject's task is to explain the behaVior. Twenty minutes is allowed for the completion of the ten items. The test is scored similarly to the TAT, and also yeilds both a l—ach and a n-aff score. Both the TOI and the TAT were scored by James Mullin, an advanced doctoral student in Psychology. The Edward's Personal Preference Schedule (EPPS) -- The EPPS is a series of items in which the person chooses one of two paired statements. Each of the pair is a state- ment referring to one of 15 needs. Permission was secured from the Psychological Corporation to reproduce 56 of the 59 items,* all of which include either affiliation achieve- ment need items, or both. It is an ipsative scale, and high scores on one trait necessarily imply somewhat lower than maximum possible on all other traits. Of the 56 items, four couple achievement and affiliation items. This ipsativity was partially reduced by eliminat- ing two items on each scale which had statements of both needs. The TAT, T01, and EPPS yield both affiliation and motivation scores, while the M-Scales yields only a motiva- tion score. One otherJZ-ach measure was also collected for the study. A self-rating inventory contained a rephrased des- cription of affiliation and achievement needs and asked the subjects to rate themselves from O - 100 on these trait des- criptions. This scale is reproduced in the Appendix. Three other measurements were collected on Sample C. 1963 and 1964 grades, Lorge-Thorndike intelligence tests and retests of the M-Scales after a nine-week interval were supplied for students in this sample. *Since there is some evidence that item responses obtained to selected items isolated from the context of the Edwards Personal Preference Schedule are not comparable to those obtained within the context, the results of this research cannot be considered applicable to the standardized complete form of the EPPS. 60 Statistical Hypotheses For clarity in the following discussion, the variables contained in the hypotheses will be numbered. TAT n—achievement score TAT n-affiliation score TOI )‘L-achievement score TOI ‘fZ-affiliation score EPPS n-achievement score EPPS fl-affiliation score Self-report‘nwachievement score Self-report Reaffiliation score M-Scales Total score GPA - 1963 average GPA - 1964 average Lorge-Thorndike IQ score v—u—u—a Nt—IOOOOVChUl-DCJNr—a The following hypotheses-proceed from a general assump- tion that, if a trait measure has construct validity, then certain empirical relationships between that trait measure and other trait measures obtain. A research procedure which erects null hypotheses and confirms the research hypotheses by rejecting these null hypotheses is not clearly appropriate for all the research hypotheses. A special note about the purpose of the study may be helpful in interpretation of the hypotheses. The study's main intent is to study the validity of the achievement motivation construct. As such, no one measure of achieve- ment motivation occupies a preeminent position as the criterion measure. All the tests are presumptive measures of the same trait. (While all the test developers may not consider this to be true, it is curious that they all build their theoretical rationale from Murray and McClelland.) Thus, a correlation exhibited within the matrix may provide 61 mutual evidence for the validity of two tests or the inval- idity of both tests. The question of what is evidence for that is complicated by the fact that: (1) one particular trait measure may have a longer and more glorious history than others; and (2) the matrix considered independently of prior research provides both evidence for the validity of the construct, and the construct validity of the trait measures. The question really being, is the trait measure in question, or is the construct in question. The possible outcomes of this study could shed light on both issues. There is also a question about the role of affilia- tion motivation. While it is not the central focus, the research bears similarly on its construct validity. It was chosen as a contrasting or comparable trait because some of the trait measures of achievement motivation also yielded scores on this trait. All the evidence considered together provides the basis for the judgment of construct validity of trait measures and validity of the construct. Rejection of the null hypothesis may not be sufficient evidence for this judgment; therefore, the following hypotheses are stated in the directional form. Statement of Hypotheses H: 1. Independent measures of the l-ach trait will correlate positively, and significantly dif- ferent from zero. ‘ A. Symbolically 62 H: -achievement r13, 435, r57, 479, r15, r37’ r59: r17, r39, r19) 0 and H; -aff111ation r24: r46’ r68’ r26, r48’ r28 > 0 The correlation of two independent measures of the -ach trait, will be larger than any correlation of these trait measures and any other trait measured by the same or different mentods. Symbolically A r13) rift, r16’ r18’ rig, I"36’ r38 B r35>r&_5_, r25, r_§, r38, r23: r58 C r57>r47, r27, r_7_, rss’ r25: r45 D r79>r69’ r49, r29, rss’ r27, r47: r67 E r15>r14, r18’ r45, rig, rgi, r58 r37>rfl’ r67’ r27: r36, r3§’ r34 G r59'>r89’ r99} r29, r58’ r45: r25 H r17)r14’ r16’ r18’ r27: r47, r67 r39)]:38’1336’ 1' 9’ r49, r69’ r89’ r32 r19>r22, r49, -899 r10. r1r. r., H: 6. 63 Independent measures of the same trait will be more highly correlated than either of these independent measures will be correlated with other traits measured by the same method.. Symbolically 1 r13>r12 and r34 2 r15') r12 3 r56 r17) r12 ; r78 r19) r12 r35) r34’ r56 r37>r3cn r723 r39)"34 r57) r56 3 r78 r59) r56 10 r79)r78 The correlations between the self-report -ach scores and academic achievement scores will be lower than the other correlations between achievement motivation scores, measures and academic achievement scores. \OOONO‘tUl-l-‘bo ° ° ° and r r7 11’(r3 11’ r5 11’1‘9 11 111 The correlations between achievement motiva- tion scores and academic achievement scores will be higher than the correlation between affiliation motivation scores and academic achievement scores with method being the same. 1 11 > r2 11 > r3 11 > 1‘4 11 > r5 11 > r6 11 > r7 11 > r8 11 > The correlation between achievement motivation scores and intelligence scores will be lower 64 than the correlation between achievement motivation scores and academic achieve-— ment scores with methods controlled. r1 11>r1 12 > r 3 11)r2 12 > r5 11>r5 12 > r7 11)r7 12 > r9 11)r9 12 > H: 7. The correlation between affiliation motiva- tion trait measures and intelligence will be lower than the correlation of affiliation motivation and academic achievement. r2 11)r2 12 ) r4 11>r4 12 > r6 11)ro 12 > r8 11>r8 12 ) H: 8. The factor loadings of the -achievement tests will be concentrated on a first order factor, with lower loadings on a second factor, a factor which will have high loadings of7l-affili- ation tests. Analysis Procedures Hypothesis 1 will be tested by comparing the obtained correlations between independent measures of achievement moti- vation (hereafter known as the validity coefficients), and their sampling distribution given no relationship between these measures. The statistic used will be _ ___£_____ 1 t_ 2 1'2 1- r d.f. =1- 2 1Paul Blommers and E.P. Lindquist, Elementary Sta- t'1' or; n91 M...-t....:.. u-..._1.._.... M:££1.2.. n- 13--..-- 10am AC1 65 Hypotheses 2 and 3 were examined by counting the reversals intfluepredicted rankings of the correlation co- efficients. The binomial test was applied to the number of predicted relationship with the hypothesis that the probability of a reversal was = .50. Hypotheses 4 and 5 were tested by examining the differences in the coefficients with a "t" test for the dif- ferences between correlation coefficients obtained on the same sample. 1H.M. Walker and J. Lev, Statistical inference, Henry Holt & Co., New York, 1953. The formula proposed by Walker and Lev is: (11- 3) (1 + r r12 ‘ r13 23) t = 2 2 2 )2(1 ’ r12) ' r13 ' r23 + 2r12r13r23 CHAPTER IV ANALYSIS OF RESULTS Convspgent and Discriminant Validity The first three hypotheses were attempts to examine the convergent and divergent validity requirements outlined by Campbell and Fiske in their discussion of the multi- trait-multimethod matrix. The subjects used were the total membership of samples A, B, and C. Table 4.1 con- tains the intercorrelations of all the ‘R-achievement and lwaffiliation test scores for 112 male high school students. This matrix provides the raw data for the assessment of the first three hypotheses. The first hypothesis states that the intercorrela- tions of theIZ-achievement scales will be positive and will be statistically significant. This is called the convergent validity requirement and is the most basic of all the dif- ferent validity requirements. It stems from the assumption that traits can be measured in independent ways and that scores yielded by different methods of measurement should relate in a non-chance fashion. A similar rationale is appropriate for the X-affi- liation tests, and date is also presented for the assessment 66 67 No.oe mo.oo coo-z u o moeoom-z n m mm.me ee.oe coo-z u o oN.oH oo.HN coe-z u N ooooom show u o No.m Hm.NH ooo-z u o NN.m mm.mH coe-z u m meme u o oo.e om.N one-z n e om.N NN.N coe-z u m Hoe n o om.N em.e hoe-z n N oe.o Ne.o com-z u H Baa u a coauma>on pudendum new: mannanm> Ccowoq mso.- Non. eNo.- mNe. eeo. NeN. NNo.- ooa. o m Nee. NeN. mmm.- NNo. ooo. can. woo. m o oNH.- oom. Noo. ANN. HHH.- ems. N NNm.- emo. mNH.- Noe. eao.- o o oHH.- ooh. oma.- mos. m omo.- omo.- oNo.- e m eon. eel. m ooN.- N a cocoo: o N o m e m N H manmaum> mHZMQDHm AOOEUm mUHm MAmHmomn QM¢QZ mHzmHUHmmmOU QMZHHHQHA¢> mo (-.289, -.322) H: 3 r17 r12, r78 (.156) > (-.289, —.113) H: 4 r19 r12 (.190) ) (-.289) H: 5 r35 r34, r56 (.168) > (-.056, -.322) H: 6 r37 . r34, r78 (.237) ) (-.056, .113) H: 7 r39 r34 (6.242) ) (-.056) H: 8 r57 r56, r78 (.396) ‘> (-.322, .113) H: 9 r59 r56 (.425) > (—.322) H: 10 r79 r78 (.597) > (.113) Legend: 1 = TAT N-ach 6 = EPPS N-aff 2 = TAT N-aff 7 = Self-Report N-ach 3 = TOI N-ach 8 = Self-Report N-aff 4 = TOI N-aff 9 = M-Scales 5 = EPPS N-ach If the probability of a correlation being a reverse of one predicted is .5, then the probability associated with the outcome is P = .227 (with signs of coefficients not being considered). Discussion The convergent and discriminant validity of [-achieve- ment scores were examined in Hypotheses 1,2, and 3. The 73 convergent validity requirement is met in a moderate sense, although all of the coefficients are of moderate size. They are, however, all positive and seven of the ten co- efficients are statistically higher than zero. The coeffi- cients involving the projective tests are lower than those involving the more objective measurements. The convergent validity of the fl-affiliation is clearly inferior to the l-achievement tests. Not only are the coefficients lower but one is negative and one is sta- tistically significant. The discriminant validity require- ment is also met to a substantial degree. Reversals in the predicted patterns of intercorrelations occur when strong negative relationships between I-affiliation and machieve- ment emerge. I A number of factors may be responsible for the resul- tant pattern of inter-correlations. The discrimination of the projective tests at this level severely restricts the variance of the motive scores. Many zero scores were re- corded for both the TAT X-affiliation and the TOI fl-achieve- ment and fl-affiliation. This major restriction of variance was a likely factor in the size of the projective test rela- tionships. Unwanted response sets and methods variance also appears to be present. The negative correlations between l-affiliation and fl-achievement are probably partially a function of methods variance, especially when these traits 74 are measured by the same method. To what extent a true cor- relation exists between these two traits is not known. Since there are negative relationships (for example, the EPPS n:ach, Self-Report nraffiliation r = .353) independent of method, it is likely that there is an actual relationship between the two traits. Methods factors may function differently in the various tests. In the projective tests the elicitation of scorable achievement imagery by the picture may mean that scorable affiliation imagery output is depressed.1 This would be especially true if timing severely limits the quantity of output. On the other hand, verbal fluency may cause the scorable imagery for both traits to covary. The first explanation would seem to be more appropriate for the TAT. The forced choice nature of the EPPS with the item overlap increase the likelihood of a spurious negative rela- tionship. The so-called ipsative nature of the EPPS was examined by rescoring the EPPS eliminating two overlapping items (items which matched EPPS n-ach and 7L-affiliation). The intercorrelation changed from r = .37 to r = -.32. Eliminating all item overlap would likely reduce it further. 1See R. Christie and F. Lindauer, "Personality Structure," Annual Review of Psychology, Vol. 14, 1963, 217. 75 “The Self-Report method produced a positive relation- ship between n-achievement and maffiliation. The tendency to use the upper or middle parts of the scale may have been a factor in this relationship. The net effect of these possible response sets and methods factors may have been a factor in the low and nega- tive correlations obtained between the motive measurements. The general tendency would appear to be a minor negative relationship between fl-achievement and X-affiliation vari- ables, with stronger negative relationships appearing when certain of the methods factors operate. The Predictive Quality of the Motivational Scales The relationship of the motivational measures to academic achievement and intelligence test performance, and the differential prediction of these criterion measures by the motivational scales are examined in Hypotheses 4 through 7. Hypothesis 4 states that the correlation of the Self-Report Il-achievement scores and academic achievement will be lower than the correlations between other achieve— ment scores and academic achievement. The correlation between Self-Report n-achievement scores and 1963 and 1964 GPA was r = .279, and r = .301. The correlation between 1963 and 1964 GPA was r = .842 and can be considered as a stability reliability coefficient. In Table 4.6, the correlations of all the motivation variables 76 and academic achievement for both 1963 and 1964, the signi- ficance tests, and the "t" values are presented. TABLE 4.6 SIGNIFICANCE TESTS AND CORRELATIONS BETWEEN ACHIEVEMENT MOTIVATION MEASURES AND 1963 AND 1964 GRADE POINT AVERAGES FOR 54 MALE STUDENTS OF SAMPLE C r H II GPA Compariso: of 1964 Motivational Test 1963 1964 GPA, with Self-Report GSCI .242 .145 PJCS .190 .074 WRL .397* .494* HTI .304* .325* M-Scale Total .375* .357* .417 TOI N-ach .207 .200 -.610 TAT N-ach -.O67 -.098 -.202 Self-Report N-ach .279* .301* EPPS N-ach .227 .360* .418 *Probability r = 0 < .05 The correlation between the Self-Report X-ach measure and 1963 and 1964 grades is nearly as high as all the other Arach - GPA correlations. The Word Rating List is cor- related most highly with GPA with the M-Scale total score and the EPPS exceeding slightly the Self-Report relationship. However, both the T01, the TAT, the GSCI and the PJCS are only moderately related to academic achievement. This would tend to indicate that the more elaborate measurement procedure embodied in the TAT and T01 does not 77 produce a score which relates highly to achievement behavior in any direct fashion. The correlations of the TOI to achievement behavior are about .20 and the TAT is not related at all. This findings is consistent with Mitchell's1 and Broverman's2 finding but is inconsistent with other investi- gators' positive findingsB;4 who found correlations of r = .34, and r = .51, respectively. The hypothesis of superiority of complex methods of motivation measurement in the prediction of academic achieve- ment behavior is clearly not supported by these findings. The more complex methods may have a greater ease of inter- pretation, but they do not correlate significantly higher than the self-rating device with achievement behavior. It can be safely stated, however that to the extent to which one wishes to predict academic grades, a self-rating device is not inferior to more elaborate methods. Certainly other justifications could be made for more complex methods. The fifth hypothesis states that the relationship between achievement motivation scores and academic achieve- ment will be larger than the relationship between affilia- tion motivation scores and academic achievement. In Table 4.7, the correlations of the motivation scales and academic achievement are presented. 1Mitchell, op. cit. 2Broverman, et al., op. cit. 3Weiss, et al., op. cit. 4McC1e11and, op. cit. 78 Hm>oH mo. um unmoNMNawNm 05Hm> us so«.m o«H.- Hom. amo.N Hoa.- oNN. ooooom-osom ao«.N NNE.- oom. oo.H Nao. NNN. mama coo. Nmo. ooN. No.a omH.- NoN. Noe o«m. «No.- ooo.- -mmo.- ooo.- Noo.- Noaco<-z “e cosooanaoo<-z ocoso>oNco<-z oocooz. «mo «cos «mo moon mHZMQDHm doomom mon dm mom Hzm2m>mHmo< OHZmQ¢U< Qz< mmqHHOS ho mZOHH¢-achievement scales with)z:affiliation scales. Furthermore, the correlations of urachievement scales and academic achievement (achievement oriented behavior) was in general positive and statistically significant. These correlations with academic achievement were also higher than the correlations of araffiliation and academic achievement. A factor analysis of the test score matrix produced a set of factor loadings which was generally consistent with the predicted result. However, the loadings were moderate, especially for the projective type instruments. There was also much contradictory evidence. Most crucial were the extremely low correlations produced within the study. Although the multitrait-multimethod matrix was generally structured in the predicted direction, the obtained correlations were so low as to be practically insignificant. While this was not a hoped-for result, it was consistent with other studies which investigated these relationships with other samples, and found disappointingly low inter- correlations. There was also the relationship of the one item self- report inventory to academic achievement and intelligence. In most cases the correlations obtained with this measure were as high as those with the more complex measures. While this is perhaps evidence more for the lack of utility of the 91 more complex instruments, it is also evidence that the traits being measured may be more accurately measured. The one- item test would be expected to be somewhat unreliable and thus relate to other variables in an attenuated fashion. There was also the high correlations obtained between theifl-achievement scores and the intelligence scores. Although one'nrachievement measure was more significantly related to GPA than to I.Q., others did not have this dif- ferentiation. The factor analysis was also a mixed blessing. The loadings of the projective fl-achievement measures on the first factor were very low, and the most widely used Jz-achievement measure (TAT) had its highest positive load- ing on a third factor. The second factor was positively loaded with the.fl-affiliation tests. The evidence of construct validity of these tests was therefore both confirming and disconfirming. The evi- dence considered in total would indicate that there is possibly something being measured in common by allifl-achieve- ment scales, but that at least for the projective measures of fl-achievement, there is also something very unique being measured. This conclusion must be tempered, however, in light of the procedures used in the study. These are pre- sented and discussed in the following section. 92 Experimentsl Procedures Possibly Distorting the Results of the Study ‘ The choice of a high school sample may have in- fluenced the results. Although some studies have used the TAT method on high school samples, the M-Scales was the only test which has been specifically validated on high school populations. The high school sample, however, did consistently produce scorable stories. The mean and stan- dard deviation ofid-achievement scores, moreover, were consistently higher than those reported by Atkinson, sp_si., for older samples. However, theJn-affiliation scores ob- tained from the TAT scoring were much lower than those obtained in other studies. The means and standard devia- tions of the TOI z-achievement and n-affiliation scores were not reported in Atkinson, sp_si.,2 but the means and S.D.'s of the present study were likely to be lower. The selec- tion of high school samples may then have seriously affected the elicitation of affiliation imagery. Reasons for this could lie in the nature of the testing session. The tests were administered within the school setting where extra- test cues would be higher for achievement than for affilia- tion (this is, however, a debatable point). The mixing of the eleventh and twelfth grade stu- dents in sample C, may also have influenced the relative prediction of GPA. The fact that both the M-Scales and EPPS)¢-achievement were moderately related to the GPA would 1Atkinson, et al., op. cit., "The Achievement . . . 2Ibid. 93 obviate somewhat the conclusion that the lack of prediction of the projectiveyz-achievement scales was due entirely to this heterogeneity. In any case, high school samples would have sufficient variablity in ability so that all rela- tionships would not be severely attenuated. It is also obvious that the subjects taken by eleventh and twelfth grade students differ. However, the rather high inter- correlation between eleventh and twelfth grade GPA (r = .84) would indicate that this is not a completely invalidating condition. There is also the post-dictive nature of the rela- tionships of motivational scales and intelligence and academic achievement. Both the GPA and intelligence scores are based on behavior samples obtained prior to the moti- vational testing. While this likely leads to the lower correlations consistently exhibited in the study, all these traits are considered to be sufficiently stable in time so that post-dictions would be similar to true pre- dictions. The ipsative nature of the EPPS and the possible ipsative result of the projective scoring may also be a significant factor. The ipsative quality of the EPPS re- sults from the forced-choice method of testing. The negative correlation of fl-achievement and 7z-affiliation results from method of the projectives. For any given fixed amount of content elicited,t1mascoring of this content for 94 ‘n-achievement probably limits the amount of scorable imagery forgnraffiliation. The knowledge of the net impact of these somewhat artificial negative correlations would await a study in which these factors are controlled. The variations in testing conditions throughout the three school systems undoubtedly influenced the results, especially for the projective tests. Examination of the correlation matrices for the three samples shows definite variation in the patterns of intercorrelation. In one sample the TAT-motivational scores are much more highly related to other variables than for the total group. While sampling variation of correlation coefficients is highly likely with small samples, the amount of reversals in the sub-samples is likely due to some of the experimental arrangements. While no justification for these arrange- ments is offered, this condition has to be acknowledged as a contaminating factor in the study. The basic psychometric qualities of the TAT and T01 have to be considered. Atkinson has marshalled evidence that the serial position of the pictures affects the pre- dictive validity of these tests. These serial position effects are believed to be due to a "set of response vari- ability"1 which seriously limits the imagery available after four pictures. This same phenomenon would likely be in evidence when two projective type instruments are administered 95 in succession. Other studies have not controlled this fac- tor and may also have attenuated relationships between ,lrachievement scores and other variables. Finally, there was the variation in the pictures used to elicit thematic imagery. While the pictures were the same, the clarity was less than the original pictures. The consistent elicitation of achievement imagery would indicate that the pictures are perceptually interpreted, there is the possibility that this was a factor in the lack of production of affiliation imagery. However, Atkinson has demonstrated that predictive validity coefficients obtained by pictures judged relevant to achievement fantasy elicita- tion were no better than pictures judged relevant to affili- ation imagery when predictions were made about achievement oriented behavior. Similar reasoning would indicate that as long as imagery is elicited, the clarity and content of the picture is not a very important factor. Recommendations and Impiications of the Study The results of the study and the discussion of the possible meaning of these results lead to a set of recommen- dations about needed research and possible additional re- quirements of theories of achievement and affiliation motivation. 1. In the first place, the low interrelationships ob- tained within this study should be cross-validated 96 on other more carefully chosen samples, before any assertion is made that these tests are measuring similar traits. The length of projective tests should be varied to see whether increased items can elevate the stability reliability of these kinds of instruments. Although four items have been shown to be an optimum for predictive validity requirements, this number of items may simply be too few to reliably measure the trait. The tendency for investigators to use group differ- ences as a criterion for validity of projective .fl-achievement scales may seriously mask the low underlying relationships obtained under correlational methods. While group differences are adequate beginnings for test validation, the conceptions presently advanced about the -achievement trait indicate that adequate correlations should ulti- mately be obtained when using —achievement as an independent variable. The multitrait-multimethod matrix, as a method of test validation should include statistical tests of correlations as an overall evaluation of trait validity. When interrelationships of traits are studied, artificial relationships, such as those obtained by the EPPS, should be removed as much as possible. 97 The possible artificial negative correlations pro- duced by multiple scorings of projective instruments should be reduced as much as possible. An alterna- tive procedure such as separate sets of pictures, used with adequate test intervals, scored only for )Z-achievement and n-affiliation, would eliminate some of the spurious negative correlation. Large data samples should be used for this kind of work. The labor and cost of scoring of sample sizes needed for proper analysis, are too prohibitive for separate studies outside the larger programmatic studies. There are also a number of implications for theory develop- ment which emerge from the study. 8. Broverman's1 conception of fantasy as an alternative to the behavioral expression of achievement motiva- tion should be seriously examined as an explanation of the relationship of fantasy to achievement oriented behavior. The results of the study indicate that TAT;fl-achievement scores are only moderately related to one kind of achievement behavior, and there is the slight suggestion that this may be a negative relationship. 1Broverman, Jordan and Phillips, op. cit. 10. 11. 12. 98 The insertion of a.n-achievement term in theoretical equations may await a more reliable measurement of the trait represented by this term. While there is no finality in the statements made about7Lrachieve- ment scores produced by scoring schemes such as those in Atkinson} there is the implicit suggestion that these scoring schemes produce stable scores as well as scores which have high inter-rater consistency. Conceptions of intelligence may have to include moti- vational components. The rather consistent rela- tionships of nrachievement scores to the I.Q. measure would indicate that intelligence test performance may be more highly weighted with motivational com- ponents, than was previously believed. Affiliation motivation may have to be considered in newer formulations of so-called achievement oriented behavior. The somewhat consistent negative rela- tionship may only be a result of the experimental arrangements, but it may also add predictable com- ponents to any new conceptions of achievement oriented behavior. The unitary trait conception of7Z-achievement may have to be modified. While the evidence is certainly 1Atkinson, et al., 0 . cit., "The Achievement . . . 99 not compelling, there are hints that1nrachievement may be multi-faceted, some facets relating to achievement behavior, others being unrelated. In conclusion, two statements from Atkinson are highly informative as to the general condition of the con- struct of achievement motivation. In 1958 he and Reitman wrote: In 1964 "It is clear however that studies using thematic apperceptive measures can contribute little to real psychological progress until such time as these studies proceed from a firm factual knowledge of the basic intrinsic properties of the instruments."1 he stated: "Research on achievement motivation has come a long way from the earliest studies which proceeded to explore the simple hypoth- esis that TAT“nmachievement scores should be positively related to achievement oriented behavior."2 These statements clearly illustrate the dilemma of investigators who wish to use the projectively measured fl-achievement construct in their studies of behavior. On the one hand, theoretical analyses have progressed very much, but the "real psychological" progress may have to await "a firm factual knowledge of the basic intrinsic pro- perties of the instruments." It is obvious that one cannot proceed without the other. 1Ibid. 2Atkinson, op. cit., An Introduction . . . . 100 The more objective instruments clearly relate more highly in predicted directions than do the more elaborate projective tests. However, even these are not much more highly successful than a short self-rating measure of the ‘fl-achievement trait. BIBLIOGRAPHY American Psychological Association, Committee on Psychologi- cal Tests. Technical Recommendations for Psychological Tests and Diagnostic Techniques. Washington, D.C.: APA, 1954. Atkinson, John W. An Introduction to Motivation. D. Van Nostrand and Co., Inc., 1964. Atkinson, John W. (ed.) Motives in Fantasy Action and Society. D. Van Nostrand Co., Inc., 1958. Atkinson, John W” et al. "The Achievement Motive, Goal Setting and Probability Preferences," Journal_of Abnor- mal and Social Psychology, 99, No. 1, 1960, 27-36. Bechtoldt, Harold. "Construct Validity: A Critique," American Psychologist, Vol. 14, No. 10, October, 1959, 619-6290 Bendig, A.W. "A Preliminary Factor Analytic Investigation of Need Achievement Items," Journal of Psychological Studies, ii, 1959, 32-38. Bendig, A.W. "Manifest Anxiety and Projective and Objective Measures of Need Achievement," Journal of Consulting Psychology, 2i, 1957, 354. Bernardin, A.C. and Jessor, IL. "A Construct Validation of the Edward's Personal Preference Schedule with Respect to Dependencyf'Journal of Consulting Psychology, 2i, 1957, 63-67. Birney, Robert C. "The Reliability of the Achievement Motive," Journal of Abnopmal Social Psychology, id, 1959, 266-267. Blommers, Paul and E.F. Lindquist. Elementary Statistical Methods. Boston: Houghton Mifflin Co., 1960, 451. Broverman, Donald M., Jordan, Edward J., and Phillips, Leslie. "Achievement Motivation in Fantasy and Behavior," Journal oisdbnorgsl and Sociai:§sychology, pp, No. 3, 1960, 374-378. Burdick, Harry. "Interrelationships of Three Measures of Motivation," Psychologisal Reports, d, 1961, 225-226. Campbell, D.T. "Recommendations for APA Test Standards Re- garding Construct Trait or Discriminant Validity," American Psychologist, Vol. 15, No. 8, August, 1960, 546-553. 101 102 Campbell, D.T. and Fiske, D.W. "Convergent and Discriminant Validation by the Multitrait-Multimethod Matrix," Psychological Bulletin} Vol. 56, No. 2, 1959, 81-105. Cole, D., et si. "The Relation of Achievement Imagery Scores to Academic Performance," Journal of Abnormal and Social Psychology, Vol. 65, No. 3, 1962, 208-211. Cronbach, Lee J. and Meehl, Paul E. "Construct Validity in Psychological Tests," Psychological Bulletip, Vol. 52, No. 4, July, 1955, 281-302. Cureton, Edward. "Validity," in E.F. Lindquist, Educational Measurement. Washington, D.C.: ACE, 1951, 621-692. Dicken, Charles F. "Convergent and Discriminant Validity of the California Psychological Inventory," Educational and Psychological Measurement, Vol. 23, No. 3, 1963, 449-459. Ebel, Robert L. "Must All Tests Be Valid?," The American Psychologist, 1961, 640-647. Edwards, A.L. Personal Preference Schedule Manual. The Psychological Corporation, 1954. Farquhar, W.W. Motivation Factors Related to Academic Achievement. United State Office of Education C00per- ative Research Project No. 846, 1963. French, Elizabeth G. "Some Characteristics of Achievement Motivation," Journal of Experimental Psychology, Vol. 50, No. 4, 1955, 232-236. French, Elizabeth G. and Chadwick, Irene. "Some Character- istics of Affiliation Motivation," Journal of Abnormal and Social Psychology, Vol. 52, No. 3, 1956, 296-300. Green, Robert Lee. "The Predictive Efficiency and Factored Dimensions of the Michigan State M-Scales for Eleventh Grade Negro Students, An Exploratory Study," Thesis for the Degree of Ph.D., Michigan State University, 1962. Hempel, C.G. "Fundamentals of Concept Formation in Empiri- cal Science," Internapipnal Encyclopedia of Unified Science, Vol. II, No. 7, Chicago: University of Chicago Press, 1952. Herron, E. Wayne. "Intellectual Achievement Motivation: A Study in Construct Clarification," The University of Wisconsin-Milwaukee, 1964. Hills, John R. "Needs for Achievement, Aspirations, and College Criteria," Jougpal of Educational Psychology, Vol. 41, No. 1, 1958, 156-161. Himelstein, R. and Kimbrough, W.W., Jr. "Reliability of French's 'Test of Insight'," Educational and Psycho- logical Measurement, Vol. 20, No. 60, 1960, 737-741. 103 Himelstein, P., Eschenbach, A.E., Carp, A. "Interrelation- ships Among Three Measures of Need Achievement," Journal of Consulting Psychology, Vol. 22, No. 6, 1958, 451‘452 o Jessor, R. and Hammond, K.R. "Construct Validity and the Taylor Anxiety Scale," Psychological Bulletin, Vol. 54, 1957, 161-170. Johnson, Van C. "An Assessment of the Motivation Factor in the Estimation of Academic Achievement of Eleventh Grade Indian Students and the Factored Dimensions of the M-Scales." An Exploratory Study, An unpublished doctoral dissertation, Michigan State University, 1963. Jordan, T.E. and DeCharms, R. "The Achievement Motive in Normal and Mentally Retarded Children," Amssican Journal of Mental Deficiency, d3, 1959, 457-466. Kagan, Jerome and Moss, Howard A. "Stability and Validity of Achievement Fantasy," Journal of Abnormal and Social Psychology, Vol. 58, No. 3, May, 1959. Karolchuk, P. and Worell, L. "Achievement Motivation and Learning," Journal of Abnormal and Social Psychology, gs, 1956, 255—257. Kausler, D.H. and Trapp, E.P. "Methodological Considera- tions in the Construct Validation of Drive Oriented Scales," Psychological Bulletin, 2d, 1959, 152-157. Kipfmueller, Mark K. "The Predictability and Factored Di- mensions of the M-Scales for Eleventh Grade Parochial School Students," unpublished doctoral dissertation, Michigan State University, 1963. Krumboltz, J.D. and Farquhar, W.W. "Reliability and Validity of the N-Ach Test," Journal of Consulting Psychology, Vol. 21, No. 3, 1957, 226-228. Littig, Lawrence W. and Yeracaris, Constantine A. "Academic Achievement Correlates of Achievement and Affiliation Motivations," Journal of Psychology, 52, No. 1, 1963, 115‘119. Loevinger, Jane. "Psychological Tests as Instruments of Psychological Theory," Psychological Reports, 1957, 635-694. Longnecker, E.D. "Perceptual Recognition As a Function of Anxiety Motivation, and the Testing Situation," Journal of Abnormal and Social Psychology, Q4, 1961, 215-221. Marlowe, D. "Relationships Among Direct and Indirect Measures of the Achievement Motive and Overt Behavior," Journal of Consulting Psychology, Vol. 23, No. 4, 1959, 329-332. 104 McClelland, D., Atkinson, J., et al. The Achievement Motive. New York: Appleton-Century-Crofts, 1953. Melikian, Levon H. "The Relationships Between Edward's and McClelland's Measures of Achievement Motivation," Journal of Consulting Psychology, Vol. 22, No. 4, 1958, 296-299. Mitchell, James J. "An Analysis of the Factorial Dimensions of the Achievement Motivation Construct P," Journal of Educational Psychology, 1961, 179-187. Murray, H.A. Exploration in Personality. New York: Oxford Press, 1938. Payne, David A. "The Concurrent and Predictive Validity of an Objective Measure of Academic Self-Concept," Educational and Psychological Measurement, 1962. Phares, E.J. and Adams, C. "The Construct Validity of the Edwards PPS Heterosexuality Scale," Journal of Con- sulting Psychology, 1961, 25, 341-344. Reitman, Walter R. "Need Achievement, Fear of Failure, and Selective Recall," Journal of Abnormal and Social Psychology, Vol. 62, No. 1, 1961, 142-144. Riccuiti, Henry N. "The Prediction of Academic Grades with a Projective Measure of Achievement Motivation I Initial Validating Studies," Princeton, New Jersey, ETS, 1954. Shaw, M.C. "Need Achievement Scales as Predictors of Academic Success," Journal of Educational Psychology, Vol. 52, No. 6, 1961, 282-285. Thorpe, Marion D. "The Factored Dimensions of an Objective Inventory of Academic Motivation Based on Eleventh Grade Male Over- and Under-achievers," unpublished doctoral dissertation, Michigan State University, 1961. Uhlinger, C.A. and Stephens, M.W. "Relation of Achievement Motivation to Academic Achievement in Students of Superior Ability," Journal of Educational Psychology, Vol. 51, No. 5, 259-266. VanZandt, B.R., and Himelstein, P. "The Role of Verbal Fluency on a Projective Measure of Motivation," Edu- cational and Psychological Measurement, Vol. 21, No. 4, 1961, 873-878. Walker, H.M. and Lev, J. Statistical Inference, Henry Holt and Co., New York, 1953. 105 Weiss, P., Wertheimer, M. and Groesbeck, Byron. "Achieve- ment Motivation, Academic Aptitudes, and College Grades," EducationalanuiPsychological Measurement, Vol. 19, No. 4, 1959, 663-666. Zuckerman, M., et al. "Concurrent and Construct Validity of Direct and Indirect Measuring Dependency," Journal of Consulting Psychology, Vol. 25, No. 4, 1961, 316-323. 106 omo. oo~.: 09%. 5N0. Hmo. wNH. Nwo. HNH... mam. MON... m2. mac... moo. HNH. HmN... mHN. TNO... mNo... mmo.n 9:. N00... mmo. No1 oom. moo. owH. «NM. 02. 000... moi: omit 1Com. m2... HNO. moo. 2N... oNN. HNo. «on. «as. NNm. HHH.- NNo.- mos. mmm.- o mos. omN. «oo.-. mos. ooN.- m«o. Nmo.- m«a.- o«o. o oNa.- HoN. oNH. ««o.- on. oNo. HNo.- NNH.- «ma.- N oma.- oma.- oNo.- NNH. NNN.. o«a.- oNo.- moo. HEN. o O O O “mngz ooo. NNN.- Nm«.- o«o. Hom. HNH. omo owe - NHN - m oaoosoo> « m N a m w u o m 1N . m N H umnfidz HODMMMJ whzmnbhm NHH mom mmmoom Emma. ZOHHmHmU¢Iz m>Hm mo mHm>A