PERFORMANCE 0M PROFESSIONAL-EVE. SELECTION TESTS BY SOCMLLY- SENSITIVE DEMOGRAPHIC GROUHNGS Thesis for the Degree of M. A. MICHIGAN STATE UNIVERStTY OBDULIA C. BECERRiL 1971 ‘fwe‘sss E: BmEmd tar" “UM; & SUNS ' “5 ’ BUUK W“ INC. ' M 13mm alrhrns mu: ABSTRACT PEREORMANCE ON PROFESSIONAL-LEVEL SELECTION TESTS BY SOCIALLY-SENSITIVE DEMOGRAPHIC GROUPINGS BY Obdulia C. Becerril A state civil service commission put together a battery of tests which was then administered to 112 applicants for the sensitive and important position of civil rights repre- sentative, a position calling for at least the bachelor's degree. The commission had to rely on its judgment in choos- ing tests for the battery, since the job was completly new and no criterion data could possibly be available. This study is concerned with the analysis of the data collected from the administration of this test battery. Factor analysis of the battery revealed three factors: general verbal facility or "G", civil rights knowledge (a kind of achievement often useful in selection testing), and ability to write reports in good English. From the point of view of fairness, an especially criti- cal matter in the seleétion of persons themselves to be involved with implementing fairness, it was possible to obtain information on four sensitive demographic characteristics of the 112 applicants: race, present civil service status Obdulia C. Becerril (those already on civil service status could conceivably do _better on traditional civil service tests because of superior test wiseness), education and sex. No statistically significant relationships emerged from an analysis of the cross-relationships among these demo- graphic variables for the sample of applicants. Analysis of the relationships between test scores and the demographic variables showed that significant differences in performance on the tests was associated only with race and not with any of the other demographic variables. Whites did significantly better on the "G" type tasks, while non- whites outperformed whites on the civil rights knowledge tasks. Fortunately scores on these two types of tests roughly counterbalanced each other so that the battery as a whole probably was relatively free from any racial differen— tiation. None of the other test performances showed any significant relationships with race. With respect to the "G" type tests, analysis showed that the better performance of whites occurred through all educational levels represented in the sample, including graduate level. These results with the "G" type tests should be added to the accumulating literature on relationships between such tests and demographic characteristics. Approved::9¥~ddhda;. ”2. C~L-4éAvf§ Advisor Date: QM“ /’) /97/ U PERFORMANCE ON PROFESSIONAL-LEVEL SELECTION TESTS BY SOCIALLY-SENSITIVE DEMOGRAPHIC GROUPINGS By ,. g"; '1 it.‘ Obdulia Clwhecerril A THESIS Submitted to Michigan State University in partial fulfillment of the requirements for the degree of MASTER OF ARTS Department of Psychology 1971 ACKNOWLEDGEMENT S I would like to express sincere appreciation to Dr. Frederic R. Wickert who served as chairman of my thesis committee. His unfailing encouragement, under- standing and guidance was immensely valuable. My appreciation is also extended to Dr. John H. Wakeley and Dr. Frank L. Schmidt for their assistance and willingness to serve on the committee. Grateful acknowledgement is also due to the Civil Service Commission who provided the data which made this research possible. ii TABLE OF CONTENTS CHAPTER LIST OF TABLES. . . . . . . . . . . . . . . . I. INTRODUCTION. . . . . . . . . . . . . . . . . Problem . . . . . . . . . . . . . . . . . Review of the Literature. . . . . . . . II. METHODS . . . . . . . . . . . . . . . . . . . Components of the Study . . . . . . . . . subjects and demographic groupings. . Biographical data . . . . . . . . Tests . . . . . . . . . . . . . . . Statistical Methods . . . . . . . . . . . III. RESULTS . . . . . . . . . . . . . . . . . . Introduction. . . . . . . . . . . . Interrelationships Among the Predictors . Interrelationships Among the Four Demo- graphic Variables . . . . . . . . . Interrelationships Between the Test Scores and the Four Demographic Variables . . . . . . . . . . . . . Summary of Results. . . . . . . . . . . . IV. CONCLUSIONS AND INTERPRETATIONS . . . . . . . REFERENCES . . . . . . . . . . . . . . . . . . . . APPENDICES . . . . . . . . . . . . . . . . . . . . . A. CIVIL SERVICE APPLICATION . . . . . . . . . . B. ANNOUNCEMENT: CIVIL RIGHTS REPRESENTATIVE. . C. CROSS-RELATIONS AMONG NUMBERS OF SUBJECTS IN EACH OF THE FOUR DEMOGRAPHIC VARIABLES. . . . D. MEAN SCORES AND STANDARD DEVIATIONS . . . . . iii Page iv 11 11 ll 11 12 16 17 17 18 21 24 30 32 36 37 37 38 41 42 TABLE 1-1 2-1 LIST OF TABLES Number of Subjects by the Four Demographic Variables Who Took the Written and Oral Tests. Racial Composition of Subjects Who Passed and Failed the Total Test Battery. . . . . . . Intercorrelations Among the Tests in the Battery Used for the Selection of Civil Rights Representatives. . . . . . . . . . . . . . . . Factor Loadings on a Factor Analysis of the Intercorrelations Among the Tests Used in the Selection of Civil Rights Representatives. . . Chi Square Contingency Table for the Four Demographic Variables. . . . . . . . . . . . . Number of Subjects in Each Race and Civil Service Status Category. . . . . . . . . . . Summary of Analysis of Variance for CST Against Each of the Four Demographic Variables Summary of Analysis of Variance for KPACRM Against Each of the Four DemOgraphic Variables Summary of Analysis of Variance for CRANDHR Against Each of the Four Demographic Variables Summary of Analysis“of Variance for I and I Against Each of the Four Demographic Variables Summary of Analysis of Variance for the Essay Against Each of the Four Demoqraphic Variables Summary of Analysis of Variance for the Total Weighted Score Against Each of the Four Demographic Variables. . . . . . . . . . . . . Summary of Analysis of Variance for the Oral Against Eadh of the Four Demographic Variables Significance of Mean Test Score Differences of Whites and Non—whites on the CST, KPACRM, CRANDHR and Oral subtests. . . . . . . . . . . iv Page 16 19 20 22 23 25 25 26 26 27 27 28 29 CHAPTER I INTRODUCTION Problem The Civil Service Commission of a midwestern state with significant numbers of concerned and active minority group members found itself, in 1966 for the first time, con- fronted with putting together a civil service battery of tests to select fairly not from among the usual, less skilled level applicants but applicants to fill a profes- sional level job. This professional level job carried the title Civil Rights Representative. The applicants for such a job, would, it could be expected, be especially concerned that they be selected without bias and that the tests used for their selection be a model of fairness. To further complicate matters, the law authorizing the hiring of these representatives had only recently been passed and the job description was, necessarily, far from complete. In other words, no present employees were functioning as civil rights representatives so job analysis and job description in the customary way could not be carried out. The Civil Service Commission did the best it could with the limited information about the job and the time at its disposal. It put together a battery of four multiple-choice group paper and pencil tests, and an essay test. To those who came above the cutting score on the weighted total score for all five written tests, the usual board oral examination was administered at a later date. The population taking the battery on its first adminis- tration was made up as shown in Table 1-1. Table 1-1. Number of Subjects by the Four Demographic Vari- ables Who Took the Written and Oral Tests Took the five Took the Category written tests Oral test Race Whites 35 6 Non—whites 48 17 Non-responders 29 8 Civil Service Status State employees 38 ll Non-state employees 39 ll Non-responders 35 9 Education Undergraduate work only up to and including college graduation 54 15 Beyond college graduation 57 16 Non-responders l —— Sex Males 81 21 Females 31 10 Total in each category 112 31 It will be observed that there was sufficient variance on the four socially-sensitive demographic variables, race, civil service status, education, and sex, to make it possible to compare performance on the tests among the sub-groupings of peOple delineated by the four demographic variables. It will further be observed that substantial numbers of sub- jects did not report their race or current civil service status, such substantial numbers, in fact, that non-responders may, in the analysis of the data, have to be regarded as a separate group. The availability of data on still another dimension, namely, the types of tests included in the battery, makes possible an analysis of special value to the psychology of test construction. One of the tests in the battery was one that the Civil Service Commission had used many times with other groups of about the same educational level as the appli- cants for the civil rights representative positions. Predictably this test was largely measuring general verbal facility or verbal intelligence. In view of continuing questioning now going on in American society concerning the fairness of using such tests with minorities, it should be illmminating to see how well highly educated minorities as cmmpared with a group of about equally highly educated majorities perform. Another type of test that the Civil Service Commission evidently considered had some promise of validating was designed to measure knowledge of people associated with the civil rights movement. It is a well-established practice in employment testing to assume that a relevant knowledge on achievement test might predict job success. A third type of test was, again, knowledge or achieve— ment, but this time the concern was with social science principles applied to minorities including some techniques of social investigation and conciliation. A fourth type of test also measured knowledge of investigation techniques but in the case of this test, the focus was more on inter— viewing and doing research. The essay test asked each applicant to write an essay to assess his ability to write in English, a skill required for the considerable amount of report writing the civil rights representatives would probably have to do. To summarize about the battery of tests used, it is clear that the Civil Service Commission was ingenious in developing a battery of promising tests among the kinds of tests customarily used in civil service testing. As is usual in civil service testing, however, there was some danger that all the tests would be saturated with general verbal facility, a kind of facility that in other selection testing has been sometimes shown to discriminate against minorities. The achievement and knowledge tests, provided the general verbal facility component was not too great, could be tapping other abilities or aptitudes associated with civil rights work. Review of the Literature Few studies in the recent literature address themselves to internal analyses of test batteries used in the selection of socially sensitive groupings of peOple. This lack of attention to internal analysis is somewhat surprising in View of the expense and difficulty involved in validating such batteries. Advance internal analysis could reveal situations that call for additional test battery development ‘work before validation itself is attempted. For example, one might check to assure himself that one has a factorially heterogeneous battery rather than too homogenous a one. The prevailing view tends to be that expressed by Bartlett and O'Leary (1) who state, "The problem of discrimi- nation against minority groups is then not only a question of differences between mean test performance for two races, but also a question concerning the nature of the relation- ship between test and criterion performance," in other words, validity. Cronbach (2, p. 298) says "The real issue is relevance." Kirkpatrick gt Q1. (3, p. 6) and Schmidt (9), each in their own way, differentiate between differential validity and test bias, both of which concepts again involve validity. The generally accepted approach to minority group discrimination is testing as described above could not be followed here in the absence of criterion data. An approach that involves both group differences and validity is that of groups matched by standing on the criterion. Wollowick gt a1. (11) reports means and standard deviations on three tests (verbal ability, non-verbal reasoning, and arithmetic reasoning), two demographic vari- ables (age and education), and two criterion measurements (supervisory rank achieved and salary) for Caucasian and Negro administrative personnel in branch offices of a large electronics organization. Average education for both Cauca— sians and Negroes was a little less than a year and a half of college; the groups are somewhat less well educated than those in the present study. Two Caucasian groups, one matched for salary and the other for supervisory rank (separate analyses for each criterion) in this situation had significantly higher mean scores on all three tests in the battery, including the verbal ability test. These higher mean scores suggest that the kind of tests used could be discriminating against Negroes. Moore gt a1. (7) also reports a matching study, but in this case the matching was limited to demoqraphic varia— bles and no criterion variables. subjects were Negro and White applicants for refinery work who were matched on age and education. Average education was not reported, but education was reasonably comparable with that of the 83 used by Wollowick §§_§l,, but a little lower than the group used in the present study since the following educational data are reported in one of two groups (Southwest United States): 43 were high school graduates only, 32 had less than 2 years of college, while 16 had more than two years of college; in the second group (Northeast United States) 29 were high school graduates only, 20 had less than 2 years of college, and 6 had more than 2 years of college. The battery of five tests included a spiral omnibus test consisting of block counting, vocabulary, and arithmetic tasks, a test of spatial reasoning utilizing geometric symbols, a test of chemistry comprehension, the Bennett test of mechanical comprehension (Form BB), and the mathematics section of the advanced California Achievement Tests (Form W). The authors believed that the battery roughly measured two factors: spatial and verbal. As in the Wollowick gt 3;. study, whites did sig- nificantly better on the tests except for the verbal tests in the Northeast sample, but actual differences were small. At this point, the suggestion is that the Civil Service Commission in the present study could have been on not especially safe ground in using the battery that they did. Another set of results reported in the Moore §t_al, article concerns the intercorrelations among the tests in the battery. The correlations are remarkably uniform: only 10 of the 60 coefficients were .70 or larger or .29 or smaller. No clear internal structure was evident. One by-product of the Moore gt 31. study was that mean scores on tests of the kind used in this study were substan— tially higher for Negroes who had some college over those who were no more than high school graduates. Such a finding supports the Civil Service Commission's action in the present study to go ahead and use a range of paper and pencil tests for their relatively well educated applicants for civil rights representatives without much fear of racial differen- tiation. Ruda and Albright (8), reported in their analysis of Wonderlic scores among applicants for office jobs, that Negro- White differences remained in favor of the Whites at high school graduate, some college, and college graduate levels of education. On the general matter of Negro-White differences in performance on general mental ability tests, usually but not necessarily, tests with a substantial verbal facility com- ponent, Krug (6) states, "By usual psychometric standards, we can say that we 'knew' that mean scores for Negroes are lower than the mean scores for whites on most tests of general ability, intelligence, academic aptitude, or whatever you choose to call these measures of 'G'. Kirkpatrick §t_§l, (3) were successful in compiling a series of five investigations on testing and fair employment. Two of these studies demonstrate a situation in which appli- cants performed in a comparable manner in the selection situ— ation. In a study involving female clerical employees in an insurance company, 34 black and 102 white clerks were given one of the Short Employment Tests. Results showed that the white and non—white groups were approximately equal in terms of mean test performance. Since no differences were also present in the mean criterion scores, the investigators concluded that the predictors did not discriminate unfairly in this situation. Comparable results were found in a second study involv- ing a new sample of female clerical workers. After a battery of vocabulary, checking, and numerical tests plus two non- verbal tests were administered, the authors found no signifi- cant differences between minority and non-minority group members in terms of mean test performance. Few studies comparing Negroes and Whites on general ability or achievement at higher educational levels seem to have appeared in the literature beyond the two cited earlier here. A check of the Dreger and Miller reviews (4,5) of Negro-White differences revealed no comparable studies. Two additional such studies were located. In the first of these two studies, Kirkpatrick (3), in his study 4 on nursing stu- dents did not have very comparable groups since the white '§;§ attended nursing schools for their race and the Negro S's schools for theirs. Science research associates (10) describe a study comparing test performance (FIT Ingenuity and Arithme- tic) for non-whites and whites but gives only an expectancy table and a multiple correlation for their group of presumably fairly well educated claims auditors. 10 The review of the literature, then, reports little on (1) Negro—white differences in performance on general ability type tests of the sort used by civil service commissions, and (2) the internal structure of test batteries designed for personnel selection where both Black and White applicants are involved and, in fact, where differences could prove to be a sensitive matter. CHAPTER II METHODS Components of the Study Subjects and demographicggroupings The subjects and demographic groupings are those given in Table 1-1. As that table indicates, race, civil service status, education, and sex information were generally avail- able on all 112 subjects. Biographical data: Information regarding the subjects' race and civil service status came from a special civil service application form (Appendix A). The information was requested as part of a program designed to assure equal em- ployment Opportunity and was collected jointly by the Civil Service Commission and the Civil Rights Department. The form was administered before the applicants took the examination. As part of the instructions, the examiner stated that one need not answer the questions that reflected demographic information. However, after the purpose of asking for the information was made clear, the cooperation of the applicants was strongly urged. Also, the applicants were advised that the information they supplied would not affect their examina- tion score in any way. As was noted in Chapter I, many did 11 12 not answer the questions on race and civil service status. Rather than the special form used to obtain race and civil service status, the other two demographic variables, education and sex, were obtained from the standard civil service application form. Answering these two questions was mandatory if one wanted to take the tests. As Table 1-1 shows, there was almost'complete information on education and sex. Although sex would appear to be irrelevant to the job, educational requirements were listed in the civil service announcement. As that announcement shows, a bachelor's degree with additional credit for a master's degree was a prerequisite, although experience could be substituted for college education year for year up to two years (see Appendix B). 16111.81 The tests were those briefly described in Chapter I. A more complete description of each of these tests follows. It should be noted that civil service regulations prohibit any more detailed description of the tests than that shown in the following paragraphs. The Civil Service Test (CST) is a typical civil service test, a test of general mental ability, designed to evaluate the fitness of applicants and their capacity to develop so as to merit advancement. It has 50 multiple choice items, each one having 4 choices. The score is the number of correct answers . 13 Knowledge of pe0ple associated with the Civil Rights Movement (KPACRM) is a matching test made up of 25 items. The score is the number of correct responses. The purpose of this test is to evaluate the applicant's knowledge of the recent developments in the field of civil rights and the persons associated with these develOpments. The applicant must know the resources which can be employed to assist minority groups and the recent developments in state, federal and local statutes, court orders, rulings, and programs affecting the civil rights movement. The Civil Rights and Human Relations test (CRANDHR) has 56 items and the Interviewing and Investigation test (I and I) has 32 items. Both are multiple choice tests with 4 choices for each item. The score is the number of correct answers. ‘The purpose of this test pertains to the knowledge each applicant must have of the techniques used to ease intergroup tension and conflicts, knowledge of the socio- lOgical and economic forces which affect the welfare of minority groups, and knowledge of the principles of sociology, psychology, and human relations. The test judges the appli- cant's ability to maintain an impartial attitude, his tact and similar traits while investigating and conciliating civil rights problems. I and I emphasizes the techniques of effective investi- gation and knowledge of social investigation. The test is designed to measure an applicant's ability to perform research studies concerning minority groups. 14 The Essay is the last section of the written part. The applicant writes an evaluation of equal employment Oppor- tunities in the building trades. The aim is to measure the applicant's interpretation of the purpose of a research study on this tOpic and then to make a detailed yet compre- hensive report. His ability to use correct English and to write effectively and accurately is assessed. Grading for this examination is based on the quality of the writing mechanics and content. With respect to mechanics, the length of the review, punctuation, spelling, grammar, and orderliness and neatness are evaluated. With respect to content, the applicant must state the purpose of the study, note not only the racial patterns in the con- struction labor force, but also problems that limit the upgrading of Negroes in the construction industry and the role of unions, companies, and union leaders in relation to these problems. The total possible raw score is 100 points, 50 points for contents and 50 for mechanics. This raw score is developed into a weighted score. The two examiners, a civil service examiner and a college professor, give two completely independent evaluations.1 The score given by the professor is multiplied by two while the score given by the examiner is given a weight of one. The maximum weighted score is 300. 1Data were not available to calculate an estimated reliability coefficient for this Essay test. 15 The last part of these examinations is an oral appraisal. The resulting rating is supposed to be limited to an evalu- ation of the applicant's (l) personality and appearance, (2) experience and education, and (3) ability to deal with people and speak correctly and effectively. The examiners are part of the CSC oral board. Its members come from private industry, the professions, other agencies of government, and civil service staff members. The total oral score in this case of this test ranged from 29 to 40 points. The Civil Service Commission, to keep the expense and time of appraisal at a minimum, substantially reduced the number of applicants invited to appear for the orals by a screening process which considered education, experience, and/or performance on the written test. For purposes of overall evaluation, the written and oral examination scores were both considered. The written part was weighted 60% and the oral 40%. The passing score was 70%, as is set by law. A summary of experience with the total battery indicates that 163 applied, 112 actually took the written tests, 53 passed the written test and were invited to take the oral, 31 took the oral, and 29 passed the complete examination. Table 2-1 shows the racial composition of those failing and passing the total examining process. 16 Table 2-1. Racial Composition of Subjects Who Passed and Failed the Total Test Battery* 4 Failed Passed Total N .%i N % N % Whites 29 83 6 17 35 100 Non-whites 32 67 16 33 48 100 Non-responders 22 76 7 24 29 100 Total 83 29 112 *Of the 31 applicants who appeared for the oral, a total of 29 passed. Of the two who failed, 1 was non-white and l was a non-responder. Percentages of each race who passed or failed were not greatly affected. Statistical Methods Intercorrelations among the tests, all continuous varia— ble measures, were Pearson product-moment correlations. As checks on each other both quartimax and varimax rotations were performed to analyze out the factors present in the matrix of test intercorrelations. Interrelationships among the non-continuous demographic variables were computed by means of chi-square. Interrelationships among the continuous variable test scores and the non-continuous demographic variables were analyzed by one-way analyses of variance. Means and standard deviations of each of the tests by each of the four demo— graphic variables are presented in Appendix C. All computations were carried out by computer. CHAPTER III RESULTS Introduction The results presented in this chapter can be classified according to the three issues which this study is designed to answer. The first section is concerned with the inter— relationships among the predictors, or, in other words, the tests the Civil Service Commission used putting together the battery to select civil rights representatives. Ques- tions to be answered revolve around what factors were present in the battery. Were the tests all general verbal ability, was one or more types of achievement being tested, or what other factors were operating in this situation? The second section concerns the interrelationships among the demographic variables: race, civil service status, education and sex. The purpose of the analysis in this second section is to answer such questions as the following: Were those with current civil service status of one or both races or high or low education cu: one or both sexes? Were those with high education from one race or both races, with or without current civil service status, or of predominantly one sex or both? Were the men or the women generally those of 17 18 one race or both races, of higher or lower education, or present civil servants or not? The third section is the pay-off section. It lays out the relationships among test performance and the demographic variables. It describes the extent to which race performed better on each of the tests, whether those already with civil service status or those from outside the system did better or less well on the tests, and the extent to which education and sex affected test performance. Interrelationships Among the Predictogg Seven of the fifteen possible intercorrelations among the predictors as shown in Table 3-1 were significant at the .05 level or better. In other words, there was a fair amount of intercorrelation running through the tests in the test battery. CST, the test of general verbal facility, had significant correlations with both CRANDHR and I and I but not KPACRM. Also CST did not enter into the essay or oral tests. KPACRM had its most substantial relationship with CRANDHR. Both tests, it will be recalled are tests of a similar type of achievement. KPACRM seemed very free from general verbal facility as measured by CST although KPACRM did enter into whatever the Oral was measuring. CRANDHR was related to all the other tests in the battery. Its highest relationship, however, was with KPACRM, 19 Table 3—1. Intercorrelations Among the Tests in the Battery Used for the Selection of Civil Rights Representa- tives** ___—"‘—"=___ KPACRM CRANDHR I AND I ESSAY ORAL CST .125 .449* .416* .129 .217 KPACRM .635* .124 .122 .411* CRANDHR .364* .206* .502* I AND I .170 .262 ESSAY .332 *Significant at the .05 level or better **Note: N=112 on all tests except the Oral. On the Oral N=31. (Significant levels shown have been adjusted for the changes in N.) a closely related achievement test. CRANDHR unlike KPACRM, however, was significantly correlated with CST; evidently CRANDHR has more general verbal facility in it than KPACRM. CRANDHR entered substantially into the Oral but had less over— lap with I and I and the ESsay. I and I had some CST in it as well as some CRANDHR. Otherwise it was a rather separate test. The Essay was not related to any of the other tests except slightly with CRANDHR. The Essay may have been measur- ing something quite different from them but its relative independence may be a function of the low reliability often characteristic of essay tests. 20 The Oral and medium correlations with the more achieve— ment type tests, KPACRM and CRANDHR, but with little else, except for the slight relationship with the Essay. Evidently the examiners in the Oral especially focussed on race relations. In view of the patterns of intercorrelation among the tests in the predictor battery, factor analyses (both quartimax and varimax rotations to serve as checks on each other) were performed. Table 3—2 shows that three factors emerged. Factor loadings are also shown. Table 3-2. Factor Loadings on a Factor Analysis of the Intercorrelations Among the Tests Used in the Selection of Civil Rights Representatives I II III uartimax Rotation Analysis CST .83 .13 —.01 KPACRM -.00 .94 .03 CRANDHR .44 .80 .08 I and I .81 .05 .ll Essay .ll .09 .98 Varimax Rotation Analysis CST .83 .15 -.00 KPACRM -.02 .94 .04 CRANDHR .42 .80 .10 I and I .81 .07 .12 Essay .09 .08 .98 Factor I seems to be made up of general verbal facility. Its main loadings were on CST and I and I. I and I turned 21 out to be a typical general verbal civil service test rather than any measure of specific interviewing and investigative ability. Factor II has its main loadings on the two civil rights knowledge or achievement tests, KPACRM and CRANDHR. The third factor is mostly whatever is special about the Essay test, presumably mastery of written English. It will be observed that the Oral, in view of the much smaller number of cases who took the test, was omitted from the factor analysis. Of the two civil rights knowledge tests, CRANDHR has a substantial element of general verbal facility in it while KPACRM does not. On the whole, the factor structure came out startlingly clear; The whole battery was not measuring only the all too often encountered general verbal facility. Rather, other factors were being measured as well, quite independently of general verbal facility. Interrelationships Among the Four Demographic Variables The analysis of these interrelationships is presented in the chi square results shown in Table 3-3. In general, no significant chi squares occurred. The only significant chi square occurred in the race vs. civil service status analysis. Further study shows that race and civil service status were the two demoqraphic items that substantial numbers of sub- jects refused to report. In other words, the one significant 22 Table 3-3. Chi Square Contingency Table for the Four Demographic Variables Signifi— Coeff. of Chi— cance contin- N square df level gency Race Civil service status (W—NW-NR) 112 79.79 4 .05* .645 Civil service status (W-NW) 76 1.36 l n.s. .127 Education 19.71 12 n.s. .386 Sex 3.37 2 n.s. .171 Civil Service Status Education 17.32 12 n.s. .366 Sex 1.84 2 n.s. .127 Education Sex 7.27 6 n.s. .246 chi square seems traceable to the non-answering behavior that occurred on these two variables. Table 3—4 shows the calculation of chi square for the race and civil service status relationships with and without the non-answering respondents. It is quite clear from this table that the chi square with non-responding behavior eliminated, is not significant, just as were the chi squares for the relation- ships among all the other demographic variables. The numbers on which the chi squares in Table 3-4 are based may be found in Appendix C. 23 Table 3-4. Number of subjects in Each Race and Civil Service Status Category Including non-responders X2 = 79.79 with 4 degrees of freedom; significant at 5% level RACE Non— No Total Whites, whites, response Civil Service Status State employees 38 13 25 0 Non-state employees 39 18 20 1 No response 35 4 3 28 Total 112 35 48 29 With non-responders omitted x2 = 1.36 with 1 degree of freedom; not significant RACE Total Whites Nonewhites Civil Service Status State employees 38 13 25 Non-state employees 38 18 20 Total 76 31 45 It may be noted in passing that Appendix C shows that among the non-whites the pr0portion of females was higher than that among whites and non—responders to the question identifying race. This chi square, however, was not quite 24 significant at the 5% level. Similarly, non-white female applicants tended to have less education than males, but again the chi square was not significant at the 5% level. Interrelationships,Between the Test chggg and the Four Demographic Variables This third and last set of results is concerned with showing performance on each of the tests by each of the four demographic groupings of race, civil service status, education, and sex. Taken into account in the discussion of these results will be the factor structure of the test battery on one hand, and the rather although not perfectly even distribution, of demographic characteristics in relation to each other on the other hand. More specifically, the results for each test are in the form of a one—way analysis of variance. Tables 3-5, 3-6, 3-7, 3—8, and 3-9 present the analyses of variance for CST, KPACRM, CRANDHR, I and I, and the Essay tests respectively. the five tests, the interrelationships among which were factor analyzed as shown in Tables 3-1 and 3-2. Table 3-10 and 3—11 respectively show the results for the Total Weighted Score and the Oral. The means and standard deviation on which the analyses of variance are based are given in Appendix D. Inspection of Tables 3-5 to 3-9 inclusive shows that the only demographic variable with a significant F is race. 25 Table 3—5. Summary of Analysis of Variance for CST Against Each of the Four Demographic Variables Source SS df MS F Race 580.84 2 290.42 7.45* Error 4244.01 109 38.93 Civil service status 32.60 2 16.30 0.37 Error 4792.25 109 43.96 Education 146.87 6 24.47 0.54 Error 4677.97 105 44.55 Sex 64.89 1 64.89 ’ 1.49 Error 4759.96 110 43.27 Total 4824.85 111 *Significant at the .05 level or better. Table 3-6. Summary of Analysis of Variance for KPACRM Against Each of the Four Demographic Variables Source SS df MS F Race 334.19 2 167.09 6.03* Error 3016.91 109 27.67 Civil service status 1.38 2 .69 0.02 Error 3349.72 109 30.73 Education 242.36 6 40.39 1.36 Error 3108.74 105 29.60 Sex 27.28 1 27.28 .902 Error 3323.82 110 30.21 Total 3351.10 111 *Significant at the .05 level or better. 26 Table 3—7. Summary of Analysis of Variance for CRANDHR Against Each of the Four Demographic Variables Source SS df MS F Race 341.32 2 170.66 2.90 Error 6411.59 109 58.82 Ciyil service status 144.92 2 72.46 1.19 Error 6607.99 109 60.62 Education 358.11 6 59.68 0.98 Error 6394.80 105 60.90 Sex 40.59 1 40.59 0.66 Error 6712.32 110 61.02 Total 6752.91 111 *Significant at the .05 level or better. Table 3—8. Summary of Analysis of Variance for I and I Against Each of the Four Demographic Variables Source SS df MS F Race 49.95 2 24.97 0.60 Error 4472.03 109 41.02 Civil service status 117.25 2 58.62 1.45 Error 4404.73 109 40.41 Education 264.43 6 44.07 1.08 Error 4257.55 105 40.54 Sex .50 1 .50 0.01 Error 4521.49 110 41.10 Total 4521.99 111 *Significant at the .05 level or better. 27 Table 3-9. Summary of Analysis of Variance for the Essay Against Each of the Four Demographic Variables Source SS df MS F Race 27.11 2 13.55 .73 Error 2020.16 109 18.53 Civil service status 74.82 2 37.41 2.06 Error 1972.45 109 18.09 Education 158.14 6 26.35 1.46 Error 1889.13 105 17.99 Sex 19.92 1 19.92 1.08 Error 2027.35 110 18.43 Total 2047.27 111 *Significant at the .05 level or better. Table 3-10. Summary of Analysis of Variance for the Total Weighted Score Against Each of the Four Demographic Variables *- - ‘—_. 4 Source SS df MS F Race 460.94 2 230.47 1.34 Error 18664.54 109 171.23 Civil service status 458.75 2 229.37 1.33 Error 18666.73 109 171.25 Education 748.15 6 124.69 0.71 Error 18377.33 105 175.02 Sex 35.03 1 35.03 0.20 Error 19090.45 110 173.54 Total 19125.49 lll *Significant at the .05 level or better. 28 Table 3-11. Summary of Analysis of Variance for the Oral Against Each of the Four Demographic Variables Source SS df MS F Rage 638.70 2 319.35 1.39 Error 24951.26 109 228.91 Civil service status 42.99 2 21.49 0.09 Error 25546.96 109 234.37 Education 867.09 6 144.51 .61 Error 24722.87 105 235.45 Sex 143.37 1 143.37 .61 Error 25446.58 110 231.33 Total 25589.96 111 *Significant at the .05 level or better. Race accounts for a significant proportion of the variance in the case of CST and KPACRM and approaches significance (at the .059 level and not shown on Table 3-7) for CRANDHR. It will be recalled that these are the three tests that in Table 3-2 showed substantial loadings in Factors I and II, respectively the general verbal facility factor and the achievement or knowledge of civil rights information. The Essay test, with its exclusive and heavy loading in Factor III, however, showed no significant F's with any of the demographic variables. Reference to the mean scores in Appendix D reveals that of those tests having a significant or nearly significant 29 proportion of variance accounted for by race, whites had the higher mean scores on CST, the general verbal facility factor. Non-whites scored lowest with non-responders in between. On KPACRM, however, the test with the highest loading in Factor II, the civil rights information test, the non—whites achieved the highest mean scores with the whites scoring lowest and, again, the non-responders in between. On CRANDHR, with its moderate loadings in both Factors I and II, the non-responders scored highest with the Whites and non-whites scoring at about the same level. Significances of the differences between white and non-white mean test scores were calculated for CST, KPACRM, and CRANDHR, and are shown in Table 3—12. Table 3—12. Significance of Mean Test Score Differences of Whites and Non-whites on the CST, KPACRM, CRANDHR and Oral Subtests i. s2 t N CST Whites 30.2 49.00 3.5* 35 Non-whites 25.1 39.69 48 KPACRM Whites 14.2 40.96 3.4* 35 Non—whites 18.3 18.49 48 CRANDHR Whites 24.2 86.49 0.8 35 Non—whites 25.7 44.89 48 ORAL Whites 6.0 182.25 0.7 6 Non—whites 11.6 256 17 *Significant at the .05 level or better. 30 The CST and KPACRM differences are significant while the CRANDHR difference is not. The meaning is clear. Whites performed better on the general verbal facility factor while non-whites performed better on civil rights information. Further inspection of Appendix D shows that the non- whites far outperformed the whites on the Oral test. The mean of the non-whites was 11.6 but that of the whites only 6. The N was too small for the large difference of 5.6 be- tween the means to be significant. Nevertheless, it could be suspected that the Oral largely duplicated KPACRM and possibly CRANDHR. Further evidence on this point is avail- able in Table 3-1 which shows significant correlations between the Oral on the one hand and both KPACRM and CRANDHR on the other. Summary of Results Factor analysis of the intercorrelations among the tests used in the selection battery revealed three factors: a general verbal facility factor, a civil rights knowledge factor, and an-ability—to—write-English factor. The four demographic variables of race, civil service status, education, and sex tend not to be highly inter- related. In relating test performance to demographic variables, the only demographic variable among the four that accounted for a significant proportion of the variance on any of the 31 tests was race. Testing the significance of white vs. non- white mean test scores showed that whites performed better on the general verbal facility test, non-whites performed better on the civil rights knowledge test, while there was no performance difference on the ability to write English test. Non-whites far outperformed the whites on the Oral test, but the N was too small to produce a significant dif- ference. CHAPTER IV CONCLUSIONS AND INTERPRETATIONS The purpose of this research was to explore the internal structure of a selection test battery the first time the test battery was given and before criterion and therefore validity data were available. Such a preliminary exploration could provide guidance with respect to the construction of test batteries to be administered for the next waves of applicants, possibly before stable criterion data were available. Also, this preliminary exploration could provide some notiOn with respect to possible discrimination among sensitive demographic groupings and among the selection devices used; after all, selection had to be based on some sort of predictors even though criterion data could not possibly be available; Another rather unique factor in this situation was that selec- tion was to take place not among lower level personnel but at rather high professional levels. Fortunately for the study a variety of tests were tried out. The types of tests evaluated were those measuring (1) general verbal facility or "G", so often found in civil service selection testing; (2) knowledge of civil rights; (3) knowledge of interviewing and investigating techniques 32 33 common to much government work; (4) ability to write the kind of English used in preparing reports of investigations and the like; and (5) the board oral. Demographic character- istics available for analysis were such sensitive ones as race, civil service status (those on civil service could be more test wise and I'know the rOpes" better), education, and sex. Previous research on white, non-White test performance differences were conflicting. At least one study reported no white, non-White differences among those with some college or better, while others reported that differences persist among the better educated of both races. Previous studies compared up to and including college graduates. This present study was able to extend the ceiling since substantial numbers of subjects had taken graduate work. Findings revealed that the tests used in this battery designed to select from among high level professionals could be fairly well accounted for by three factors; (1) general mental ability or verbal facility or "G"; (2) knowledge or achievement with respect to civil rights; and (3) ability to handle written English. Because of the much smaller number of applicants who took the Oral, the Oral had to be omitted from the factor analysis but was included in the table of intercorrelations among the several tests. The analysis of the interrelationships of the four demo— graphic variables turned up no significant patterns although 34 some such patterns might well emerge more clearly with a larger number of cases. Analysis of the great number of interrelationships among the tests by demographic variables revealed only one demo— graphic variable, race, that accounted for a significant proportion of variance on any of the tests. When mean test score differences were tested for significance, conclusions were that whites outperformed non-whites on "G" type tests but that non-whites outperformed whites on civil rights in- formation. No racial differences were evident on the third factor, ability to handle written English. Performance on the Oral turned out to be highly related to civil rights knowledge. From a practical viewpoint, the results showed that the Civil Service Commission was fortunate and wise in the choice of tests used for their first selection testing with no criterion and of course validity data available. All of the types of test tried out showed no "preliminary bias" against any demographic grouping except race. For the total battery. civil service test wiseness was not a factor, nor was sex or education. It turned out that the one test of "G" favored whites while another test, civil rights knowledge, favored non-whites so that the two tests roughly counterbalanced each other. Results also showed that it was possible to develop a verbal selector test (KPACRM) not loaded with "G" yet also to develop one loaded with both "G" and with a separate) knowledge or achievement component (CRANDHR). 35 The Oral presents a problem for possible correction. The examining board might try to avoid duplicating variance accounted for by other tests in the written battery, in this case knowledge of civil rights (KPACRM and CRANDHR). Another, they could concentrate on assessing for example ability to handle oneself in interpersonal situations, an ability pre- sumably needed by civil rights representatives. Psychologically, the findings of this study have to be aligned with those that indicate that whites perform better than non-whites on "G" type tests, no matter how much educa- tion both racial groups have. The superior performance of whites continues even among those applicants who have gone as far in their education as graduate work. This may be the first study done in an industrial (governmental) selection situation to show that "G" differences among whites and non— whites persist even at the highest level of formal education. 10. 11. REFERENCES Bartlett, C. J. and O'Leary, B. A., A differential model to moderate the effects of the heterogeneous groups in personnel selection and classification, Personnel Psychology, 1969, 22, 1-19. Cronbach, Lee, J., Essentials of psychological testing, 3rd ed., Harper and ROW, Publishers, N. Y., 1970. Kirkpatrick, J. J., gi.ai., Testing and fair employment: Fairness and validity of personnel tests for differ- ent ethnic groups, New York University Press, N. Y., 1968. Dreger, Ralph M. and Miller, Kent S., Comparative psydho- logical studies of Negroes and whites in the United States, Psychological Bulletin, 1960, Vol. 57, No. 5, pp. 361—402. Dreger, Ralph M. and Miller, Kent S., Comparative psycho- logical studies of Negroes and whites in the United States 1959-1965, PsycholOgical Bulletin: Monograph Supplement, Vol. 70, No. 3, Part 2, Sept. 1968. Krug, R. E., Some suggested approaches for test develop- ment and measurement from the Industrial Psycholo- gist: Selection and equal employment opportunity (a symposium) Personnel Psychology, 1966, 19, 24-35. . Moore, C. L., MacNaughton, J. F. and Osburn, H. 6., Ethnic differences within an industrial selection battery, Personnel Psychology, 1969, 22, 473—482. Ruda, E. and Albright, L. E. Racial differences on selec- tion instruments related to subsequent job performance, Personnel Psychology, 1968, 21, 31- 41. Schmidt, Frank, Differential validity vs. Test Bias, December 10, 1970 (mimeo.). Science Research Associates, A study of the job Claims Auditor, Clerical and office data briefs. Wollowick, H. B. 2; gi., Psychological testing with a minority group population, Proceedings of the 77th Annual Convention of the American Psychological Association, 1969, 609—610. 36 APPENDICES APPENDIX A CIVIL SERVICE APPLICATION The information on this part of the application is collected jointly by the Civil Service Commission and the Civil Rights Commission as part of a program designed to assure equal employment opportunity. The information you supply on this sheet will not affect your examination score. Social Security Number Race: White[::] Non-white] I Are you now employed in the‘state classified service? Yes [:3 No 1:] From what source did you first learn of this examination? Please Check One. 1. A state employee you know 2. Civil Service examination announcement 3. Department of Civil Service office 4. State Employment Security office 5. State employer recruitment 6. School referral 7. Newspaper advertisement 8. Newspaper story 9. Radio announcement 10. Television 11. Private organization (civic, fraternal, etc.) 12. Other (please specify) DDUUDDDDDDDU 37 APPENDIX B ANNOUNCEMENT: CIVIL RIGHTS REPRESENTATIVE GENERAL DESCRIPTION An employee in this class, in the central or branch office of the Civil Rights Department, participates in programs for assuring equal opportunity for citizens of the state; and performs related work. EXAMPLES OF WORK Serves as an experienced civil rights representative; assumes responsibility for the more complex or sensitive cases; assists in the training of new representatives. Makes investigations of problems affecting minority groups or of cases of alleged discriminationand pre— pares reports on findings. I Attempts to concillate differences between participants involving claims of discrimination. Investigates incidents involving intergroup tension, and recommends and carries out procedures for handling tense situations. Assists in the training of local officials to handle intergroup tension and conflicts. Serves as speaker or"panel participant in meetings to promote desegregation. Meets with public and private officials to educate and persuade them to take affirmative actions to achieve desegregation in the area of housing, employment, public accommodations and education. Gathers information for research studies concerning minority groups. Keeps informed of develOpments in the field. Participates in professional meetings and conferences. EXPERIENC§_AND EDUCATION REQUIREMENTS Education *Possession of a bachelor's degree in the social sciences from an accredited college. *Minimum requirements--degree, experience. 38 39 Additional credit will be given for possession of a master's degree in the social sciences. Note: One additional year of the experience described below (see 2) may be substituted for one year of col- lege up to a maximum of two years. Experience 1. *Two years of experience as a Civil Rights Repre- sentative 2. or, *Three years of experience in community organi- zation, social work, counseling and guidance, educa- tion, investigations dealing with peOple, or other work dealing in human relations, preferably in civil rights programs. Note: Performance of duties should have included conference planning and participation and/or depth investigation, mediation and negotiation, as well as the preparation of detailed and comprehensive reports, case histories or briefs. OTHER REQUIREMENTS Physical condition adequate for performance of the work. Tact and similar qualities necessary in meeting and dealing effectively with others. Willingness to participate in inservice training programs. Personal commitment to the fundamental objectives of the Civil Rights Department programs. Impartial attitude and ability to maintain it in the investigation and conciliation of civil rights problems. Knowledge of the principles of psychology, sociology and human relations. Knowledge of individual or group counseling. Knowledge of social investigation. Knowledge of state, federal and local statutes, orders, court rulings and prOgrams affecting the civil rights field. *Minimum requirements——degree, experience. 40 Knowledge of the sociological and economic forces which affect the welfare of minority groups. Knowledge of the resources which can be employed to assist in meeting problems relating to minority groups. Knowledge of the practiCes used in the sale, rental and financing of homes. Knowledge of labor and management employment practices and policies. Knowledge of the techniques of effective investigation and conciliation of cases involving alleged discrimina- tion. Knowledge of the techniques used in easing intergroup tension and conflicts. Ability to use good English and to write and speak effectively. 5 Ability to interpret pOlicies and regulations uniformly. Ability to work well with individuals, committees, and organizations. Ability to speak before groups, conduct meetings and negotiate settlements on civil rights complaints. APPENDIX c CROSS-RELATIONS AMONG NUMBERS OF SUBJECTS IN EACH OF THE FOUR DEMOGRAPHIC VARIABLES CIVIL SERVICE STATUS State employees Non-state employees Non—responders EDUCATION 19 yrs. school 18 l7 16 15 14 Non-responders SEX Males Females 41 RACE 0) H a) 'U 3’] C: . u 8. a: E 3 3 ’7‘ ’f‘ "-1 C: C: S o 9 Z Al 13 25 18 20 1 4 3 28 l 3 6 4 16 16 ll 14 9 6 ‘2 11 3 6 3 l 29 31 21 6 17 8 CIVIL SERVICE STATUS m m m 0 H >1 0 O G H G O: 00) O 5 #0) Q. m1 0 m «no H y 1.4 I m 23¢: G # (as o (0 Z Z 1 3 5 5 8 20 15 14 9 6 9 3 4 4 2 3 l 25 28 28 13 ll 7 Males SEX Females n—awxuomw APPENDIX D MEAN SCORES AND STANDARD DEVIATIONS X. s N For CST Race Whites 30.2 7.0 35 Non-whites 25.1 6.3 48 Non-responders 28.9 5.1 29 Total 27.7 6.6 112 Civil Service Status State employees 27.4 6.2 38 Non-state employees 27.3 7.5 39 Non-responders 28.5 6.0 35 Total 27.7 6.6 112 Education 14 25.2 7.7 9 15 28.4 5.2 16 16 27.8 5.8 29 17 27.2 7.2 43 18 29.5 7.5 l3 19 33.0 0.0 1 No response 28.0 0.0 1 Total 27.7 6.6 112 Sex Males 28.2 7.0 81 Females 26.5 5.2 31 Total 27.7 6.6 112 For KPACRM Race Whites 14.2 6.4 35 Non-whites 18.3 4.3 48 Non—responders 16.6 5.2 29 Total 16.6 5.5 112 Civil Service Status State employees 16.6 5.4 38 Non-state employees 16.7 6.0 39 Non-responders 16.4 5.2 35 Total 16.6 5.5 112 continued 42 43 X N Education 14 13.6 4.7 9 15 17.5 4.5 l6 16 15.5 5.9 29 17 16.8 6.0 43 18 19.3 3.1 13 19 19.0 0.0 1 No response 14.0 0.0 1 Total 16.6 5.5 112 & Males 16.3 81 Females 17.4 31 Total 16.6 112 For CRANDHR Race Whites 24.2 9.3 35 Non—whites 25.7 6.7 48, Non—responders 28.8 6.9 29 Total 26.0 7.8 112 _Civil Service Status State employees 25.5 7.5 38 Non-state employees 25.0 7.9 39 Non-responders 27.7 7.9 35 Total 26.0 7.8 112 Education 14 25.0 5.5 9 15 26.4 8.3 l6 16 24.0 7.8 29 17 26.5 8.2 43 18 29.8 7.0 13 19 20.0 0.0 1 No response 26.0 0.0 1 Total 26.0 7.8 112 $42!. Males 25.7 7 9 81 Females 27.0 7 7 31 Total 26.0 7.8 112 continued 44 X s N For I and I Race Whites 22.3 6.3 35 Non—whites 20.7 7.0 48 Non—responders 21.4 5.4 29 Total 21.4 6.4 112 Civil Servige Status State employees 22.7 5.8 38 Non-state employees 20.2 7.3 39 Non-responders 21.2 5.7 35 Total 21.4 6.4 112 Education 14 21.6 5.6 9 15 22.6 7.2 16 16 22.3 6.4 29 17 19.6 6.8 43 18 23.7 3.3 l3 19 20.0 0.0 1 No response 21.0 0.0 1 Total 21.4 6.4 112 £85 Males 21.4 6.2 81 Females 21.3 7.0 31 Total 21.4 6.4 112 For the Essay Race Whites 19.3 5.2 35 Non-whites 18.8 3.8 48 Non-responders 20.0 3.8 29 Total 19.3 4.3 112 Civil Service Status State employees 19.7 4.7 38 Non-state employees 18.2 4.4 39 Non-responders 20.1 3.5 35 Total 19.3 4.3 112 continued 45 X s N Education 14 17.9 4.9 9 15 18.0 6.2 16 16 19.5 2.8 29 17 19.5 3.5 43 18 19.8 5.6 13 19 28.0 0.0 1 No response 25.0 0.0 1 Total 19.3 4.3 112 5&3; Males 19.6 3.8 81 Females 18.6 5.4 31 Total 19.3 4.3 112 For the Total Weighted Score Race Whites 33.5 15.2 35 Non-whites 33.6 12.7 48 Non-responders 38.2 10.7 29 Total 34.7 13.1 112 Civil Service Status State employees 35.4 13.7 38 Non-state employees 32.1 13.8 39 Non-responders 36.9 11.5 35 Total 34.7 13.1 112 Education 14 31.1 8.5 9 15 35.6 14.0 16 16 32.7 14.0 29 17 34.7 13.0 43 18 39.4 13.8 13 19 45.0 0.0 1 No response 45.0 0.0 1 Total 34.7 13.1 112 15.91 MaleS‘ 34.4 13.1 81 Females 35.6 13.4 31 Total 34.7 13.1 112 continued 46 X s N For the Orai Race Whites 6.0 13.5 6 Non—whites 11.6 16.0 17 Non-responders 9.2 15.4 8 Total 9.3 15.2 31 Civil Service Status State employees 9.8 15.8 11 Non-state employees 9.5 15.4 11 Non-responders 8.4 14.6 9 Total 9.3 15.2 31 Education 14 3.4 10.3 1 15 10.1 15.6 5 16 10.8 16.4 9 17 8.8 15.4 ll 18 9.6 15.0 4 19 29.0 0.0 1 No response -- -- -- Total 9.3 15.2 31 £9.11 Males 8.6 14.7 21 Females 11.1 16.5 10 Total 9.3 15.2 31 '33” ‘2 4 1.9.71 "'TITififlflfilLTlllllfiifnlfiUflflTflIfliflMflTfllfll’t‘f“