VALIDITIES o; A PERSONAL HISTORY FORM FOR AUTOMOBlLE SALESMEN IN GENERAL commas WITH suavmmnas OF AUTOMOBELE SALESMEN Thai: for the Dame 0% Ph. D. MICHQGAN STATE UNNERSITY James Edward Kennedy 1956 Tweets This is to certify that the thesis entitled i Validities of a Personal History Form for Automobile Salesman in General Compared with Subvarieties of Automobile Salesman presented by James E. Kennedy has been accepted towards fulfillment of the requirements for Ph 0 D o PsyChOIOgy degree in 23%” Major professor Date November 16, 1956. 0-169 VALIDITIES OF A PERSONAL HISTORY FORM FOR AUTOMOBILE SALESMEN IN GENERAL COMPARED WITH SUBVARIETIES OF AUTOMOBILE SAIESMEN By JAMES EDWARD KENNEDY AN ABSTRACT Submitted to the School of Advanced Graduate Studies of Michigan State University of Agriculture and Applied Science in partial fulfillment of the requirements for the degree of DOCTOR OF PHILOSOPHY Department of Psychology 1956 Approved %Lanaa , QJ W JAMES E. KENNEDY ABSTRACT Purpose. The purpose of this study was to compare two methods of constructing a selection instrument for use with car salesmen. One, an "Over-all Method" assumed the sample of car salesmen studied were suf- ficiently homogeneous to warrant treatment as a single group in the course of developing the instrument. The other, a "Subvariety Method," assumed sufficient differences among subvarieties of salesmen within the sample to warrant unique treatment of each in the course of developing the instrument. Predictors. Two-hundred and ninety objective type items were used as trial predictors. These could be classed as: personal data or bio- graphical, personality, attitude, or interest. The items were divided and two trial forms of the questionnaire were prepared; Form A and Form B. Those items which were found to be most promising in the item validity analyses were used to prepare Form C. Criterion Analysis. Five measures of job performance were consid- ered. Two were eliminated as impractical after a pilot study; three were collected for the item analysis sample. High intercorrelations among the three measures suggested little would be gained from using a composite criterion. Earnings over a standard time period was chosen as the sole criterion. Population and Samples. The car salesmen investigated were em- ployed in car dealerships throughout the country. Form A and Form B were administered to salesmen from Samples I and II respectively. Those questionnaires were used for the item validity analyses and for develop— ing scoring keys. Form C was administered to Sample III for validation JAMES E. KENNEDY ABSTRACT and cross validation. The following numbers of questionnaires were available for analysis: Form A, 358; Form B, 335; Form C, 7A9. Subvarieties of Salesmen. Eight subvarieties of salesmen were originally considered. The limited size of Sample III required a re- vision of the design; only six subvarieties were actually explored. An Analysis of Subvarieties was conducted to determine which pair of sub- varieties could most defensibly be eliminated. Item Validity Analysis. One item validity analysis was conducted for Form A items considering all Sample I salesmen as a single group; six item validity analyses were conducted considering each subvariety from Sample I as a separate group. This was repeated for Form B using Sample II. The A0 items which had been found to be most predictive from each of the seven analyses were identified. Scoring keys were de- veloped for the items. This resulted in an Over-all Key and six dif- ferent Subvariety Keys each with ho items. Validation and Cross Validation. Sample III was subdivided into 1h sub-samples. Seven of the samples were used for estimating the va- lidities of the seven scoring keys; seven were used for estimating the cross validities. The relationships between the predictors and the cri- terion were analyzed in four ways. Results and Conclusions. The four methods of analyses on the whole gave consistent results. No significant differences were obtained in the predictive efficiencies of the two kinds of Keys. It was concluded that the less laborious Over-all Method was as efficient as the Subvariety Method. VALIDITIES OF.A PERSONAL HISTORY FORM FOR AUTOMOBILE SALESMEN IN GENERAL COMPARED WITH SUBVARIETIES 0F AUTOMOBILE SALESMEN By JAMES EDWARD KENNEDY A THESIS Submitted to the School for Advance Graduate Studies of Michigan State University of Agriculture and Applied Science in partial fulfillment of the requirements for the degree of DOCTOR OF PHILOSOPHY Department of Psychology 1956 (A/ ,1. .4/5V7 1.- ‘ G» #5“ 3 3\ TABLE OF CONTENTS List of Tables 11 List of Figures in List of Appendices ‘ iv Acknowledgement v Vita Vi Chapters I. Background and Purpose 1 II. Field Interviews 5 III. Subvarieties of Salesman 7 IV. Trial Predictors 9 V. Population and Samples 13 VI. Criterion Analysis 17 VII. Analysis of Subvarieties 30 VIII. Item Validity Analysis 38 IX. Validation and Cross Validation Results 146 X. Sumnary and Conclusions 56 XI. Some Limitations of mis Study 57 Appendix ' 58 Biblingaphy' 87 LIST OF TABLES Table No. I. Intercorrelations among Criteria Based on SMS Cases from the Item Analysis Sample II. Intercorrelations among the Three Criterion Measures Computed Separately for Selected Subvarieties III. Correlations between Tenure and Earnings for all 5A5 Cases and Selected Subvarieties IV. Numbers of Items Significant (PEZ.20) for Various Com- parisons in the Analysis of Subvarieties - V. Differences in the Biasing Effects due to Low Theoreti- cal Frequencies in the Four Pairs of subvarieties in the Analysis of Subvarieties VI. Number of Significant (PZ..20) Items Observed in the Item Validity Analysis VII. Subdivision of Sample III by Subvarieties VIII. Validity and Cross Validity Coefficients for Various Scoring Keys IX. Indices of Forecasting Efficiency for "A" Samples and "B" Samples Using Optimal Cutting Scores from "A" Samples X. Chi Squares Between Observed Frequencies from "A” Samples and Observed Frequencies from "B" Samples Using Optimal Cutting Scores from "A" Samples and the Observed Frequencies from "A" Samples as Frequencies Expected in the "B" Samples. XI. validity Coefficients Using "A" and "B" Samples Combined ii 23 2h 26 35 1*9 SO 51 LIST OF FIGURES Piggy No. l. 2. Relationship between Earnings and Tenure for Car Make A New-Car Salesmen Relationship between Earnings and Tenure for Car Make B Used-Car Salesmen Relationship between Earnings and Tenure for Sample III iii 27 28 1+7 LIST OF APPENDICES Appendix No. A. B. C. L. M. Examples of Personal Data or Biographical Type Items Examples of Personality Type Items Examples of Interest Type Items Examples of Attitude Type Items Criterion Form Used with Samples I and II Criterion Form Used with Sample III Illustrations of Procedures for Eliminating Bias due to Low Theoretical Frequencies Empirical Evidence to Support the Decision to Limit the Over-all Key to the Forty Most Promising Items Illustration of Procedure Used in Selecting Items for the Sub- variety Keys, e.g. the New-Car Salesman Key Conventions Used in Assigning Score Values to Item Alternatives Procedure Used for Drawing Sub-samples from Sample III A Comparison of the Validities of Early and Late Returned Question- naires Raw Score Distributions for "A" and "B" Samples iv ACKNOWIEDGEMENT The writer would like to acknowledge the help of his advisor Dr. James S. Karslake. He is also indebted to Dr. Orlo L. Crissey for permission to use the data upon which this study was based and his generous help and guidance in a variety of ways. He would like to thank the members of the P.E.S. Staff for their many helpful sugges- tions. JAMES EWARD KENNEDY candidate for the degree of Doctor of Philosophy Final Examination, November 10, 1956, 10:00 A.M. Psychology Department Dissertation: Validities of a Personal History Form for Automobile Salesman in General Compared with Subvarieties of Automobile Salesmen Outline of Studies Major subject: Industrial Psychology Biographical Items Born, June 21, 1921, West New York, New Jersey Undergraduate Studies, Fordham University, 1939-1918 Graduate Studies, Michigan State University, 19h8-1956 Experience: Graduate Assistant, Michigan State University, 19118- 1951. Research Associate, Roger Bellows and Associates, 1951-1952. Senior Staff Specialist, General Motors Institute, 1953- Member of American Psych010gical Association -.vi CHAPTER I BACKGROUND AND PURPOSE Introduction The study reported here is one on sales selection in the dealer- ships of two divisions of a large corporation. The two divisions manufacture and sell automobiles. Within this framework, the purpose of the study was (a) to develop the best possible over-all test for selection of automobile salesmen in general, (b) to develop the best possible tests for the specific se- lection of particular subvarieties of salesmen within the group, (c) to examine the validity and predictive effectiveness of each of the several tests, and (d) to consider the relative worth of constructing over-all versus constructing specific tests for the selection of automobile salesmen. Part (d) of this purpose may be restated in terms of the null hy- pothesis: 0n comparing over-all with specific tests for the selection of automobile salesmen, any differences in validity and predictive value may be accounted for by chance. Literature Survey In the thirty year period from 1925 to 1955 a large number of ar- ticles have been published concerning the research on the selection of salesmen. For the most part the literature proved to be of only lim- ited value. Only one study (12) was found that was directly concerned with the selection of retail car salesmen and it was of little consequence. Since there appears to be no evidence to indicate how similar or dis- similar retail car selling is to other types of selling, results from research done with other types of salesmen had to be considered with caution. Particular attention was paid to trends among a number of studies in an effort to make generalizations with implications for the problem under study. The procedure was not very rewarding. Two problems contributed to- ward making this a difficult task. One, the methodology used in many of the studies did not meet the current day, minimum standards of sound personnel research. Insufficient detail in research reports, small sam- ples, questionable use of statistics, and absence of unbiased validation were more the rule than the exception. Although no actual count was made, it is estimated that something over one half of the published studies are rendered meaningless because of the absence of cross valida- tion, or other suitable replication, in research designs that we now know require such replication. A second problem limiting the value of the reported research was the uneven distribution of studies across a number of different kinds of sales jobs. By far the most and the best research has been done on the selection of life insurance salesmen. Although this provides Opportu— nity for some kinds of generalizations it limits other kinds. Three generalizations seemed tenable: l. The over-all weight of the evidence seemed to indicate that it has been possible to predict success of salesmen on a number of dif- ferent kinds of sales Jobs by using paper and pencil selection devices. 2. There appeared to be no conclusive evidence that any parti- cular items, factors, tests, or inventories had consistent validity for a variety of different types of sales jobs. This is not to say that from our general knowledge of the results of selection research on a number of different kinds of jobs in many occupations it would not be reasonable to hypothesize a certain amount of communality among the personal attributes associated with success on different kinds of sales Jobs. It is to say that empirical evidence supporting this view is meager. 3. The validity of a selection device for salesmen appeared to be a highly specific thing. Not only might it be specific for "specific" types of sales jobs, such as life insurance selling compared to retail department store selling, but it also might be specific to subvarieties of salesmen within these relatively specific types of sales jobs. Support of this last generalization follows. Bolanovich and Kirk- patrick (A), Kbrnhauser and Shultz (lO), Stokes (11), and Husband (8), have suggested, upon reviewing research done on the selection of sales- men, that consideration of the specific nature of each type of sales job should yield improvement in validity. Fairly typical of these views are Husband's comments that "The writer is convinced that sales selec- tion will have to be like other selection involving motor skills, namely specific. Each type of selling and also each type of product and serv- ices sold will require independent analysis with individual selection adapted to its particular requirements." A series of scattered studies further suggested that the "speci- ficity" of this validity may be more extreme than it is usually thought. Bills (3), working with the Strong Vocational Interest Blank in predic- ting success of insurance agents, showed differences in validities for different subvarieties of insurance salesmen based upon (a) previous experience compared to no previous experience at selling life insur- ance, (b) age and (c) preferences for secondary occupations. Else- where, Bills (2) reported differences for subvarieties based upon amount of education. Wallace and Twitchell (it), also working with life insurance agents and using the Aptitude Index, showed varying va- lidities for subvarieties based upon (a) type of remuneration (salary compared to commission), and (b) the applicant's stated monthly income requirements (an item on the Aptitude Index). Vallace (13) also re- ported differences for subvarieties based upon age. Chapple and Donald (5) and Anderson (1), working with retail department store sales clerks claimed, with no supporting evidence however, different prediction pat- terns for various sub-types of sales clerks based upon specific situa- tional factors. . It is repeated for emphasis, that this series of studies was only suggestive of the highly specific nature of the validity of selection instruments. In no case were the reported differences tested for sta- tistical significance; in several of the studies the differences were not quantified. No studies were reported in which possible differences between sub- varieties of salesmen were exploited in the development of a selection device for the purpose of enhancing the validity of the predictor. CHAPTER II FIEID INTERVIEWS At the outset of the project it was felt that much might be gained from calling upon the knowledge and opinions of experienced personnel actually engaged in the car selling business. Hence, preliminary inter- views were conducted with selected personnel from the retail and whole- sale organizations. The interviews were conducted by six psychologists from the research unit of an automotive manufacturer sponsoring this study. All had been thoroughly briefed on the tentative research design. The purposes of the interviews were: 1. To get suggestions for possible measures of job performance for use as criteria. 2. To get suggestions for possible trial predictors. 3. To provide the investigators with an opportunity to become gen- erally more familiar with the retail car selling business. A fourth purpose, and one that does not directly concern this study, was to get the personnel from the wholesale organizations ego-involved since they were to play an important role in the collection of the data and would ultimately implement the use of the final selection instrument if it were recommended for use at the dealerships. The interviews were semi-structured and between one—half and one hour and a half long. The first ten minutes were spent making a general presen- tation of the purpose of the study, the possible design to be used, the 5 anticipated results, and the way such results might be used by the dealers in selecting new salesmen. The remaining time was spent eliciting sugges- tions for criteria and predictors as well as getting information about the business in general. The results of the interviews have been summarized in the chapters to which they are most related. These are Chapter III, subvarieties of Sales- men; Chapter IV, Trial Predictors; and Chapter VI, Criterion Analysis. 0\ CHAPTER III SUBVARIETIES OF SALESMEN Four pairs of subvarieties were considered for separate treatment in the study. Two of the pairs of subvarieties were based upon differ- ences in the product, one pair on the basis of a personal factor, and one pair on the basis of a situational factor. A definition of these subvarieties and the rationale for their inclusion follows. 1. New-car salesmen and used-car salesmen. A substantial number of the operating personnel contacted in the field interviews expressed the opinion that the attributes associated with success at selling new cars were different from those for selling used cars. They suggested that customers buying new cars were a different "class" of people from those buying used cars. Also that different selling techniques were used for the two products. New—car salesmen were distinguished from used-car salesmen for the purposes of the Subvariety analysis in the same way they were classified for the criterion purposes (cf. Chapter VI). 2. Car Make A and Car Make B salesmen. A number of the personnel in- terviewed also suggested that Car Make A salesmen were quite different from Car Make B salesmen. Although the prices of the two cars overlap consid- erably, Car Make B models were, on the average, more expensive than Car Make A models. There were also relatively wide differences in the poli- cies and practices of the wholesale organizations for the two Divisions. 3. Younger salesmen and older salesmen. These subvarieties were con- sidered because of their demonstrated relevance in several sales selection research studies reported in the literature (cf. Chapter I). Younger salesmen were defined as those below the median age for the total item analysis sample; older salesmen were those above the median age. There was no significant difference in the median age for Car Make A salesmen compared to Car Make B salesmen. h. Salesmen from larger volume dealerships and salesmen from smaller volume dealerships. The research personnel who conducted the field in- terviews suggested that there appeared to be fairly large differences in the selling practices, administration policies and procedures, and gen- eral "atmosphere" of the larger volume dealerships compared to the smaller volume dealerships. These apparent differences suggested possible dif- ferences in the kind of salesmen who would be successful at the two types of dealerships. The median sales volume was determined for Division A dealerships and for Division B dealerships. Since the medians were quite different, the assignment of dealerships to the "larger volume" or "smaller volume" categories was made separately for Division A and Division B deal- erships. CHAPTER IV TRIAI.PREDICTORS Sources General ideas for items used in the trial version of the Personal History Form were derived from two general sources: survey of the lit- erature of sales selection research and from the field interviews. 1. From a review of the literature of sales selection research it was concluded that three general types of items were most frequently reported as useful for predicting job success of salesmen. These types of items were: (a) personal history or biographical, (b) interest, and (c) personality. 8.. Personal history or biographical. These items for the most part were concerned with information usually obtained from compre- hensive application blanks, for example: age, education, pre- vious work experience, dependents, etc. Interest items reported as useful were most frequently those from the sales keys of the Strong Vocational Interest Blank. Personality items most frequently reported as predictive were those purporting to measure: sociability, extroversion, ag- gressiveness, emotional stability, self-sufficiency or self- confidence, and dominance. From field interviews it became apparent that measures of at- titude and belief were of interest. The field interview ex- perience permitted the writing of items that were specific to 10 the beliefs, attitudes, and practices of retail car salesmen that oth- erwise could not have been written. Furthermore, it was felt that as a result of the general familiarity with the practices, mores, and vernacular of the car selling business, it was possible to write more realistic items which were less likely to be annoying to the popula- tion to which they would be administered. Types of Items - Format Two-hundred and ninety items were used as trial predictors. In order to limit the administration time of the trial form of the ques- tionnaire, the items were divided between two forms. These were re- ferred to as Form A and Form B. All items were of the multiple choice type. Four-fifths of the items required the testee to choose one alternative. The number of al- ternatives for this type of item ranged from two to nine. One-fifth of the items required the testee to rank all alternatives in order of pref- erence. This type of item always had four alternatives. Types of Items - content. For convenience of discussion the types of items used may be classified into four types: (a) personal history or biographical, (b) personality, (c) interest, and (d) attitudes. a. Personal history or biographical. This type of item comprised 33 percent of the total. Some of the items were specifically written to tap suggestions obtained in the field interviews. The majority of these items were taken from standard applica- tion blanks and written in the form of multiple choice items or were taken from standard biographical data inventories developed elsewhere. These items were concerned with areas such as: C. II 'physical characteristics, education, previous work history, early environmental influences, activities or achievements, family, health, finances, and housing. Examples of items for each of these areas are found in Appendix A. Personality. This type of item comprised #3 percent of the to- tal. Many of these items were adaptations of items taken from scales that had been reported as predictive of successful per- formance of salesmen in previous research. Others were adapted from scales that purported to measure personality dimensions that were hypothesized by the investigators to have potential for prediction. The majority of the items used could be arbi- trarily classified into the following areas: optimism, personal relations, impulsiveness, introversion, dominance, self- confidence and self-sufficiency, argumentativeness, sociability, and emotional stability. Another type of item was concerned with a number of different aspects of personality within the same item. Examples of each of these types of items are found in Appendix B. Interests. This type of item comprised ten percent of the total. With few exceptions they were modified items from the sales keys of the Strong Vocational Interest Blank. These items were con- cerned with two areas of interest: interest in various kinds of jobs and interest in various types of non-Job activities. Examples of these two types are found in Appendix C. Attitudes and knowledge. This type of item comprised 1h percent of the total. A minority of the items were modified from sel- ection devices that had been used elsewhere. The majority of 12 the items were written from knowledge acquired about the job of retail car selling gained from the field interviews. These items could be classified into the following areas: work in general, selling in general, salesmen in general, car selling and car salesmen, and knowledge of good car selling practices. Examples of each of these types are found in Appendix D. CHAPTER V POPULATION AND SAMPLES The Population This research was conducted for two Divisions of a large auto- motive corporation. For our purposes, these Divisions will be referred to as Division A and Division B and their products as Car Make A and Car Make B. Each Division distributes its products through a network of dealerships located throughout the country. With few exceptions these dealerships are independently owned and Operated. No dealership handles both makes of cars. The size of the dealerships vary. The largest employs about AO full time salesmen and the smallest only one. This study is concerned with dealerships employing at least four full time new-car salesmen or four full time used-car salesmen. The Sample. The salesmen used in the study were those who were employed at dealerships selected in the following way. For administrative purposes the wholesale organizations have sub- divided the country into 25 geographical zones and have assigned deal- erships to zones according to their locations. Alphabetical lists of these zones and the dealerships assigned to them were prepared for Division A and Division B.1 All dealerships in the first, third, fifth, etc. zones on the "A" list were designated as Group I dealerships; 1In the case of Division B this list was limited to dealerships that had participated the previous year in a national sales contest. They comprised approximately two-thirds of the total Division B dealer- ships employing four or more salesmen. 13 1A those in the second, fourth, and sixth, etc. zones were designated as Group II dealerships. All dealerships in the second, fourth, and sixth, etc. zones on the "B" list also were designated as Group I dealerships; those in the first, third, and fifth, etc. zones also were designated as Group II dealerships. The name of each of the dealerships from Group I and II was type- written, separately on a three by five index card. The cards from each zone were shuffled and one-third of the dealerships from each zone were randomly drawn and designated as Group III.1 The number of salesmen employed at the dealerships in Groups I, II, and III were counted. The numbers of salesmen in the three groups were slightly different, hence, dealerships were randomly reassigned from one group to another until the totals were approximately the same. Copies of Form A.were mailed to all dealers in Group I and Form B to all dealers in Group II. Dealers were requested to distribute the questionnaires to all of their full-time salesmen. Return envel- opes were provided and salesmen were requested to mail the completed questionnaires directly to the sales departments of the two Divisions. Salesmen were requested to identify their questionnaires by name. At the same time the Personal History Forms were mailed, the cri- terion forms were sent to dealers. The dealers also were requested to return the forms directly to the sales departments of the two Divisions. The number of salesmen questionnaires that were returned and were usuable was far below expectations. Some of the factors contributing 1Group III was subsequently sub-divided into a number of sub- samples. This is further explained in Chapter IX. 15 to this were: 1. Salesmen simply did not return their questionnaires despite four followaup efforts to get them to do so. 2. Salesmen returned their questionnaires but did not provide their names or left many items incomplete. 3. Dealers did not return their criterion forms. This rendered all salesmen questionnaires from that dealership useless. h. Salesmen did not have the required tenure for inclusion on the criterion form. 5. Salesmen listed on the criterion form were no longer in the employ of the dealer and hence could not complete their questionnaires. 6. Only three salesmen were listed on the criterion form for some dealerships. All three cases were lost as a result of this since four salesmen had been set as the minimum for dichotomizing the ranks into upper and lower halfs. This is explained more fully in Chapter VI, Criterion Analysis. After 18 months from the time the questionnaires were originally sent out, it was decided not to wait any longer. The total of the usable cases at that time was 358 for Form A and 335 for Form B. No effort was made at the item analysis stage to estimate the pos- sible biasing effect of only using salesmen's questionnaires that were returned.1 Salesmen from Group I and II were designated as the "item analy- sis sample." After the completion of the item analyses which had been 1An effort was made in the cross validation stage to estimate this possible biasing effect. (c.f. Appendix L.) conducted on Group I and II, Form C of the Personal History Form was prepared. Form C consisted of the items which had been found in the item analyses to be most promising. Form C was administered in the same manner as Form A and B, to sub-samples from Group III for the purposes of validation and cross validation. CHAPTER VI CRITERION ANALYSIS Suggestions for a number of possible criteria of job performance were obtained from the review of the literature and from the field in- terviews. Several factors rendered many of these possibilities imprac- tical. Some factors limiting the use of some criteria are listed below. 1. Salesmen used in the study were employed at over AOO widely scattered locations throughout the entire country. Collection of cri- terion data by the investigators personally visiting these dealerships was judged to be impractical. This eliminated many possible criteria, among which were work sample measures. The only alternative available for collecting the criteria data was to mail requests for the data to the dealerships. This in turn meant that in order to have assurance that the data would be uniformly collected, the explanation of the criteria and how they should be collected had to be relatively simple and brief. 2. At an early stage it became apparent that there was not equal op- portunity to sell from one dealership to another. Salesmen selling at a well established dealership with good local advertising, good management, good service facilities, etc. had a distinct advantage over salesmen sell- ing at an opposite kind of dealership. The possibility of equating all dealerships for opportunity to sell would have required, if done ade- quately, a market research project of unreasonable proportions. This prob- lem was resolved by using criteria based upon comparisons of salesmen within 17 the same dealership and hence minimizing any differences in selling op- portunity. 3. During the early stages of the field work it was learned that salesmen who either exclusively or predominately sold new cars would have to be considered separately with regard to criteria of job perfor- mance from those who exclusively or predominately sold used cars. New- car salesmen's earning, for example, were substantially higher on the average than were used-car salesmen's. This problem was handled by classifying all salesmen within each dealership as new-car salesmen or used-car salesmen. Criterion compari- sons were then always made within each group. The wholesale organizations had a convention for defining a salesman as a new or used-car salesman and that convention was used. If 60 percent or more of the cars a sales- man sold during a given period were new cars he was considered to be a new-car salesman. All others were considered to be used-car salesmen. Criteria Investigated From the field interviews five criterion measures were suggested that seemed to warrant further consideration. These are discussed below. 1. Number of units sold. The most frequently offered suggestion was simply the number of cars a salesman sold over a given period of time. The following problem arose in connection with this criterion. Salesmen sold both new and used cars. A decision had to be made as to whether the sale of a used car should be equal to that of a new car, and if not, what should be the fractional value of a used car sale. At the time of the study the wholesale organizations valued the sale of a used car 70 per- cent as much as that of a new car. This permitted the computation of an 19 index for each salesman that could be used as a joint expression of both the number of new'and the number of used cars sold. 2. Gross sales volume. It was suggested that the number of cars sold was a relatively gross measure of proficiency since it did not con- sider the ranges of prices for various models of cars nor did it take into consideration the sales of optional equipment and accessories. The gross dollar volume of sales for a given period of time was felt by some per- sonnel to reflect better these additional factors. 3. Earnings. There were a number of different remuneration systems in use at the dealerships. It was argued that despite these differences each system was the one that the dealer believed would most equitably re- flect differences in performance of his salesmen. Dealers with a rela- tively short-ranged viewpoint might put more emphasis on commissions and those with longer-range viewpoints might emphasize salary. In either case the choice would be the one which the dealer felt was most appropri- ate to his over-all operation. A. Profit. It was reasoned that the fundamental purpose Of the dealership was to Operate at a profit and that the best criterion of a salesman's performance was the amount of profit his efforts returned to the dealership. 5. Ratings. It was believed by some that the best basis for eval- uating salesmen's performance was simply to have the Dealer or Sales Man- ager rate his salesmen on whatever bases he thought were most appropriate. Pre-test of the Criterion Form A tentative set of directions was prepared for use by the dealers in providing the necessary information on the five above described measures 20 of job performance. Prior to committing the form to the field on a full scale basis, it was decided to conduct a pilot study to evaluate the clarity of the instructions and the practicality of getting the required information. Four fairly representative dealerships were chosen for the pilot study. The forms were sent by mail to the dealers with the cover letters that would be used in the main data collection. Upon completion of the form, each dealer was visited by one of the research personnel in order to get the reactions and suggestions of the bookkeeping personnel and the supervisory personnel who had used the form. From this experience two conclusions were reached. 1. The measure of profit was eliminated from fUrther consideration as a measure of job performance. The amount of clerical time required to process the individual sales transactions of each salesman was too de- manding on the limited bookkeeping facilities of the dealers. The direc- tions that had been used for obtaining the profit measure had been devel— oped in consultation with the accounting personnel of the wholesale or- ganization; no simpler approach to the problem was seen that could still provide a meaningful measure. 2. The measure of gross dollar volume of sales was found also to be relatively difficult to obtain from the records. This caused a certain amount of apprehension among the investigators. If the directions were not clear and the task of collecting the data not something that could readily be done, it was feared that dealers would return incomplete or incorrect forms. As will be seen later, this apprehension was well founded. In order to be able to defend the use of this measure, at the expense of possible loss of cases for the analysis, a check was made to determine its 21 relationship with the other criteria. A correlation was computed be- tween the measure of gross dollar sales and the measure of gross earn- ings in the following way. New-car salesmen were ranked within each dealership according to the size of their earnings and then according to their gross dollar sales. Each set of these ranks within dealerships was then dichotomized. This procedure was repeated for used-car sales- men. The new and the used-car salesmen, Al in all, were then assigned to a four-fold table in order to compute a tetrachoric correlation be- tween the two measures. It was found that all salesmen who were in the upper half of their dealership on earnings were also in the upper half on gross sales volume; all who were low on earnings were also low on gross sales volume. On the basis of this finding it was decided not to include the measure of gross sales volume as a criterion measure. Criteria measures collected on dealer groups I and II. A seven month time period was selected over which the criterion data were col- lected for groups I and II. The instructions to the bookkeeping departments for the recording of the number of cars sold and the gross earnings were straightforward and warrant no detailing here. The instructions given to the Dealers or Sales Managers for making the ratings are reproduced in Appendix E. In summary, the instructions were as follows: The rater's attention was called to the number of cars each salesman had sold and to his earnings for the criterion period. He was told that either of these measures could be considered as a basis for measuring salesmen's performance. Then he was told that the ratings of the Dealer or Sales Manager could also serve as a basis. He was then asked to rank his salesmen from best to least effective. The instructions 22 read: We would like you to make these judgments on WHATEVER BASIS YOU THINK BEST. YOu may want to consider the information in C01- umns B through D (units sold and earnings) or you may not - that's up to you. You may or may not want to consider things like co- operation with other salesmen, public relations, quality of deals, cooperation with dealer programs or campaigns, etc. Processing the criterion data. The number of new cars and the number of used cars sold by each salesman was converted to a single index. This was called a "units index" and was the number of new cars sold multiplied by ten added to the number of used cars sold multiplied by seven. New- car salesmen within each dealership were then ranked according to the size of this index; ties were handled in the conventional manner. New- car salesmen were ranked then according to the size of their earnings. This procedure was repeated for used-car salesmen. Statistical relationships among the criteria. In order to gain some understanding of the relationships among the three criterion measures, an intercorrelation matrix was prepared. The correlations were not based on the entire item analysis sample; only the first 5A5 complete cases were used. The new-car salesmen within each dealership were dichotomized on each criterion measure into those who were ranked among the upper half and those ranked among the lower half of the salesmen within a dealership. The middle case in dealerships with an odd number of salesmen was classi- fied into one or the other categories according to a pro-arranged con- . vention. This procedure was repeated for used-car salesmen. The new and lSeventy-six cases had one or more parts of the criterion data missing and were not included. Sixty-nine cases arrived from the field after the criterion analysis was completed. InspeCtion of the relationship between the criteria for these cases supported the findings of the criterion anal- ysis and the cases were included in the item analysis sample. 23 the used-car salesmen were pooled and tetrachoric correlations computed for the three measures.1 These results are found in Table I. TABLE I INTERCORRELATIONS AMONG CRITERIA BASED ON 5A5 CASES FROM ITEM ANALYSIS SAMPLE II III I. Units Index .9h .8l II. Earnings --- .8A III. Ratings --- --- Some concern was felt over the possibility that the inter-criterion relationships among new-car salesmen compared to used-car salesmen or among Car Make A salesmen compared to Car Make B salesmen might have been different from the relationships found among the 5A5 cases representing all types of salesmen considered as a total group. To investigate this possibility the intercorrelations were computed separately for Car Make A new-car salesmen (N 75), Car Make B new-car salesmen (N 335), Car Make A used-car salesmen (N 18), and Car Make B used-car salesmen (N 117). These results are presented in Table II. lTwo alternate correlations were considered that would have per- mitted maintenance of the criterion data in continuous form. (I) Ranks could have been converted to standard scores, salesmen from all dealer- ships pooled, and Pearsonian correlations computed. Widely divergent standard scores could only have been obtained by salesmen employed at dealerships with larger numbers of salesmen; this in turn would have meant that larger dealerships would contribute disprOportionately to the correlations. (2) Pearsonian correlations could have been computed for each dealership separately and then averaged. A substantial num- ber of dealerships had only five or six salesmen. Pearsonian correla- tions based on sub-samples of that size were judged to be no more reliable than tetrachorics based on the entire sample. TABLE II DITERCORREATIONS AMONG THE THREE CRITERION MEASURES COMPUTED SEPARATELY FOR SELECTED SUBVARIETIES II III I. Units Car Make A.- New Car .90 .93 Car Make B - New Car .98 .78 Car Make.A - Used Car .76 .76 Car Make B - Used Car .87 .58 II. Earnings Car Make A.- New Car --- .8h Car Make B - New Car --- .82 Car Make A.- Used Car --- .9h Car Make B - Used Car --- .83 III. Ratings Selection of the criterion measure. Comparison of the results ob- tained on the subvarieties of salesmen considered separately with those obtained on the SAS cases considered as a group were quite consistent when the sampling errors of the correlations were considered. Although all correlations in Table I and II are significantly different from zero, those in Table II based on the smaller numbers of cases are rel- atively unstable. The correlations based on the 17 Car Make A used-car cases were included only to complete the pattern and should best be considered only in the context of the other correlations. The standard error, for example, of the two correlations of .76 computed on this group was .38. From the above comparison, it was concluded that con- sideration of the salesmen as a total group was reasonable for the pur- poses of selection of a criterion. The consistently high correlations among the criteria for the total group (Table I) strongly suggested that little would be gained from com- bining two or three of the measures to obtain a composite criterion. The problem became one of selecting a single criterion measure from the three available measures. The average correlations between each of the measures with the other two measures (using Fisher's z transformation) were: earnings, .903; units, .892; and ratings, .823. This suggested that earnings and units be given preference over ratings. The correlations between earnings and tenure (.18) and between units and tenure (.19) were not significantly different from one another and hence did not provide a basis for making a choice. The decision was made on a practical basis. The form used with the item analysis sample to collect this criterion data was completed in part by the Bookkeeping Department at the dealership and in part by the Dealer or Sales Manager. If ratings had been chosen as the single cri- terion measure then this procedure would necessarily have to be repeated in the validation and cross validation stage. If earnings had been chosen then only the Bookkeeping Department would have to be involved in the data collection in the validation and cross validation stage. Con— sidering the difficulty encountered with the item analysis sample in getting the criterion forms returned, the elimination of one of the two participants was seen as a possible way of eliminating one of the sources of the poor returns. On this basis, earnings was chosen as the sole cri- terion. The criterion form used is reproduced in Appendix F. Earnings and tenure. The correlation of the criterion (earnings) with tenure was computed separately for the total of 5&5 cases and for each of the sub groups. These results are summarized in Table III. TABLE III CORREIAEIONS BETWEEN TENURE AND EARNINGS FOR ALL 5A5 CASES AND SELECTED SUBVARIETIES All Cases . .18 Car Make A - New Car .h3 Car Make B - New Car .08 Car Make A - Used Car .16 Car Make B - Used Car ~25 The correlation of .18 based on the 5&5 cases was significantly dif- ferent from zero and indicated further investigation of the effects of tenure was warranted. Curves were plotted separately for the two sub groups of salesmen for which the highest correlations had been obtained. 100 30 20 PERCENT IN HIGH EARNINGS CATEGORY v1 0\ O O 10 9-12 13-15 16426 27-33 3h-69 70-lh9 150-2EO TENURE IN MONTHS Figure 1. Relationship between earnings and tenure for car make A new-car salesmen 27 100 80 7O 60 50 30 PERCENT IN HIGH EARNINGS CATEGORY 10 Figure 9-13 11+-17 18-2h 25-31 32-59 (SO-71+ 75-2740 TENURE IN MONTHS 2. Relationship ,between earnings and tenure for car make B used-car salesmen. 29 These were Car Make A new-car salesmen and Car Make B used-car salesmen. These curves are seen in Figures 1 and 2. The vertical axis in these figures represents the proportion of salesmen for various tenure ranges who had been classified as high on earnings. The horizontal axis represents tenure expressed in months. The class intervals on the horizontal axis are different in Figures 1 and 2. These particular seven intervals were choosen so that approxi- mately the same number of cases would appear within each interval. The curve seen in Figure 1 seems to indicate a linear relationship and the curve in Figure 2 does not. No way to adjust systematically for tenure appeared to be readily available. A token step to reduce the ef- fects of tenure was taken by dropping the 6 Car Make A new-car salesmen with less than 12 months tenure and the 13 Car Make B used-car salesmen with less than 13 months tenure. The analysis of criteria resulted in the use of a single criterion in this study -- that of earnings. CHAPTER VII ANALYSIS OF SUBVARIETIES In the early planning of the study it was estimated that Sample III would consist of approximately 1000 cases and would be used for vali- dation and cross validation. The poor returns of the questionnaires from Samples I and II led to a revision of the number of cases expected from Sample III. The estimate of 1000 was revised downward to 600. The original plan of the study called for nine separate item va- lidity analyses. One item validity analysis was to be conducted for all of the car salesmen considered as a single group. Eight item va- lidity analyses were to be conducted for each of the eight subvarieties of car salesmen considered separately. It became Obvious that Sample III was not large enough to permit proper validation and cross validation of the revised form of the questionnaire -- Form C and that a change in detail of this study was desirable. The decision was made to reduce the number of item validity anal- yses that would be conducted for the subvarieties of salesmen from eight to six. An intermediate step was included in the design to serve as a basis for deciding which of the two subvarieties would be eliminated. This step was referred to as the Analysis of Subvarieties. Rationale If one pair of subvarieties could have been eliminated from the study then the size of the validation and cross validation samples could have been reduced about 25 percent below what otherwise would have been required. 30 31 The original choice of the four pairs of subvarieties was based upon the assumption that the differences between the subvarieties within each of the pairs were sufficiently great to warrant individual explor- ation in the item validity analyses. That pair of subvarieties could be eliminated which could be shown empirically to have the least difference between the two subvarieties comprising the pair. The most logical basis for making this comparison appeared to be the Personal History Form itself. For example, if it could be shown that there was little difference in the A way new-car salesmen anewered the questionnaire compared with used-car salesmen but that the subvarieties within the remaining three pairs an- swered the questionnaires quite differently from one another, then the new and used-car subvarieties could be eliminated most defensibly from subsequent analyses. One further problem had to be considered. The fact that new-car salesmen, for example, as a group did not give responses appreciably dif- ferent from used-car salesmen as a group did not preclude the possibility that items discriminating between high criterion and low criterion new- car salesmen could be’different from those discriminating between high criterion and low criterion used-car salesmen. This can be seen more clearly from following through a hypothetical caSe. let us assume an item with two alternatives. Comparison of the to- tal high criterion group with the total low criterion group (i.e. all salesmen pooled together) could have yielded the following distribution: Alt. 1 Alt. 2 Total All high criterion car salesmen 10 10 20 All low criterion car salesmen 10 10 2O 32 From this distribution we would have inferred that the item did not discriminate between the two criterion groups of car salesmen in general. we might then have asked whether a pair of subvarieties with- in that total group, for example new-car salesmen and used-car salesmen, were sufficiently different from one another to warrant separate treat- ment during the item validity analysis. Comparison of these two groups on the same item might have yielded the following distribution: Alt. 1 Alt. 2 Total All new-car salesmen 10 10 20 All used-car salesmen 10 10 20 If we had inferred from this distribution that separate treatment of the two subvarieties during the item validity analysis stage would have been unrewarding, we would have been in error. Separate treatment of the subvarieties in the item validity analysis might have yielded the following distributions: Alt. 1 Alt. 2 Total High criterion new-car salesmen 0 10 10 Low criterion new-car salesmen 10 0 10 and Alt. 1 Alt. 2 Total High criterion used-car salesmen lO O 10 Low criterion used-car salesmen 0 10 10 From these two distributions it is apparent that separate treat- ment would have been warranted. This fact could have been detected in advance of the item validity analysis if the criterion had been taken into consideration in the following way: Alt. 1 Alt. 2 Total High criterion new-car salesmen 0 10 10 High criterion used-car salesmen 10 0 10 and Alt. 1 Alt. 2 Total Low criterion new-car salesmen 10 O 10 Low criterion used-car salesmen O 10 10 Design for the Analysis of Subvarieties. The design selected for the Analysis of Subvarieties consisted of the comparison of the high criterion members of a subvariety with the high criterion members of its opposite subvariety. This was repeated for the corresponding low cri- terion groups. Since salesmen in the item analysis sample were admin- istered only one form of the trial questionnaire it was necessary to make separately all of the comparisons for Form A items and Form B items. The chi square technique applied to a two by j contingency table was chosen for the item validity analysis and is discussed in some de- tail in Chapter VIII. Adoption of the same technique for the Analysis of subvarieties simplified the computations considerably. One further comment about the design is necessary. The size of the item analysis sample did not permit, in either the Analysis of subva- rieties or in the item validity analysis, the use of mutually exclusive sub-samples. It was necessary to re-use the same cases in both the treatment of the total group as well as in the various subvariety item validity analyses. For example, any one salesman might have appeared in the analysis for the salesmen in general, then in the new car sub- sample, the Car Make A sub-sample, the younger salesmen sub-sample, and the larger volume dealership sub-sample. Interaction effects among the various subvarieties could not be considered. Results of Analysis of Subvarieties. The comparison of each pair of subvarieties resulted in four arrays of 1A8 chi squares each. 3A Probability values were assigned to each of the chi squares using stand- ard tables. The number of chi squares that achieved the 20 percent level of significance or better was determined. These results for the four pairs of subvarieties are summarized in Table IV. Inspection of the table indicates that the fewest number of dis- criminations was made in the comparisons based upon differences in the volume of the dealership where the salesmen were employed. The decision was made to eliminate that pair of subvarieties from further study. Biasing effects due to low theoretical frequencies. Perusal of the contingency tables prepared for the comparisons of the various groups revealed that a substantial number of the theoretical frequencies were below the minimum requirements. Since the biasing effects due to low theoretical frequencies always Operated to inflate the value of the chi square, it was necessary to obtain some assurance that the biasing ef- fects were reasonably consistent among the four pairs of subvarieties. Toward this end the following analysis was made. Cochran (6) has suggested the following criteria for low theoreti- cal frequencies. If 80 percent or more of the cells in a contingency table have theoretical frequencies of five or more then the minimum the- oretical frequencies for the remaining cells is one. ‘If less than 80 percent of the cells have theoretical frequencies of 5 or more, then the minimum acceptable for the remaining cells is two. All contingency tables used in the Analysis of Subvarieties that had chi squares significant at the 20 percent level were identified. Each of the tables was inspected; those failing to meet Cochran's criteria were identified and counted for each of the four pairs of subvarieties. TABLE IV NUMBER OF ITEMS SIGNIFICANT (P2 .20) FOR VARIOUS COMPARISONS IN THE ANALYSIS OF SUBVARIETIES Form Form Forms Groups Compared A, B A and B High Crit. YOunger Salesmen vs. High Crit. Older Salesmen 65 68 133 Low Crit. YOunger Salesmen vs. -Iow'Crit. Older Salesmen 73 57 130 Total 138 125 263 High Crit. Car Make A Salesmen vs. High Crit. Car make B Salesmen 37 35 72 Iow'Crit. Car Make A Salesmen vs. Low Crit. Car Make B Salesmen 30 A7 77 Total 67 82 1A9 High Crit. New-Car Salesmen vs. High Crit. Used-Car Salesmen 36 3O 66 Low Crit. New-Car Salesmen vs. IOW'Crlt. Used-Car Salesmen 35 36 71 Total 71 66 137 High Crit. Larger VOl. Dealer Salesmen vs. High Crit. Smaller VOl. Dealer Salesmen 3h 30 6h Low Crit. Larger Vol. Dealer Salesmen vs. Iow'Crit. Smaller Vbl. Dealer Salesmen 32 28 60 Total 66 58 12A These results are entered in the first column of Table V. Since the number of items observed at the 20 percent level varied among the four pairs of subvarieties, the amount of potential biasing is most meaning- fully expressed in the form of a percent of the total. This percent is seen in the third column of the table. TABLE V DIFFERENCES IN THE BIASING EFFECTS DUE TO LOW THEORETICAL FREQUENCIES IN TEE FOUR PAIR OF SUBVARIETIES IN THE ANALYSIS OF SUBVARIETIES No. of No. of Items Percent of Pairs of Sub- Items with Observed Observed That ' varieties Biased Cells (P2 .20) Were Biased New vs. Used 85 137 ' 62 Car Make A.vs. Car Make B 107 lh9 71 Larger Vblume Dealers vs. Smaller Volume Dealers 73 12A 59 Younger vs. Older lhh 263 55 Although the differences in the percent of potentially biased items did not appear to be very great, further information was desirable con- cerning,the number of items that achieved the 20 percent level solely as a result of the biasing. A check was made on this in the following way. Since the greatest potential biasing (71 percent) occurred in the Car Make A versus Car Make B analysis, these items were chosen for further investigation. Each of the potentially biased items was further ex- amined to identify those which could possibly have achieved the 20 37 percent level by biasing influences alone. The contingency tables for those items that were at all suspect were recomputed to remove the biasing effects. This was accomplished by grouping columns in the contingency tables to achieve the required theoretical frequencies. The chi squares resulting from the re-computation were then re- interpreted for significance using the new degrees of freedom. Eleven items from the total of 107 "biased" items failed to reach the 20 per- cent level. Repetition of this re-computation for the other pairs of subva- rieties did not appear necessary. InSpection of Table V shows that if 11 items were lost due to biasing effects in the new-used comparisons, the number remaining significant would still be greater than that for the subvarieties based upon size of volume of the dealership. It was concluded that the possible differences in the biasing effects due to low theoretical frequencies could not have accounted for the obtained differences in the Analysis of subvarieties. CHAPTER VIII ITEM.VALIDITY ANALYSIS Purpose The purpose of the item validity analysis was to identify the items on Form A and Form B of the Personal History Form that were signifi- cantly related to the criterion measures. This analysis was undertaken for the entire item analysis sample as a single group and was referred to as the "Over-all" item validity analysis. The analysis then was re- peated separately for selected cases from the item analysis sample rep- resenting each of the six subvarieties of salesmen. These were referred to as the "New—car" item validity analysis, "Used-car" item validity analysis, etc. Re-use of the sample As mentioned in the discussion of the Analysis of Subvarieties, the item analysis sample was too small to permit the use of mutually exclu- sive sub-samples for each of the separate item validity analyses. After the over-all analysis was completed, the cases were subdivided into new- car salesmen and used-car salesmen and separate item validity analyses were conducted for each. The cases were then re-assembled and subdivided again into Car Make A and Car Make B salesmen and separate item validity analyses made for each. This procedure was repeated for "younger" and "older" salesmen. Item statistic used. The statistical technique used in the item validity analyses was the chi square test applied to 2 by j contingency 38 39 tables. A cell square contingency coefficient was computed for the com- parison of the high and low groups on each alternative of the item. The sum of these cell square contingency coefficients provided a chi square by which the entire item could be evaluated for significance. Examples of these contingency tables are found in Appendix G. There was no a priori scoring key for the items in the Personal History Form. Scoring keys were developed on the basis of the findings in the item va- lidity analyses. The use of the chi square technique permitted the as- signment of significance levels to individual alternatives when score values were assigned to them as well as to the item as a whole. Results. The seven analyses yielded seven arrays of lh8 chi squares each for Form A items and the same number for Form B items. Probability values were assigned to each of the chi squares. The number of items that were found to be significantly related (P;2.20) to the criterion were counted for each of the analyses. These results are summarized in Table VI. Beta Errors. ~Inspection of the results in Table VI suggested the possibility of a relationship between the size of the sample used in each of the analyses and the number of significant items observed. To ex- plore this further a rank order correlation was computed between the two variables. It was found to be .65 and prObably was significantly dif- ferent from zero.1 This result was interpreted as reflecting differences in the inci- dence of beta errors as a function of the size of the samples used. lWith an N of 1h, a Pearson correlation coefficient of .66 would have been significant at the one percent level and a coefficient of .53 significant at the five percent level. TABLE VI NUMBER OF SIGNIFICANT (P 2.20) ITEMS OBSERVED IN THE ITEM VALIDITY ANALYSES Form A and B .FormpA .FogmgB (Avgll Group No. No. of No. of Items.Obs'd. Items Obs'd. Items Obs'd. N (P; .20) N (P: .20) N (P2. .20) Over-all 358 37 335 Al 3A6 39.0 Car Make A 80 27 97 3o 88 28.5 Car Make B 278 A9 238 38 258 h3.5 New-Car 257 37 253 38 255 37.5 Used-Car 101 17 82 31 92 2u.0 Younger 169 29 1A6 A2 158 35.5 Older 189 22 189 35 189 28.5 A1 Analyses based upon the larger samples had fewer beta errors than those based upon smaller samples. This finding suggested that in order to make an unbiased test of the various keys in the validation and cross validation it was desirable to limit each of the keys to the same num- ber of items. The number arbitrarily chosen for this purpose was ho. Inspection of Table VI shows the used-car item validity analysis to have yielded the least number of items significant at the .20 level or better, i.e. AB items. .It seemed advisable not to include items in the keys that were much below this level of significance, consequently ho items was chosen as the arbitrary limit for each of the keys.1 Selection of items for the over-all key. In selecting the items for the over-all key an attempt was made to select the ho items which would most likely effectively discriminate when applied to the validation and cross validation samples. The following procedure was used. The 50 items which had achieved the highest levels of significance in the over-all item validity analysis were identified. The contingency tables for each of these items were then evaluated for low theoretical frequencies; Cochran's criteria were applied. If a contingency table failed to meet the criteria, a new value of chi square was computed to eliminate the biasing effects and a new P value was assigned. (Illustra- tions of this procedure are found in Appendix G.) The 50 items were then evaluated in terms of the number of dis- criminations each had made. It was possible for an item to have achieved the .10 level of significance and to have had all the significance at- tributable to one alternative which had been chosen by as few as three lEmpirical evidence in support of this decision will be found in Appendix H. #2 percent of the sample. To guard against the inclusion of items of this type, the following criterion was adopted. An item had to have at least one alternative which had discriminated at the .32 level and which had been chosen by at least ten percent of the sample.1 The 50 items were screened against this criterion and those failing to meet it were re- jected. UBing the unbiased P values, the #0 most significant items remain- ing were then chosen for the over-all key. Selecting items for the subvariety keys. The same procedure used in the selection of items for the over-all key was used in selecting items for the subvariety keys with the following major exception. The data from the over-all item.validity analysis and the Analysis of sub- varieties considered jointly, permitted an additional refinement in se- lecting items for the subvariety keys. In any one of the item validity analyses conducted for a parti- cular subvariety, e.g. newbcar Salesmen, a total of 296 chi squares was computed. Of these 296 chi squares it was expected that a certain num- ber would have achieved significance as a result of chance factors alone. Chi squares achieving significance as a result of chance factors may be referred to as alpha errors. Knowledge of the results obtained in the over-all item validity analysis and in the Analysis of Subvarieties was TA conservative level of significance was desirable for this pur- pose. The .32 level was chosen since it conveniently corresponded to a chi square of 1.00 when the degree of freedom equals one. This crite- rion was not rigidly held. If an item had, for example, an alternative which was highly significant but had been chosen.by 8 or 9 percent of the cases and had another alternative which had just fallen short of the .32 level and had been chosen by a large percent of the cases, the item was not rejected. 1&3 used to identify these errors. Although the identification was not put forward as infallible, it was judged to be markedly superior to the uncritical acceptance of these errors. The logic of the procedure used is best illustrated by the dis- cussion of a single, hypothetical item. Let us assume that an item (A) was found to be significantly related to the Criterion (Pa: .20) in the item validity analysis for new-car salesmen, i.e. high criterion. new-car salesmen compared to low criterion new-car salesmen. The chi square for item.$ was one of 296 chi squares computed for these groups. Failing knowledge of the intercorrelation among the 296 items it is not possible to estimate the number of these 296 chi squares that would have achieved significance from chance factors alone (9). It is known that if the items could have been considered as completely independent events it would have been expected that 20 percent, or 59 items, would have appeared significant at the 20 percent level by chance. In our case we could say only that there was some unspecified probability that the significance of item i was a chance occurence. The data from the pre- vious analyses provided a basis for identifying those items that most likely achieved their significance due to chance. let us assume that in the Analysis of subvarieties item i_was found to be not significantly related (Pf.20) to the class memberships for new-car salesmen and used-car salesmen. That is, the item failed to discriminate between high criterion newhcar salesmen and high criterion used-car salesmen, as well as between low criterion new-car salesmen and low criterion used-car salesmen. In addition to this, let us also as- sume that item i failed to discriminate (P:- .20) between the high and Ah low criterion groups in the over-all item validity analysis. These two facts are strong evidence that the significance obtained for item 1 in the new-car item validity analysis was an alpha error. This logic was followed in selecting the items for the six sub- variety keys. A.pool of items was set up consisting of the 60 or so items which had been found in the appropriate item validity analysis to have the most significant relationships with the criterion. Items were then eliminated from this pool if they had not reached the .20 level or better in the over-all item validity analysis or in either of the two appropriate Analyses of Subvarieties. The selection of the ho items, from those remaining in the pool, was then done in exactly the same way as were the items for the over-all key. An illustration of the entire procedure for selecting the items for the New-car Key is found in Appendix I. Scoring the items. Having identified the A0 items for each of the seven scoring keys, the next step was to assign score values to each of the alternatives to the items. Following the recommendations of Gul- likson (7) and others, a simple scoring system was used. Alternatives favored by the high criterion group were assigned a score of two, those favored by the low criterion group were assigned a score of zero, and those that were neutral were assigned a score of one. The general principle in assigning scores to alternatives was to infer from the contingency table in which the two contrasting criterion groups had been compared for a given item, how future samples of car salesmen from the same population would most likely distribute themselves when that item was applied to them. A5 If the difference between the two proportions Of the criterion groups that had chosen a particular alternative was significant at the .32 level, then a score of two or zero was assigned to that alternative depending upon whether.it had been favored by the high or low criterion group. If an item did not discriminate between the two groups at the .32 level of significance it was assigned a score of one, unless its re- lationship to the distributions of the other alternatives to the item warranted scoring it as a two or a zero. This latter decision was made on a judgmental basis. In an effort to keep these decisions consistent for each of the seven scoring keys, a series of conventions were em- ployed. These conventions, with illustrations of their applications, are found in Appendix J. CHAPTER IX VALIDATION AND CROSS VALIDATION RESULTS Background Form C of the Personal History Form consisted of the items from Forms A and B which had been found to be most promising in the seven 'item validity analyses. Form C was administered to car salesmen in Sample III. Completed questionnaires were returned by 85h salesmen for whom criterion information was available. A tetrachoric correlation between tenure and the criterion was found to be .20 which was significantly different from zero. The re- lationship was plotted and is seen in Figure 3. The strongest relation- ship appeared at the lower end of the tenure scale. The decision was made to eliminate all cases with seven or less months tenure. This in- volved 105 cases. The 7A9 remaining cases were sub-divided into 1h sub-samples. These 1h sub-samples consisted of two sets of seven samples each. One set was designated 1A, 2A, 3A, etc. These seven samples were used to estimate the validity of the seven scoring keys. The other set was designated 1B, 2B, 3B, etc. These seven samples were used to estimate the cross validity of the seven scoring keys. Each of the seven pairs of sub-samples were comprised of appropri- ate subvarieties of car salesmen for evaluating the seven keys. For example, Sub-samples 1A and 1B Were comprised of "car salesmen in gen- eral" and were used for the validation and cross validation of the A6 100 90 E E3 80 :5 70 CD pg 60 H z E; 50 Fri! gg A0 H 811 z 30 H g; 20 8 g; 10 9-: 0 h-9 lO-lu 15-21 22-29 30-h2 A3-85 86 and above TENURE IN MONTHS Figure 3. Relationship between earnings and tenure for Sample III. A7 Over-all Key. Sub-samples 2A and 2B were comprised of Car Make A salesmen and were used for the validation and cross validation of the Car Make A Key. The procedure used in drawing the fourteen sub- samples from Sample III is found in Appendix K. The Personal History Forms for the car salesmen in all of the 1h sub-samples were scored using the Over-all Key. In addition, the Per- sonal History Forms for the salesmen in all samples, except 1A and 1B, were scored with their appropriate Subvariety Key. Point biserial cor- relations were computed between the questionnaire scores and the cri- terion. Analyzing the Data Four analyses were made of the relationships between the predictors and the criterion. The first analysis consisted in considering the cor- relations based on the "A" samples as estimates of the validity of the scoring keys. The correlations based on the "B" samples were considered as estimates of the cross validity of the scoring keys. These results are summarized in Table VIII. The second analysis consisted of obtaining optimal cutting scores from the distributions based upon the "A" samples. Optimal cutting score is used here to mean the score at which the greatest percentage of cor- rect predictions could be made. The cutting scores from the "A" samples were applied to the "A" samples and then applied to the corresponding "B" samples. Indices of Forecasting Efficiency were then computed for the "A" and the "B" samples. These results are summarized in Table IX. The raw score distributions are found in Appendix M. The third analysis consisted of applying the optimal cutting scores 1&9 TABLE VIII VALIDITY AND CROSS VALIDITY COEFFICIENTS FOR VARIOUS SCORING KEYS L Sub- Type of N Key £_Pbis. (Approp- £_Pbis. (Over- sample Salesmen Used riate subvariety all Key Used Keijsed.) Throughout) sub-samples A B A B 1. Car Salesmen A7 Over-all --- --- .A80** .138 in general 2. Car Make A A8 Car Make A .113 .h16** .29A* .328* 3. Car Make B A8 Car Make B .hll** .33A* .220 .h06** A. New-Car A8 New-Car .327* .203 .399** .066 5. Used-Car A7 Used-Car .021 .121 -.101 .16A 6. Young A7 Young .321* .175 .281 .O5A . 7. 01d A8 Old .191 .031 .3A1* .120 ** Significant at the .01 level * Significant at the .05 level 50 ~33 no. one so passages as $04.. no a. Hate .8. one as sausages 3 Sod .8 p om .m 8 . om .sm 5 S .1. S .11- 5 .mm mm So .s emu. 8.3 Swarm ma slum mood. .812 mm mass» 8 oo . mm 8 mm .o 3 mo; TM 8 .m S .3088: .m S .m mm .m on .sm 3 we 4 S .3 5.8 3 80-3% .11 omé 1.1mm SA: on em; mm.mm 8.3 13 m can: .80 .m smé 8.3 cmém we 34 mm.mm 8.8 m: < use: .80 .m Ea 83H 13.? ms .i 2. .3 .. assocom 5 scsmoasm .H moHSsm scamsmm soul assessor season has 32“ am: so hex we... no .32 ado-hose am: no hex :a... no 50% $3346 as She dingo Stucco mouoom A3 .33 Rogelio aeogfism monocm sosuoHsM magnum .8 swam dam mo .H .mé. mo .H messed so .smam .mé. mo .H sad so .H masses no case .85 mgm :5. ZOE mmmoom can-woo go OZHmD mgm :m: Q24 mgm :5. mom MDZMHUHE cggh rub man—52H gage 51 from the "A" samples to the "A" and "B" samples. The observed fre- quencies from the "A" samples were then compared to the observed fre- quencies from the "B" samples using the observed frequencies from the "A" samples as the frequencies expected in the "B" samples. These results are summarized in Table X. TABLE X CHI SQUARES BETWEEN OBSERVED FREQUENCIES FROM "A" SAMPLES AND OBSERVED FREQUENCIES FROM "B" SAMPLES USING OPTIMAL CUTTING SCORES FROM "A" SAMPLES AND THE OBSERVED FREQUENCIES FROM "A" SAMPLES AS FREQUENCIES EXPECTED IN THE "B" SAMPLES Us subvar e s s n Ker-a e Sub-samples Type of Cutting Chi- Cutting Chi- . Salesmen Score Square Score Square 1A vs. 1B Car Salesmen -- -- A5 7.87** in general 2A vs. 2B Car Make A A5 6.20* A2 1.68 3A vs. 3B Car Make B Ah 1.01 A5 1.60 AA vs. AB New-Car Al 5.39* to 8.77** 5A vs. 5B Used-Car Al 1.h2 A7 .98 6A vs. 6B Young 39 10.13** A8 1.89 7A vs. 7B Old 36 7.15** A7 9.12** ** Significant at the .01 level * Significant at the .05 level The fourth analysis consisted of combining the seven pairs of sub- samples into seven larger samples. That is, 1A and 1B were combined, 2A and 2B were combined, etc. Point biserial correlations were computed for these seven samples. These correlations were taken as estimates of 52 validity. These results are summarized in Table XI. TABLE XI VALIDITY COEFFICIENTS USING "A" AND "B" SAMPLES COMBINED Sub- Type of N Key Used 3 Pbis. (ApprOp- £_Pbis. (Over- samples Salesmen riate Subvariety all Key Used Kby Used) Throughout) 1A and 1B Car Salesmen 9A Over-all --- .313** in general 2A and 2B Car Make A 96 Car Make A .253* .311** 3A and 3B Car Make B 96 Car Make B .370** .305** AA and AB New-Car 96 New-Car .255* .209* 5A and 5B Used-Car 9A Used-Car .069 .033 6A and 6B Young 9A Young .199 .182 7A and 7B Old 96 Old .117 .23A* ** Significant at the .01 level * Significant at the .05 level Interpretation of results. The results from the first analysis (Table VIII) indicated that the Over-all Key applied to the sample of "car salesmen in general" (1A) had a validity significant at the .01 level. significantly. However, upon cross validation on Sample IE it did not predict When the Over-all Kay was applied to Car Make A sales- men (2A) its validity was significant at the .05 level and when applied to Sample 2B its cross validity was also significant at the .05 level. The only subvariety Kay which predicted significantly on both the validation and cross validation samples was the Car Make B Key applied 53 to Samples 3A and 3B. The difference in the correlation of .328 for the Over-all Key on Sample 2B and the correlation of .33A for the Car Make B Key on Sample 3B was not significant. In summary of this analysis, there appeared to be no evidence to suggest that the Subvariety Keys were more effective predictors than the Over—all Kay. The results of the second analysis indicated that the Over-all Key was most predictive when applied to Car Make A.salesmen in Sample 2B and Car Make B salesmen in.Sample 3B. The two most predictive sub- variety Keys were the Car Make A and Car Make B Keys on the same sam- ples, i.e. 2B and 3B. A.method for estimating the standard error for an Index of Fore- casting Efficiency was not known to the writer. Inspection of the raw data found in Appendix M shows that the observed differences in the raw percentages were probably not significant. For example, the Index of Forecasting Efficiency for the Over-all Key applied to Sample 2B was 25.00 percent and that for the Car Make A Key applied to the same sam- ple was 33.33 percent. The raw difference between the two indices was 8.33 percent. HOWever, this difference was the result of two additional cases, from a total of A8 cases, being classified correctly with the Car Make A key compared to the Over-all Kay. The difference of A.l6 percent between the indices for the Car Make B Key and the Over-all Key on Sample 3B was the result of the Car Make B Key making one additional correct classification. The question could be raised as to what effect the differences in the selection ratios had on these Indices of Forecasting Efficiency. The selection ratios for the following four samples were: Over-all Key on Sample 2B, Al.7 percent; Over-all Key on 3B, 35.A percent; Car Make A Key on 2B, 37.5 percent; and Car Make B Key on 3B, 62.5 percent. The only selection ratio out of line with the others was that for the Car Make B Key on Sample 3B (62.5 percent). A cutting score of A7 on this sample would give a selection ratio of Al.7 percent. Using that cutting score then, the Index of Forecasting Efficiency for the Car Make B Key on Sample 3B became 25.00 percent. It was concluded from this analysis that differences in Forecast- ing Efficiency for over-all and specific keys as applied to "B" samples for Car Makes A and B were probably not significant. In the third analysis of predictive value of the Over-all and Sub- variety Keys as shown in Table X, there was some inconsistency with previous analyses in that the Car Make A subvariety key as applied to Sample 2A versus 2B gave rise to cell frequencies in 2B that were sig- nificantly different from those in 2A. Otherwise, the results of this analysis tend to verify those of the other analyses reported in this study. The Used-Car Subvariety Key and the Over-all Key applied to used-car salesmen both seem to stand up about equally well upon applica- tion to Samples 5A versus 5B. However, as seen in Tables VIII and XI, the validities and predic- tive values of both the Used-Car Key and the Over-all Key applied to used—car salesmen are nil, and there is nothing in Table X to the con- trary. As apparent from Table X, the Over-all Key as applied to Car Make A and Car Make B salesmen works about as well or better than the corresponding subvariety keys. The results from the fourth analysis showed the Over-all Key when applied to "car salesmen in general" (1A and 1B combined) to have a validity of .313 which was significant at the .01 level. This key, when applied to Car Make A salesmen (2A and 2B combined) and when applied to Car Make B salesmen (3A and 3B combined), had validity coefficients of .311 and .305 respectively. Each was significant at the .01 level. The most promising of the Subvariety Keys appeared to be the Car Make A Key (on Samples 2A and 2B combined) and the Car Make B Key (on Samples 3A and 3B combined). Their validity coefficients were .253 (significant at the .05 level) and .370 (significant at the .01 level). An average of the validity of the Car Make A Key (.253) and the Car Make B Key (.370) was .310. This value was not significantly different from the validity of the Over-all Key applied to Samples 1A and 1B com- bined. Similarly, the validity of the Car Make B Key applied to Samples 3A and 3B combined (.370) was not significantly different from the Over- all Key applied to Samples 1A and 1B combined. The results of the fourth analysis provided no evidence to suggest that the Subvariety Keys resulted in better prediction than the Over-all Key.l 1The writer and his advisor did not agree on the use of the terms "validation" and "cross validation" or on the way the data were analyzed in this chapter. The suggestions of the advisor were followed. CHAPTER X SUMMARY AND CONCLUSIONS A.Personal History Form to aid in the selection of retail car sales- men was developed according to two methods: the Over-all Method and the Subvariety Method. The Over-all Method assumed the sample of retail car salesmen under study was sufficiently homogeneous to warrant treat- ment as a single group in the course of developing the selection in- strument. The Subvariety Method assumed that possible differences among the various subvarieties of retail car salesmen were sufficiently great to warrant unique treatment of each of the subvarieties in the course of deve10ping the selection instrument. The Over-all Method produced an Over-all Key and the Subvariety Method produced six Subvariety Keys. Of these keys, it is reasonably safe to conclude (on the basis of the available evidence) that (a) valid Car Make Ag Car Make B (and possibly New-Car) sub- variety keys were developed. (b) an Over-all Key was developed with validity other than zero when applied to Car Make A, Car Make B (and pos- sibile general as well as New-Car) salesmen. The results of the four analyses of the relationships between predictors and criterion, on the whole, were consistent with one an- other. There was no conclusive evidence to suggest that the more laborious Subvariety Method produced a selection instrument which had greater predictive efficiency than that produced by the simpler Over- all MEthod. CHAPTER XI' SOME LIMITATIONS OF THIS STUDY The degree to which the results of this study can be generalized to situations other than the one specifically studied here is limited by a number of factors. Some of these follow. 1. The study was conducted using presently employed car sales- men. The validity and cross validity referred to throughout this study is then most properly called "concurrent" validity or "concurrent" cross validity. It was not determined in this study whether or not the instrument would predict for applicant populations. 2. The degree to which it would or would not be profitable to in- vestigate subvarieties of salesmen for selling jobs other than that of retail car selling was not determined. 3. The degree to which confidence can be placed in the conclusion that the Subvariety Method was not superior to the Over-all Method for the population of salesmen studied is limited to the particular sub- varieties of car salesmen that were investigated. It is possible that the investigation of other subvarieties would have given different re- sults. A. The most predictive types of items found in this study were biographical or personal history type items as well as personality type items. If different trial predictors had been used the results of the study might have been different. 57 APPENDIX AETENDIX A EXAMPLES OF PERSONAL DATA OR BIOGRAPHICAL TYPE ITEMS Physical Characteristics How tall are you? (1) 5'5" or less; (2) 5'6"; (3) 5'7"; (A) 5'8" to 5'11"; (5) 6'0"; (6) 6'1"; (7) 6'2"; (8) 6'3" or more. Education How many ears education have you had? (1) Less than 6 yrs. (2) 7 to 9 years; (3) 10 to 12 years; (A) 13 to 15 years; (5) 16 or more years. Previous work experience You left the dealership you worked at before this one because: (1) they had a "cool" car; (2) you didn't get along very well with the sales man- ager; (3) they wouldn't accept your deals; (A) you didn't like the other salesmen; (5) you weren't employed as a car salesman before this job; (6) some other reason. Early environmental factors Yen were raised in a family with; (1) no other children; (2) 1 other child; (3) 2 other children; (A) 3 other children; (5) A other children. Activities 23 achievements You have solicited contributions for a Charity; (I) frequently; (2) a num- ber of times; (3) once or twice; (A) at no time, but you think that you could do so; (5) at no time. Famihz What is the most frequent source of disagreements you.have had with your wife?. (1) Financial; (2) Social; (3) Personal; (A) Children; (5) Working hours; (6) Other; (7) I'm not married. Health Between the ages of 12 and 21, how often were you sufficiently ill to re- quire hospitalization? (1) never; (2) once; (3) twice; (A) three times; (5) four or more times. 58 59 Finances What was your wife's gross income last year? (1) She earned nothing; (2) Less than $500; (3) between $500 and $1500; (A) between §1500 and $2500; (5) between $2500 and $AOOO; (6) between $A000 and $6000; (7 more than $6000. _ Housing At the time you took this )Ob you were living in: (1) a rented room; (2) a rented apartment; (3 a rented house; (A) your own home (mort- gaged); (5) your own home (not mortgaged); (6) a house shared with parents or relatives. APPENDIX B EXAMPLES OF PERSONALITY TYPE ITEMS timism Do you tend to be essimistic or doubtful when stepping up to a big pro- blem? (1) Yes; (2? No; (3) Undecided. Personal Relations You like to have friends because: (1) you find them useful in getting things you want; (2) you can be yourself with them; (3) they know and respect you; (A you can count on them in a time of need. Igpulsiveness Do you occasionally buy things you don't need: (1) Yes; (2) N0; (3) Undecideda Introversion Would you rather be a bookkeeper than a bus driver at the same salary? (1) Yes; (2) No; (3) Undecided. Dominance You usually go along with the group on how things Should be done? (1) Yes; (2) No; (3) Undecided. Self confidence 9§_self sufficiency Do you often wonder how you "stack-up" against the peOple you work with? (1) Yes; (2) No; (3) Undecided. Argumentativeness Do you enjoy a good hot argument? (1) Yes; (2) No; (3) Undecided. Sociability Do you dislike working by yourself in some isolated spot for long periods of time? (1) Yes; (2) No; (3) undecided. . Emotional Stability Do you frequently feel "down in the dumps" for no particular reason? (1) Yes; (2) No; (3) Undecided. 60 Multiple You like least to: (1) take orders from someone who knows less than you; (2) wait in line; (3) stand in a crowded bus or streetcar; (A) wait for someone who is late for an appointment; (5) receive too little recogni- tion for something you did. APENDIX C EXAILPIES OF INTEREST TYPE ITEMS Interest i_n_ various kinds 9: Jobs Which of the following occupations appeals to you most? (1) Technical writer; (2) Athletic coach; (3) Inventor; (A) Employment Interviewer. Interest 22. various types 9_f_‘ activities Which of the following would you enjo doing most? (1) Giving a party; (2) Bargaining with a sales clerk; (3 Balancing a check book; (A) Sol- ving a mechanical puzzle. 62 APPENDIX D EXAb‘IPLES OF ATTITUDE — TYPE ITEI18 Work 1.2 general In your opinion, the most important features of a job are the: (1) pro- spects for the future; (2) "here and now" advantages. Selling 3.2 ggneral Which of the following would you prefer to sell? (1) A " slightly over- priced" product at a high commission per unit; (2) A "slightly under- priced" product at a smaller commission per unit; (3) I have no preference. Salesmen i3 Eneral You think of yourself as being: (I) quite typical of most salesmen you have known; (2) a little different from most salesmen you have known; (3) quite different from most salesmen you have known. Car selling and car salesmen Men go into car selling for all of the following reasons. Which do you think is most often the reason? (1) It provides a chance to "get around." (2) It provides a chance to make good money. (3) It looks like an easy Job. (A) Car salesmen operate on their own most of the time. Knowledge! 93 ggod sales practices Which of the following is the most profitable way for a new car sales- man to spend his working hours when he is not working the showroom? (l) Makin blind telephone calls; (2) cold-canvassing from house to house; (3 just "getting aroun " and meeting potential buyers. 63 APPENDDI E INSTRUCTIONS TO IOOKKEEPING DEPARTMENT For Completing Part I Column: A through D You will save time by reading all the instructions through before making any entries. I. Consider only salesmen employed (continuously be- tween 9/1,’54 and 3,-’31 55. 2. In Column A list salesmen under New (Tar Salesmen if 60 % or more of the units they sold between 9/1/54 and 3131/55 were new cars. List all others in C01- umn A under Used Car Salesmen. 3. In Column I enter the number of new cars sold by each new car salesman and each used car salesman. 4. In Column C enter the number of used cars sold by each new car salesman and each used car salesman. 5. In Column D enter the gross earnings for each new and each used car salesman between 9/1/64 and 3/81/65. 5- Return this questionnaire to the Dealer so that he can complete Column E. Pogo! PART I BOOKKEEPING DEPT. L A B C D ‘U 0:- 0: $37; “a h! 13" f“ 3” as 3 E x: 52 2E Sta NAME OF NEW CAR SALESMEN l l i l a l 4 ‘ i All __.__1 .111»- l , ---__1- 7,1, NAMEOFUSED CAR SALESMEN ill _ l - .71 l _ , 1 1-- l r in 1 -__ _._______ _,l T __, ,l___._-_, _l- ll—ll __ ._.r _r,l-_ r 1-, 1-11-1.-. .11-, ,_ ____ l— 1.1 __ r - - r ___ (l l l _- l ._.-_r_ l _4,_.ll.,l __ n- - -_ 1 ,4 ‘ l ‘ . l . f f i) ‘\ l\~.R —. E .—-—1 . ‘J 'r ._(jlf Jt’ APPENDIX E , CONTINUED INSTRUCTIONS TO DEALER OR SALES MANAGER For Camplcting Part II Column E The following llil\l' in (‘IL «Lima-soul .'l.\' \\':ij. ol' room-wring hon prolirionlly :1 rurrvnily mny‘oyul s;ilvr4 I‘ldlé DU "1 l-‘i-‘N NI—‘NN H W to |._‘ laidtoiaroiuruiarotpidiaretard 81 APPENDIX M (CONT'D.) Scores Sample Ea Sample 7a Sample 1b Sample 2b Sampler High Low High Low High Low High Low High Low 62 U1 65' '63 U] I—‘I—‘CUH sat—4 HIDH l—‘I’ONI—‘Nl—‘H l-‘t-J N i-‘l-J [Di-4H l-‘ l—‘l—' NNNH l-' ......” H [UMP F‘H NWHNN NNNNI-‘I-‘H DJ .7:- [._J H Cal-4w tat-tn.) NHHHmHmmmHHHHmH HNUUNH Ht» 4-? I-‘l-‘NN U) H reviews-atom L0 UJUU U0 m \JlO‘s 00 l-‘H Hl—‘H ton) tat-won) l—‘H trt-a wH-p'l-J-F’H l—‘H l—‘H H Hm I—‘erml-‘P l-‘N APPENDIX M (CONT'D.) Scores Sample Mb Sample jb Sample 6b Sample 7b High Low High Low High Low High Low 62 61 l 60 5; U1 |._.l moo l—‘I—‘ HHH l—l t—‘I-‘ww N N-P-‘H mHHHmtomm NHNHQ H w-F‘mH-F'I-Ji-‘HHH to mm-F‘Hwi-‘H I'D NNI‘Ol-‘l-‘l—‘HNNNN H NO\|—‘I-‘I-" DO U) H H l—‘l—‘m MPH merom Pl—‘m w town) wi-‘l-‘l-‘UINH l\) \O N ...a APPENDIX M (CONT'D.) Scores Sample 6a Sample 7a Sample 1b Sample 2b Sample 3b High Low High Low High Low High Low High Low 62 61 60 \H N I’DH \fl c) F‘F’ H nausea t» $r tr \0 #r —q rauakananarr east IAto I4 idle rotaroro i4 (None) FJRJFJUJRD4TRJRJNJNJ OJF‘FJUJ F‘F‘F‘F‘F‘ F‘ F‘F‘F‘ nasacucucsruidiAta e4e=n>n1 F4 nakasrsrkJ UJF‘ torutoro U) L» c: O) F’ Id F‘ F‘F‘F‘FJNJADADUJF‘ FJRJFJ t4nan>ov~sa rd F‘F‘#r FJRJUJ [—1 F‘F’ m (D H APPENDIX D4 RAW SCORES ON FORM C USING THE OVER-ALL KEY FOR FOURTEEN SUB-SAMPLES FROM SAMPLE III Scores Sample la Sample 2a Sample_3a Sample—ha Sample 5a High Low High Low High Low High Low High Low 62 l 61 m U"! .p— HHHH H HH H :mm m mOH NPrH Hm w HH ._l m mmem NH :HHH HH H Hm ID I'D mHHrH mem HHmerH meHH m Herm wH Hm HNHwHHmeH m HHHHmHHH m rHH mm H wHHH wHNHNH w HmHMHHm HHMH U) LA.) HwH U) m HHHH *4 LA) 0 u) [._s 22 . 85 APPENDIX M (CONT'D.) Scores Sample 1th Safllejb Sample 6b Sample 7b High Low High Low High Low High Low 62 62 6O 58 l 1 U1 U1 HHH :- 4r H -l roH HHHwHHHMH H ero mH HH HHHmw l-‘l—‘w HIoHtorurom H H HH HH Hm wmw Hmm -F"l—‘ NHtoHHtomH Nl-‘HH-F" HHHm w HanmHmH [0 UL) \J’l to HH HHHHmn) HHroH roHH meHHHHm H HHou ...—l HHm BIBLIOGRAPHY 1. Anderson, V.V. ngchiatry i3 Industgy. New York: Harper and Brothers, 1929, 222-2h3. 2. Bills, Marion A. Selection of casualty and life insurance agents. g. appl. ngchol., l9hl, 25, 6-10. 3. .A tool for selection that has stood the test of time. In L.L. Thurstone (Ed.), Applications g§_Psychology. (lst Ed.). New York: Harper and Brothers, 1952, 131-137. h. Bolanovich, D.L. and Kirkpatrick, F.H. Measurement and the selection of salesmen. Educ. payphol. Measmt., 19h3, 3, 333-339. 5. Chapple, E.D. and Donald, 0., Jr. An evaluation of department store salespeople by the Interaction Chronograph. g, Marktg., l9h7, 12, 173 -185 o 6. Cochran, W.G. Some methods for strengthening the common chi square tests. Biometrics, 195A, h17-h51. 7. Gulliksen, Harold. Theory of Mental Tests. New York: Jehn Wiley and Sons, 1950, 256-257. 8. Hquand, R.W. Techniques of salesmen selection. Educ. psychol. Measmt., l9h9, 9, l29-1h8. 9. Jones, Lyle V. and.Fiske, Donald W. Models for testing the signi- ficance of combined results. Psychol. Bull., 1953, 50, 375-382. 10. Kbrnhauser, Arthur'W. and Shultz, R.S. Research on selection of Salesmen. :1. Egg 0 P§2Ch010, l9hl’ 25’ 1'50 11. Stokes, T.M. Selection in a sales organization. g, app . ngchol., 19h1, 25, h1-h7. 12. TObolski, Francis P. and Kerr, W.A. Predictive value of the Empathy Test in automobile salesmanship. g} app . ngchol., 1952, 36, 310-3110 13. Wallace, S. Rains, Jr. Validities of a selection test for different criteria and sample components. Amer. Ppyphologist, l9h8flk3,~v~ 330 - Abstract \f‘ \- ~- lh. Wallace, S. Rains, Jr. and Twitchell, C.M; Management procedureséand test validities. Personnel Psy9h01., 19h9, 3, 277-292; 8 ‘ \_ s s . s Date Due Demco-293