EFFECTS OF EDUCATiONAL COGNITIVE STYLE 0N BEGINNENG SHORTHAND PERFORMANCE Thesis for the Degree of Ph. D. MECHEGAN STATE SNWERSITY CAROL FRANCES STENCEL 1 9 7 4 This is to certify that the thesis entitled EFFECTS OF EDUCATIONAL COGNITIVE STYLE ON BEGINNING SHORTHAND PERFORMANCE presented by Carol F. Stencel has been accepted towards fulfillment of the requirements for Ph.D. - Business and de ee 1n gr Distributive Education Major professor mum av" . .' "MAB & S'flNS' 800K BlNDERY INC. _; LIBRARY amoens ABSTRACT EFFECT OF EDUCATIONAL COGNITIVE STYLE 0N BEGINNING SHORTHAND PERFORMANCE BY CAROL F. STENCEL THE PROBLEM This was a study to determine the effect of educational cognitive style on beginning shorthand performance. The study identified those elements of the educational cognitive style held in common by the highly successful beginning shorthand students. It identified those elements held in common by the unsuccessful beginning shorthand students. The study also identified the differences between the collective cognitive styles of the highly successful and the unsuccessful student groups. The secondary purposes of this study were: (l) to provide information about the types of students who might have a high probability of achieving successfully in shorthand, (2) to-provide students with information about themselves, (3) to provide information regarding the methods and instructional strategies that may need to be employed to accommodate the dif- ferent "styles" of students. CAROL F. STENCEL PROCEDURES USED A request was made to the data processing center for the available maps of the students enrolled in beginning shorthand at Oakland Community College in the fall and winter semesters of l972-l973. The maps were reviewed in terms of teacher evaluations and the sample, upper and lower quartiles, composed of 6l students was established. The individual cognitive style maps were analyzed to deter- mine the common elements existing in the students' maps identi- fied as "highly successful" and "unsuccessful." A modification of Flanagan's technique was employed to determine those elements which would be included in the collective cognitive style of the "highly successful" and the "unsuccessful" beginning shorthand student. The process of visual inspection was employed to determine the differences which existed between the high and low or highly success- ful and unsuccessful student. Consideration was given to faculty and student comments made during the informal discussions which took place with faculty and students regarding attitudes toward the subject and reasons for enrolling in the course. CAROL F. STENCEL FINDINGS AND CONCLUSIONS The cognitive style maps of 61 students (32 highly successful and 29 unsuccessful) were analyzed. A collection of elements held in common from these maps determined the collective cognitive style for the "highly successful" student group and a collective cognitive style of the "unsuccessful" student group. The differences between the collective cognitive style of the "highly successful" student group and the "unsuccessful" student group were exhibited in all three sets of the maps. In the first Set, Symbols and their meanings, the highly successful group exhibited a "minor" in theoretical visual quantitative and qualitative visual with a "major" in qualitative code ethic. The unsuccessful student group displayed a "negligible" in theoretical visual quantitative and in qualitative visual with a "minor" in quali- tative code ethic. In the second set, Cultural Determinants, the highly successful student group displayed a "major" individuality. The unsuccessful collective cognitive style exhibited a "minor" individuality. In the third Set, Modalities of Inferences, the highly successful student group indicated a "major" in magnitude and difference with a "minor" in relationship. The unsuccessful student group displayed the appraisal Modality of Inference. Conclusions from the analysis were: (l) a student with theoretical visual quantitative and qualitative visual exhibited in his or her style would appear to have two very necessary characteristics for achieving CAROL F. STENCEL successfully in beginning shorthand; (2) a student with a "major" in magnitude and in difference, with a "minor" in relationship, would appear to have traits which would assist him or her in successful achievement in beginning shorthand; (3) a student with a "minor" individuality and a "minor" in qualitative code ethics may not per- form successfully in shorthand; and (4) a student exhibiting the appraisal mode would have difficulty developing speed in shorthand. EFFECTS OF EDUCATIONAL COGNITIVE STYLE ON BEGINNING SHORTHAND PERFORMANCE By Carol Frances Stencel A THESIS Submitted to Michigan State University in partial fulfillment of the requirements for the degree of DOCTOR OF PHILOSOPHY College of Education 1974 © Copyright by CAROL FRANCES STENCEL 1974 DEDICATION To my husband John, our three sons Gary, John III, and Michael, and to my mother. Your love, patience, and constant encourage- ment made the completion of this study possible. ii ACKNOWLEDGMENTS With great appreciation and sincere gratitude I thank Dr. Robert Poland for his support, guidance, and recommendations during the course of this study. As my advisor, and Doctoral Committee Chairman, he has given freely of his time and assistance whenever there was need. The other members of my Doctoral Committee also contributed greatly to the final product. Dr. Larry Borosage rendered support and encouragement. Dr. Helen N. Green assisted generously with questions and suggestions which aided in the completion of this study. Dr. Max Raines advanced valuable suggestions and incisive comments which added to the refinement of this investigation. I also wish to thank Dr. Joseph Hill for the inspiration and assistance he gave me. TABLE OF CONTENTS Page LIST OF TABLES ......................... xi LIST OF SAMPLE MAPS ...................... xii Chapter I. THE PROBLEM ...................... I NEED FOR THE STUDY .................. 5 PURPOSES OF THE STUDY ,,,,,,,,,,,,,,,, 6 HYPOTHESES ..................... 7 SIGNIFICANCE OF THE STUDY .............. 8 DEFINITION OF TERMS ,,,,,,,,,,,,,,,,, ll ASSUMPTIONS UNDERLYING THE STUDY .......... 12 DELIMITATIONS .................... l3 ORGANIZATION OF THE STUDY ,,,,,,,,,,,,,, 13 II. REVIEW OF RELATED LITERATURE .............. 15 EDUCATIONAL SCIENCE OF COGNITIVE STYLE . . . L . . . l6 Set I - Symbols and their Meanings ,,,,,,,, 18 Set 11 - Cultural Determinants for the Meaning of Symbols ............ 24 Set III - Modalities of Inferences ........ 26 Collective Cognitive Style of a Group 32 RELATED SHORTHAND STUDIES Relationships Between Aptitude Tests and Shorthand Achievement 32 Shorthand Studies Involving Multiple ....... Research Factors in Relation to Achievement .................. 37 SUMMARY ....................... 46 iv Chapter Page III. METHODS AND PROCEDURES ................ 48 GENERAL PROCEDURES ................. 48 Population ................... 48 The Sample ................... 55 Adequacy of the Sample ............. 57 DATA COLLECTION Conditions of Data Collection .......... 58 Instrumentalities and Methods .......... 58 The Instruments ................ 59 Methods .................... 66 Hypotheses ................... 69 Procedures of Data Collection .......... 69 SUMMARY ...................... 70 IV. ANALYSIS OF DATA AND FINDINGS ............ 72 ANALYSIS OF DATA ................. 72 FINDINGS ..................... 76 V. SUMMARY, CONCLUSIONS, IMPLICATIONS AND RECOMMENDATIONS .................. 85 SUMMARY ...................... Need for the Study ............... 85 Purposes of the Study .............. 85 Hypotheses Tested ................ 86 Procedures Used ................. 85 Findings .................... 87 CONCLUSIONS .................... 88 IMPLICATIONS ................... 94 RECOMENDATIONS .................. 96 vi Table 3.] LIST OF TABLES Total Enrollment of Full-Time Beginning Shorthand Students by Campus ........... Grade and Mark Distribution of Full-Time Beginning Shorthand Students by Campus ...... Grade and Mark Distribution of Beginning Shorthand Students Included in Defined Population of the Study ............. "Highly Successful" and "Unsuccessful" Beginning Shorthand Students by Campus ...... Collective Cognitive Style Map of "Highly Successful" Beginning Shorthand Group ...... Collective Cognitive Style Map of "Unsuccessful" Beginning Shorthand Group ............ Collective Cognitive Style of "Highly Successful" Beginning Shorthand Group ............ Collective Cognitive Style of "Unsuccessful" Beginning Shorthand Group ............ Differences, Referant "Highly Successful" Student Group .................. Differences, Referant "Unsuccessful" Student Group ...................... Page 53 54 LIST OF FIGURES Figure Page 4.l "Highly Successful" Student Sample Map ............... 73 4.2 "Unsuccessful" Student Sample Map ............... 74 4.3 Differences of "Highly Successful" Students Sample Map ............... 75 4.4 Differences of "Unsuccessful" Students Sample Map ............... 76 viii CHAPTER I THE PROBLEM One of the interesting educational considerations that has emerged in the past decade is that of the Educational Sciences. The Educational Sciences form a conceptual framework for education, and have been in the process of development by Dr. Joseph E. Hill and associates during the past seventeen years. Hill points out the fact that fundamental disciplines are bodies of knowledge generated by communities of scholars who produce pure and distinctive forms of information about pheno- mena which they study. Biology, history, art, psychology and mathematics are examples of fundamental disciplines.1 To illustrate the notion of "pure and distinctive forms of infor- mation," it may be recalled that Sigmund Freud, in dealing with the psychic-self (a pure and distinctive form of infor- mation at the time of its introduction by Freud), defined such terms as "Id," "Ego," and "Super ego," to explain in part this phenomenon. Complementing the fundamental disciplines are the applied or derivative fields of knowledge. These bodies of information are generated by practitioners who deal with practical consid- erations of the human condition. Medicine, pharmacy, engineering lHill, Joseph E., "The Educational Sciences" (unpublished manuscript, Wayne State University, 1971). l and law, are examples of applied fields of knowledge. In this context, "professionals" associated with these bodies of knowledge do not attempt to produce pure and distinctive forms of information about the "practical" phenomena with which they deal. Rather, they apply and implement these bodies of knowledge. The applied fields are composed of terms and methods of inquiry borrowed from the fundamental disciplines. The practitioners in these fields are not concerned with producing pure and distinctive forms of information. In this context, within each applied field of knowledge there are sections or aspects that could qualify as scientific bodies of information, while still other parts could be defined as "arts." These respective groupings of information within an applied field of knowledge, however, are designed to explain practical phenomena and solve actual problems found in those aspects of the human situation with which the practitioners of the "field" in question deal. For example, the applied field of knowledge called "medicine" is composed of the medical sciences and the medical arts; that of Engineering, of the engineering sciences and the engineering arts; and Pharmacy of the pharmaceutical sciences and pharmaceutical arts. The Educational Sciences provide a conceptual framework and scientific language for the applied field of knowledge called education. These "sciences" approach a level of precision that is found in such other derivative fields as medicine, pharmacy, engineering and law. According to Conant: 3 "...Teachers, like physicians, think in terms of predictive generalizations as well as arguments derived from general principles. Some people would like to combine these two modes of thought and speak of a single, all-embracing science of education. The question is whether it is useful to try to cover with the word 'science' a vast field of human activity directed toward practical ends. I have come to the conclusion that it is not. I prefer not to speak of the science of engineering but of the engineering sciences. I doubt that there is or ever will be a science of medicine, yet I am sure enormous strides forward have been made in the medical sciences. Therefore, I think it would be better to discuss...what might be called "educational sciences" or ”educational disciplines“ rather than the science or the discipline of education."2 There are those who believe that with the development of the Educational Sciences, the solutions of a variety of problems and explanations of many phenomena will be facilitated. In addition to these gains, many educational problems which frequently result from inadequate communication, misinterpretation of information, and fragmentation of effort may well be alleviated. At the present time, there are seven Educational Sciences. One of these sciences which deals with the student's educational cognitive style might show some promise for dealing with some of the problems associated with the high rate of failure experienced nationally in beginning shorthand classes. Opportunities for employment in careers in which shorthand is a requirement are increasing and the demand for employees with secretarial skills 2Conant, James B., The Education of American Teachers (New York: McGraw-Hill Book Company, 1960), p. 120. 4 are expected to remain high throughout the 1970's. Business educators are seeking methods and instructional strategies which will assist them in making the learning of shorthand a successful experience for all students in their shorthand classes. Each student has his own cognitive style, i.e., way of seeking meaning or coming to know. An individual's educational cognitive style is determined by the way he takes note of his total surroundings--how he seeks meaning, how he becomes informed. Is he a listener or a reader? Is he concerned only with his own viewpoint or is he influenced in decision-making by his family or associates? Does he reason as a social scientist, as an automotive mechanic, or as a mathematician? These are only a few examples of the facets of human learning ability that are included in the student's cognitive style. Family background, life experiences, and personal goals make each of us unique. Each map reflects each student's educational cognitive style. A cognitive map provides a picture of the diverse ways in which an individual acquires meaning. The map identifies his cognitive "strengths" and in another way, his weaknesses. This information can be used to build a personalized program of instruc- tion to alleviate some of the student frustration and lack of achievement in beginning shorthand. Results from a battery of tests and inventories are processed through a computer system to produce a map of cognitive traits that describe the many ways each student might seek meaning. “A“ s l‘ Cognitive maps are printed out in the form of a Cartesian product of three sets: (1) symbols and their meanings, (2) cultural determinants, and (3) modalities of inference. The first set indicates a student's tendency to use certain types of symbols, his ability to understand words and numbers, qualitative sensory symbols, qualitative programmatic symbols, and qualitative symbolic codes. The second set indicates influences which the student brings to __- ._-.' bear in deriving meaning from symbols. These influences are effected mainly in terms of his own individuality (I), or his associate's (A) I perceptions, or those of his family (F). The third set indicates the manner in which he reasons, or the way in which he infers. Whether he thinks in categories (M), magnitude, or in terms of differences (0), or synthesizes multiple relationships (R), or uses all three of these forms with equal weight to draw an inference of appraisal (L). His modality of inference influences and is influenced by, symbols and the cultural determinants he employs in his style. These three sets of elements, i.e., symbolic mediation, cultural determinants, and modalities of inference, comprise the cognitive style of the individual. 1 NEED FOR THE STUDY Occupational Opportunities for students with secretarial skills are unprecedented. The U. S. Department of Labor estimates that during the next ten years, 200,000 new employees will be needed 6 annually for secretarial jobs. Occupational opportunities for business graduates during the years 1970-1975 will be numerically greatest for secretaries and stenographers. Employment in this field will rise from 2,225,000 to 2,825,000 by 1975.3 Research indicates that the percent of students who drop shorthand is high, ranging from 18 to 29 percent of the total beginning enrollment. Moreover, the percent of students who fail shorthand is reported to be even higher-~with the estimated failure rate to be as high as 50 percent.4 The problem of high failures compounded with perhaps an even greater educational failure--student frustration and vocational disappointment and delay--add impetus to the need for a study of the possible effects of educational cognitive style on achievement, or lack of achieve- ment, in shorthand. PURPOSES OF THE STUDY The purpose of this study is threefold: (1) to provide information about the types of students who might have a high probability of achieving successfully in that aspect of the secretarial science program associated with shorthand; (2) to l provide students with information about themselves so that if 3U.S. Department of Labor, "Occupational Outlook for Stenographers, n«- Secretaries, and Typists," Occupational Outlook Report Series, 1966 67 (Nasgington: U.S. Department of Labor, Government Printing Office, 1966 , p. 3. 4Frink, Inez, "A Comprehensive Analysis and Syntheses of Research and Thought Pertaining to Shorthand and Transcription 1946-1957" (unpublished Doctor's dissertation, Indiana University, 1961), pp. 168-98. they show minimal potential for shorthand, they may, if they wish, choose another area of study; and (3) to provide information regarding the methods and instructional strategies that may need to be employed to accommodate the "style" of a student who has exhibited little potential for shorthand. The purpose of the study will be realized by (1) obtaining the educational cognitive maps of the upper quartile students (determined by teacher evaluation) in the mastery of the shorthand skill, (2) obtaining the maps of the lower quartile students, (3) determining the collective cognitive style of the upper quartile group, and (4) determining the collective cognitive style of the lower quartile group, and (5) analyzing each student's cognitive map to identify those profiles which would help to explain that student's success, or failure, to master the beginning shorthand skill. The findings of the present study should assist teachers of shorthand to determine various methods in assisting students in the learning of shorthand. The study should also provide forms of answers to instructional problems that have arisen in the area of shorthand instruction. The findings may well suggest clues to personalizing instruction and also provide information that should assist in career counseling of students. HYPOTHESES The objectives of the study were to determine (1) whether there were common cognitive traits for students demonstrating "success" in beginning shorthand and (2) whether there were common Jav.ifl cognitive traits for students who were "unsuccessful" in their attempt to learn the beginning shorthand skill. The research hypotheses tested in this study were: H1: There is a collective cognitive style which is character- istic of the group of "highly successful" beginning shorthand students. H2: There is a collective cognitive style which is character- istic of the group of "unsuccessful" beginning shorthand students. H3: There is a difference between the collective cognitive style of the "highly successful" and the "unsuccessful" beginning shorthand student groups. SIGNIFICANCE OF THE STUDY No area of shorthand instruction has received more attention than that of the search for those student background factors which lead to success or failure in the learning of the subject. From a starting point in 1912, well over 200 studies may be identified as having been directed in this area of concern. The majority of these studies were prompted by the large number of dropouts from beginning shorthand classes. Dropouts from the first semester of instruction to the last semester of instruction is estimated to be as high as 50 to 75 percent.5 More recent studies, however, are prompted by the h0pe that differentiated instruction might be 5Lambrecht, Judith J., "Aptitude Testing in Shorthand," Business Education Forum, (October, 1972), pp. 17-24. provided for students when it can be determined that they possess more or less of those abilities or traits considered to be important to the development of beginning shorthand skill. In terms of enrollment, shorthand continues to remain among the top three business subjects offered in high school and collegiate departments 6 The problem of high failures, high dropout rates, of business education. and low achievement continues to plague shorthand teachers and guidance counselors. There is general agreement with the finding that a combination of student background traits and abilities is a better measure to predict success or failure than any one factor alone. When narrowing it down to one factor with the greatest support throughout the literature, verbal measures of ability seem to best predict student achievement in shorthand. Approximately seventy-five studies have considered the correlation between general intelligence and success in shorthand. 0n the basis of these findings, general intelligence has too low a correlation with shorthand achievement to serve alone as a valid measure of shorthand aptitude.7 Foreign language ability, and in particular the ability to . handle artificial language symbols, has been shown to be related to A success at the beginning stages of shorthand instruction. 6Tonne, Herbert A., P0pham, Estelle L., Freeman, M. Herbert, Methods of Teaching Business Subject, (New York: McGraw-Hill Book . Chmpany, 1965), p. 3. 7Lambrecht, p. 19. 10 Two shorthand aptitude tests were developed at about the same time, the Turse Shorthand Aptitude Test (1940) and the E.R.C. Stenographic Aptitude Test (1947). Both tests were directed toward two groups of basic abilities felt necessary for success in shorthand: (1) those abilities which enable a person to write shorthand from dictation and (2) those abilities which enable a person to effectively use his written shorthand notes. The present study should provide additional information regarding these matters. Why some students succeed and others fail in shorthand is a complex problem. Turse believes that: "If the difference noted among individuals can be related to differences in test scores, then we shall have instruments which may make valuable contributions toward fulfillment of the guidance ideal of individual job adjustment and satisfaction."8 The fact that educational cognitive style provides a vehicle for examining individual differences among students taking short- hand is a significant fact in itself. Prognosis is an area of considerable interest in shorthand research. At this time, however, it is not possible to predict success in shorthand with any accuracy. Several methods are now available by which a student may learn shorthand (e.g., programmed textbook, individual progress methods). With these choices available, students of varying aptitude may profit from the different approaches used. Guidance through the use of cognitive 8Turse, Paul L., "Prognostic Studies in Business Education," National Business Education Quarterly, XXXXV (Winter, 1966-1967), p. 53. 11 style mapping could be invaluable where the student has a choice of method. The present study is designed to determine whether there is a collective cognitive style profile for the ”highly successful" beginning shorthand student and a collective cognitive style profile for the "unsuccessful” beginning shorthand student. With this information, personalized programs of instruction could be developed to assist the student in his or her efforts to achieve success in the subject of shorthand. DEFINITION OF TERMS The following terms are defined according to the way they are used in this study: Beginning_shorthand. A course designed to teach the theory of Gregg Shorthand and to develop the ability to transcribe from dictated material at an appropriate speed. In this study beginning shorthand was taught in a 14-week semester. Educational Cognitive Style. The fifth "science" of the conceptual framework called the Educational Sciences is that of educational cogni- tive style. Comprised of three sets of information, cognitive style describes an individual's mode of behavior in the process of searching for meaning. It is identified by an individual's disposition to use certain symbolic forms versus others. These symbolic meanings are influenced by the roles,9 which an individual has assumed throughout 9"Role refers to the behavioral consistencies on the part of one person as he contributes to a more or less stable relationship with one or more persons." See Theodore Newcomb, Ralph Turner and Phillip Converse, Social Psychology, (Holt, Rinehart, and Winston, Inc., 1965), p. 323. r‘*7 "T‘- 124 his life span, and both of these factors ("symbols" and "roles") are influenced by the manner in which he reasons. Cognitive Style Map. At the present time the "map" is a graphic display of no more than thirty-two variables that describe how each individual derives meaning from his environment and personal experiences (e.g., pair). Educational Sciences. The Educational Sciences form a conceptual framework for the applied field of education. This structure provides not only a framework, but also yields a universe of discourse (common language) for educators as well. At the present time, the Educational Sciences are sets of information (concepts) pertaining to: 1. Symbols and their meanings; 2 Cultural determinants of the meanings of symbols; 3. Modalities of inference; 4 Biochemical and electrophysiological aspects of memory-concern functions; 5. Cognitive styles of individuals; 6. Teaching, administrative, and counseling styles; and 7. Systemic analysis decision-making. ASSUMPTIONS UNDERLYING THE STUDY The following assumptions are necessary to this study effort: A]: The mastery of the shorthand skill is a worthwhile endeavor for employment purposes in occupations requiring shorthand. 13 A2: Subjects will respond honestly to all questions during the interview according to their ability and perception. A3: The validity and reliability factors of the Oakland Community College Cognitive Test battery are sufficient for the purposes of this study. A4: Students participating in this study will have approximately the same kind and quality of teaching. A5: The assumptions associated with the Educational Sciences, see page 17, Chapter II. DELIMITATIONS The study effort is limited to students enrolled in the beginning shorthand classes at three campuses of Oakland Community College during the l972-l973 academic year. In addition, the study is concerned only with the variables specifically selected, although it is recognized that many other factors may be instrumental to the student's successful achievement in any course. No students will be included in the study if they have taken shorthand at an earlier time, and the sample employed in the study includes only those students in the upper quartile, and those in the lower quartile according to the standards established by the department. ORGANIZATION Chapter I has presented the purpose and importance of the study. Chapter II contains a review of the literature related 14 to the study. Chapter III outlines the design of the study and the procedures followed. Analysis of data and findings are included in Chapter IV, and Chapter V contains the conclusions, implications, and recommendations of the study effort. CHAPTER 11 REVIEW OF RELATED LITERATURE In an attempt to seek a solution to the high failure problem among beginning shorthand students, investigators have developed numerous instruments designed to identify those traits which would predict shorthand success. This investigation was designed to determine if a collective cognitive style of "highly successful" and of "unsuccessful" beginning shorthand student groups, respectively, exists. Under these circumstances, a review of literature pertaining to a variety of problems associated with the instruction of shorthand is important. The review of the literature is divided into two parts: l. the Educational Science of "cognitive style," and 2. related shorthand studies which a. examine the relationship between aptitude tests and shorthand achievement, and b. those that involve multiple research factors in relation to achievement. 15 16 THE EDUCATIONAL SCIENCE OF COGNITIVE STYLE* The conceptual framework called the "Educational Sciences" has been implemented at Oakland Community College since 1969. Basic research on the "Educational Sciences" was conducted at Wayne State University, Detroit, Michigan, over a period of approximately thirteen years which led to the development of the main ideas and concepts. Hill10 defines the "Educational Sciences" as a conceptual framework consisting of seven bodies of knowledge which pertain to: 1. symbols and their meanings; 2. cultural determinants of the meanings of symbols; 3. modalities of inference; 4. biochemical and electrophysiological aspects of memory-concern functions; 5. cognitive styles of individuals; 6. teaching styles, administrative styles, and counseling styles; and 7. systemic analysis decision-making. The following assumptions are considered to be essential to these bodies of knowledge: *The discussion which follows is comprised of paraphrased excerpts from a number of unpublished manuscripts by Joseph E. Hill. Permission has been granted by the author. See the bibliography for a complete list of these manuscripts. 10Hill, Joseph E., "The Educational Sciences" (unpublished manuscript, Wayne State University, 1968). 17 A]: Education is the process of searching for meaning. A2: Not content with biological satisfactions alone, man continually seeks meaning. A3: Thought is different from language. A4: Man is a social creature with an unique capacity for deriving meaning from his environment and personal experiences through the creation and use of symbols. The construct of cognitive style employed in the "Educational Sciences" is somewhat different from those defined in the field of psychology. For the purpose of the present study, cognitive style is defined as the Cartesian product of three sets, Set I, Symbols and their Meanings; Set II, Cultural Determinants of the Meaning of Symbols; and Set III, Modalities of Inference. Employing a form of Guttman's metatheory of facets as a 1,1] the concept of cognitive style is expressed as a mode Cartesian product of sets. In this context, cognitive style as employed in the "Educational Sciences“ can be considered somewhat related to Guilford's Nature of Human Intelligence,12 where Guilford's model is a Cartesian product of three sets that represent intellect, content, and "things.“ In a different vein, but directed toward the recognition of individual difference, the idea of learning sets was developed by 11The term "facet" was formally proposed by Louis Guttman in "An Outline of Some New Methodology for Social Research" Public Opinion Quarterly, 18 (Winter, 1954-1955), pp. 393-404. 12Guilford, J. P., Nature of Human Intelligence (New York: McGraw-Hill Book Company, Inc., 1967). 18 (Robert Gagné and his associates. Gagne defines learning sets as the capabilities the student has at any given stage in the learning of a given task. Although these abilities are internal to the learner, they are directly measurable as performances. It is possible to identify the sets needed to learn a particular task by working backward from the selected task to the prerequisite abilities. The simple and more general abilities form a hierarchy Gagné terms a learning structure.13 Set I - Symbols and their Meanings Set 1, Symbols and their Meanings, represents an essential aspect of man's process of searching for meaning. Symbols are a differentiating characteristic of human functioning. There are two types of symbols, theoretical--words, numbers, and onomatopoeia--and qualitative--sensory, programmatic, and codes--which are created and used by individuals to acquire knowledge or derive meaning from their personal experiences and environments. At the present time, the Educational Science of "Symbols and their Meanings" has defined four theoretical and twenty qualitative symbols which form the symbolic aspect of meaning within this conceptual framework. Since the purpose of determining an individual's orientation in each of the elements included in the three sets comprising his 13Gagné, Robert M., "The Acquisition of Knowledge," Psychological Review, 69 (1962), pp. 355-365. -L 19 cognitive style is to determine his “strengths" at a given level of educational development, the following principles apply: 1. If the percentile rank of an individual's score in a given element is in the spread of values ranging from the fiftieth through the ninety-ninth percentile (inclusively) of a population of these scores, the individual is assigned a major orientation, written as a capital letter T (VL). 2. If the percentile rank of an individual's score in a given element is in the spread of values ranging from the twenty-sixth through the forty-ninth percentile (inclusively) of a population of these scores, the individual is assigned a minor orientation, written T'(VL) and spoken T prime VL. 3. If the percentile rank of an individual's score in a given element is in the spread of values ranging from the zero percentile through the twenty-fifth percentile (inclusively) of a population of these scores, then the individual is assigned neither a major nor a minor orientation, and the element is not entered as a part of the cognitive style.14 A theoretical symbol (T), which is either a word or a number or onomatopoeia, is that symbol which presents to the awareness of the individual something other than that which the symbol itself is. 14Wasser, Laurence, The Educational Science of Cognitive Style: An Introduction (Oakland Community C011ege Press, 1971, p. 8. 20 The theoretical symbol for purposes of definition is divided into two parts, the linguistic and the quantitative elements. They are as follows: 1. Theoretical visual-linguistic T (VL), T'(VL) Indicates the ability to acquire meaning from words you read. The meaning conveyed can be interpreted individually (percepts) and generalized “II-1"! .5. (concepts). 2. Theoretical auditory-linguistic T (AL), T'(AL) Represents the ability to find meaning from the ) spoken word. The sound of words creates for the listener a set of imageries which convey meanings different from the sounds themselves. Again, the meaning of the sounds can be both interpreted individually (percepts) and generalized (concepts). 3. Theoretical visual-quantitative T (VQ), T'(VQ) Represents the ability to acquire meaning in terms of written numerical quantities, functional relations 1 which express conditions of equality or inequality, 1 and fundamental operations. I 4. Theoretical auditory:quantitative T (A0), T'(AQ) FT" Indicates the ability to find meaning in terms of hearing numerical quantities, functional relations which express conditions of equality or inequality, and fundamental operations. 21 Qualitative Symbols (Q), are those symbols which present and then represent to the awareness of the individual that which they--the symbols--themselves are to that individual. Qualitative symbols derive their meanings from three sources: (1) the five perceptive senses, (2) programmatic sources, and (3) humanly constructed formalisms such as "games" or "codes." The qualitative symbols associated with the five senses are: l. Qualitative The ability of hearing. auditory Q (A), Q'(A) I to perceive meaning through the sense For example, a major in this area would have the ability to distinguish differences in sounds. It is not uncommon for a parent to be able to distinquish a qualitatively based difference in the cry of a baby having a serious problem from a cry of lesser significance. 2. Qualitative Olfactory Q (0), Q'(O) The ability of smell. 3. Qualitative The ability 4. Qualitative to perceive meaning through the sense savory Q (S). Qf(S) to perceive meaning through the sense of taste. tactile Q (T), Q'(T) The ability 5. Qualitative The ability to perceive meaning through the sense of touch. visual Q (V), Q'(V) to perceive meaning through the sense of sight. For example, the awareness derived from viewing a work of art. 22 There are ten symbols associated with the qualitative codes "source of meaning." They are: 1. Qualitative code-empathetic Q (CEM), Q](CEM) The ability of the individual to identify with another person's role, feelings, or ideas. 2. Qualitative code-esthetic Q (CES), Q](CES) The ability to view with enjoyment the beauty of a product, situation, or idea. 3. Qualitative code-ethic Q (CET), Q'(CET) The commitment to a set of values. For example, a group of moral principles, obligations or duties. 4. Qualitative code-histrionic Q (CH), Q'(CH) Represents artificial behavior, staged conduct, or a deliberate exhibition of emotion or temperament to produce some particular effect in, or evoke responses from other people. 5. Qualitative code-kinesics Q (CK), Q'(CK) The ability to recognize and to communicate (encode and decode information) by means of non-linguistic functions and motions of the body, such as blushes, shrugs, and gestures. 6. Qualitative code-kinesthetics Q (CKH), Q'(CKH) Represents the ability to simulate motor skill abilities and bodily reactions such as those found in a variety of activities (e.g., ballet, finger dexterity, athletic performances). 10. Programmatic "sources of meaning" are: 1. 23 .' Qualitative code-proxemics Q(CP), Q'(CP) The ability of an individual to judge "critical" physical and social distances between himself and others in the act of communicating. This symbolic code pertains to the individual's ability to judge and effect the type of social and physical distance that the other person will permit between himself and others. Fr Qualitative code-synnoetics Q(CS), Q'(CS) The personal knowledge of oneself in all qualitative and theoretical symbolic forms which constitute the totality ' of the individual in relation to his environment. Qualitative code-transactional Q (CT), Qfi(CT) The ability to maintain a positive communicative interaction which influences positively the outcomes and the goals of the persons involved in that interaction. Qualitative code-temporal Q (CTM), Q] (CTM) Ability to respond or behave according to time expectations imposed on an activity by members in the role set associated with that activity. i Qualitative pruprioceptive Q (P), Q'(P) The ability needed to synthesize stimuli produced within the body and expressed through the action of the nervous system in tasks such as typing, playing a musical instrument or other complex sequential activities. 24 2. Qualitative proprioceptive kinematics Q (PK), Q'(PK) The ability to synthesize a number of symbolic mediations into a performance of a complex activity involving motion e.g., riding a bicycle. 3. Qualitative pr0prioceptive dextral Q (PD), Q'(PD) A typically right-handed person would exhibit a dominance of right-eyed, right-handed, right-footed [' tendencies while synthesizing a number of symbolic mediations in the performance of a task e.g., typewriting, playing a musical instrument. 4. Qualitative proprioceptive sinistral Q(PS), Q'(PS) A typically left-handed person would exhibit a dominance of left-eyed, left-handed, left-footed tendencies while synthesizing a number of symbolic mediations in the performance of a task e.g., typewriting, playing a musical instrument. 5. Qualitative proprioceptive temporal Q (PTM), Q'(PTM) Ability to synthesize a number of symbolic mediations into the performance of a complex physical activity which involves timing such as hitting a baseball effectively. Set 11 - Cultural Determinants for the Meaning of Symbols Man's perceptions influence and are influenced by his culture. Social relationships are, therefore, significant factors in the development of the perceptions which an individual has of his "world." 25 The basis for the Science of Cultural Determinants is that education takes place within a social context. The formation and transformation of the meaning of symbols, in both the theoretical and the qualitative domain are influenced by culturally created rules of expression and communication. The three elements of Set II are: (1) Family (F, F'), (2) Associates (A, A'), and (3) Individuality (I, 1'). One, but no more than two, of these elements has a major influence, while the other one, or two, exerts a minor influence on the meanings of symbols. The cultural influence of these three elements is not viewed in detail in the "sciences” as they might be in the fields of sociology or psychology. The concern of the educational scientist is directed towards these elements' influence upon the meaning of symbols of the person under consideration. The three cultural determinants are defined as follows: 1. Family F, F' The family determinant indicates that a member, or members of the person's family, or very close personal friends, exert influences on his symbols and their meanings. 2. Associates A, A' This determinant indicates that an individual is influenced by his associates in the way he perceives symbolic meanings. 3. Individuality I, I' The individuality determinant indicates that symbols and their meanings not only influence this type of person but also are influenced by that individual. 26 Set III - Modalities of Inference The mode of inference which the individual tends to employ in the process of deriving meanings also affects the interaction between the meanings of symbols and "determinants" found in his "style." A modality of inference pattern can be classified as either an inductive process or a deductive process, depending upon the manner in which the conclusion is reached. The third set of the Cartesian product indicating cognitive style includes elements which indicate the person's modality of inference. It is thought that man can draw upon only four inductive inference processes: (1) Magnitude (M, M'), (2) Difference (D. 0'), Relationship (R, R'). and (4) Appraisal (L, L'). They are as follows: 1. Magnitude M, M' Categorical thinking that utilizes norms, definitions, or classifications as a basis for making decisions. Persons who need to organize or define things categori- cally in order to understand them reflect this modality. 2. Difference D, D' This pattern utilizes contrasts or comparisons of selected characteristics or alternatives in order to understand. These types of "reasoners" develop a definition by determining what something is ugt_rather than what it is. Creative writers frequently possess this modality. _'.117 “‘“‘"——‘*- “‘ 4 ' i an“. -sA a...__ 27 3. Relationship R, R' The relationship modality suggests the ability to synthesize a number of elements or incidents into a meaning or conclusion. 4. Appraisal L, L' This pattern involves the use of all three of the above modalities (M, D, and R) with equal weight in coming to some meaning or conclusion. 5. Deductive K The symbol (k) is used to denote the deductive infer- ential process or the form of logic used in geometry. The process of education may be considered one in which the individual in this "culturally determined" context perceives and interprets theoretical and qualitative symbols in order to arrive at conclusions involving them by means of a variety of "modalities of inference." The uniqueness of each individual in this educational context then, may be expressed in terms of: 1. The symbols he possesses, 2. The cultural determinants of the meaning of these symbols, and 3. The modalities by which he draws inferences. For each individual there exists a unique interplay of these elements as he is engaged in the process of knowing (cognition). This distinctive mode (style) is his "cognitive style."15 15Wynn, Charles M., "The Educational Sciences: A bird's Eye View" (unpublished paper, Oakland Community College, 1973). c- I—‘ 28 A number of studies completed in the educational sciences related the educational science of cognitive style to curriculum and 16identified student performance in certain disciplines. Keith Shuert the elements of cognitive style held in common by successful math students, those elements held in common by unsuccessful math students, those elements unique to the successful math student, and those elements unique to unsuc- cessful math students. James Blanzy17was able to identify distinct collective cognitive styles for: (1) students whose achievement of performance goals placed them in the upper or lower 27 percent of the class, (2) students who withdrew, and (3) students with the most positive or least positive attitude toward mathematics. The results of ‘8 implied that among a group of sixty-seven college a study by James Warner freshmen in a life science course divided into an experimental group using a self-instructional, multi-media approach and a control group taught by the lecture-discussion method, certain elements of cognitive style appear to be amenable to instruction by a self-instructional, multi-media approach, 16Shuert, Keith L., "A Study to Determine Whether a Selected Type of Cognitive Style Predisposes One to 00 Well in Mathematics" (unpublished Doctor's dissertation, Wayne State University, 1970). 17Blanzy, James J., "Cognitive Stule as an Input to a Mathematics Curriculum System: An Exploratory Study in the Educational Sciegces" (unpublished Doctor's dissertation, Wayne State University, 970 . 18Warner, James L., "An Analysis of the Cognitive Styles of Community College Freshmen Enrolled in a Life Science Course" (unpublished Doctor's dissertation, Wayne State University, 1970). -._._ 4— ....-. I 29 while others seem more responsive to the lecture-discussion method. Warner also identified predominant elements of cognitive style demon- strated by successful and unsuccessful students in both groups. The relationship of these studies to the current effort is obvious. ‘9 identified collective cognitive styles for four Vaughn Hoogasian hundred seventy-two students who earned various letter grades in a two-course sequence in English, but the collective style did not prove to be a definitive predictor of a final letter grade in the courses when applied to the individual student who approximated one of the collective cognitive styles for any of the letter grades. Glenn McAdam20 found that in a class of fifteen community college students, those students with a cognitive style "match” with the teacher's expressed a positive attitude about their instructional experience activities in the class. Those students with a negligible match with the teacher expressed a lack of high interest in the class activities. Several studies investigated the relationship of cognitive style to other areas of education. Arlan Schroeder.21 working with 118 ninth grade students and one teacher, and Lawrence Wasser,22 working with 30 19Hoogasian, Vaughn, "An Examination of Cognitive Styles Profiles as indicators of Performance with a Selected Discipline" (unpublished Doctor's dissertation, Wayne State University, 1970). 20McAdam, Glenn F., "Personalizing Instruction through the Educa- tional Sciences of Cognitive Style and Teaching Style" (unpublished Doctor's dissertation, Wayne State University, 1971). 2ISchroeder, Arlan, "A Study of the Relationship Between Student and Teacher Cognitive Style and Student Derived Teacher Evaluations" (unpublished Doctor's dissertation, Wayne State University, 1969). 22Wasser, Lawrence, "An Investigation into Cognitive Style as a Facet of Teachers' Systems of Student Appraisal" (unpublished Doctor's dissertation, University of Michigan, 1969). r. W..- ‘.-_ .W‘M” Mgr 30 sixth grade pupils and their teachers, used the educational sciences construct of cognitive style to investigate teacher evaluation of student achievement. Both found that students with cognitive styles similar to the teacher's cognitive style received a significantly greater number of high-letter grades (A's and 8's) than did those whose cognitive styles were "different" from their teacher's style. Schroeder also found that students with cognitive styles similar to that of the teacher's, to a significant degree, evaluate the teacher as being more effective than do students with cognitive styles disjunct 23 with the style of the teacher. Marvin Fragale found similar results studying thirty-two students and two instructors in a community college. Fragale was also able to identify a collective cognitive style for industrial technology teachers and for industrial technology students. In addition to the foregoing discussion of studies in the science of cognitive style, it is desirable to investigate other possible uses of the cognitive style map. It is hoped that knowledge of an individual's cognitive style will permit prescribing appr0priate educational activities to provide a high probability of successful accomplishment of an educational task before him. It is also h0ped that an individual made knowledgeable of his own cognitive style will be better able to cape with any educational "situation" which he may experience. 23Fragale, Marvin J., "A Pilot Study of Cognitive Styles of Selected Faculty Members and Students in a Community College Setting" (unpublished Doctor's dissertation, Wayne State University, 1969). 31 Research and literature on the interpretation and use of individual cognitive style maps is sparse. Therefore, it is the researcher's intent to describe briefly and to point out the practical application of the "map" and its use in the present study. This description is not meant to be comprehensive, but rather illustrative, in order to clarify ensuing presentations in relation to cognitive style. Cognitive style maps of individual students are utilized :1 by counselors in individual student counseling sessions, and in the course or program advisement interview prior to the student's registration at Oakland Community College. After the student is ’- registered, a faculty member may request maps of individuals or of her entire class for instructional use. The cognitive style map heading contains the student's name, social security number, sex, and campus location. The first numerical listing on the left is the level of educational development. Results from Test 1 and Test 6 of the cognitive style battery establishes the level of educational development. Results are also translated into _ the element of theoretical visual linguistic (TVL). The level of educational development is based on secondary school grade level equivalents. The binomial combination of theoretical and qualitative symbols appear in Set I - Symbols and their Meanings. Set I, the first 'set' on the left, of the Cartesian product lists the major and minor orientations of the individual in the theoretical and qualitative symbolic elements. 32 The theoretical and qualitative symbols as previously defined, are denoted in terms of major and minor orientations by the use of all capitals for "majors" and the use of (') "prime,“ for those symbols showing "minors." Collective Cognitive Style of a Group_ The concept of a "collective cognitive style" of a group is an extremely useful research tool in the Educational Sciences. A collective cognitive style can be identified by determining those elements which appear in 70 percent or more of the maps for the group of persons under consideration. It is necessary to set basic criteria for including elements in the 'style' when developing a 'collective cognitive style.’ Flanagan's technique for determining the true upper end of a distribution is used as the basic aspect of the approach?4 RELATED SHORTHAND STUDIES Relationships Between Aptitude Tests and Shorthand Achievement Many investigators have studied the relationship between beginning shorthand achievement and published aptitude tests. The findings have yielded low results. The earliest test, developed in 1922, was the Hoke Prognostic Test of Stenographic Ability. Only a few of the many reports of its use gave this seven-test battery a favorable recommendation. Gregg Publishing Company took it out of print in 1948. 24Flanagan, John C., "General Considerations in the Selection of Test Items and a Short Method of Estimating the Product-Moment from the Data at the Tails of the Distribution," Journal of Educational I 0—2.4. ‘-h' Psychology, (1939), pp. 674-680. 33 Blanchard25 did a comprehensive study in fifty-one high schools and colleges and found no correlation between the Hoke test battery and shorthand achievement. Jessup,26 Kessinger,27 wood,28 and Terrill,29 supported these findings. A second test, the Bennett Stenographic Aptitude Test (1939), was developed through extensive analysis of differential prediction. 30 the scope of the test for predicting success According to Lambrecht, in "secretarial training" indicated that its breadth extended beyond those competencies required primarily for the shorthand skill. Two other shorthand aptitude tests were developed at approximately the same time. The Turse Shorthand Aptitude Test (1940), and the Educational Research Corporation Shorthand Aptitude Test (ERC) (1947). 25Blanchard, Clyde 1., "Results of a Study of the Validity of the Hoke Prognostic Tests of Stenographic Ability," The American Shorthand Teacher, (January, 1930), p. 196. 26Jessup, E., "A Prognostic and Achievement Test in Shorthand," Journal of Commercial Education, LVIII (June, 1928), pp. 173-74. 27kessinger, E., "A Prognostic Study in High School Shorthand" '(unpublished Master's thesis, Louisiana State University, 1936). 28Wood, Ethel H., "Correlation of Prognostic Test and Will-Temperament Tests with Actual Results in Gregg Shorthand" (unpublished Master's thesis, Washington State College, 1928). 29 Terrill, Chester J., "The Value of the Hoke Prognostic Test of Stenographic Ability as a Means of Selecting Shorthand Students" (unpgblished Master's thesis, New York State College for Teachers, 927 . 30Lambrecht, Judith J., "The Validation of a Revised Edition of the Byer's Shorthand Aptitude Test“ (unpublished Doctor's thesis, University of Wisconsin, 1971). 34 Maedke's3] study of 490 high school students included the relationship between the Turse Test and shorthand achievement. He demonstrated a correlation of .45 between the Turse Test and shorthand achieve- ment for the beginning shorthand student, and a correlation of .58 between the Turse Test and advance shorthand student achievement. k32 supported Maedke's study by reporting a higher Jac correlation existed between the Turse Test and advanced shorthand : achievement, .51, than beginning shorthand students, .32. To the ~ extent that the present study is designed to determine if cognitive style might be related to performance in shorthand, it is considered 5 to be related to the syndrone of studies discussed in this section. In a study on the relation between 75 beginning college shorthand students on both the Turse Test and the Educational Research Corporation Shorthand Aptitude Test (ERC), Hosler33 reported a correlation of .79 between the two tests. He also found that both gave almost identical results when correlated with intelligence scores. Hosler concluded that neither test should be used as the determining factor in predicting shorthand success. Both the Turse Test and ERC appear to have been directed toward the prediction 3lMaedke, Wilmer 0., "The Relative Prognosis Value of Selected Criteria in the Prediction of Stenographic Success or Failure in Selected High Schools in Illinois" (unpublished Ph. D. dissertation, .1- Northwestern University, 1957). 32Jack, Melvin C., "Can We Predict Success in Shorthand?" The Balance Sheet, XXXIII (January, 1952), pp. 212-19. 33Hosler, Russell J., "Aptitude Testing in Shorthand," Journal of Business Education, XXII (May, 1947), p. 25. 35 of the final stenographic objective of the transcription skill. This is born out by the higher correlations which resulted for advanced shorthand students as opposed to beginning shorthand students. The English skills so necessary in transcription appear to be stressed in the subtests of these two batteries. Those specific shorthand-learning capabilities important in the beginning stages of the course do not receive the attention which they might.34 The most recently published shorthand aptitude test was developed by Edward E. Byers in 1958»35 This test was directed toward predicting student achievement in the first semester of instruction in Gregg 36 compared the relation between high school shorthand. Takasugi shorthand grades and the scores obtained on the Turse Test (.53) and the Byers' First-Year Aptitude Test (.60). It was found that the Byers' Test was somewhat better than the Turse but concluded that neither should be used as a single device for predicting success or failure to succeed in beginning shorthand. Byers constructed five subtests for his original battery. They were: (1) Phonetic Perception, (2) Hand Dexterity, (3) Obser- vation Aptitude, (4) Retention Ability, and (5) Patterns from Parts. 34Lambrecht, Judith, "Aptitude Testing in Shorthand," Business Education Forum, XXVII (October, 1972), p. 19, 35Byers, Edward E., "Construction of Tests Predictive of Success in First-Year Shorthand" (Doctor's thesis, Boston University, 1958). 36Takasugi, Dorothy, "The Relationship Between Certain Psycholo- ical Tests and Other Selected Factors with Shorthand Achievement" Iunpublished Master's thesis, University of Southern California, 1961). 36 The importance of each of the subtests varied. The "Patterns from Parts" subtest was the weakest in the battery. This test attempted to measure a student's ability to locate missing parts in words in written context. Wright37conducted a study with 36 students and concluded that because the correlations obtained between the Byers' Tests and shorthand achievement were .3737 and .1511, they were not high enough to predict accurately success in the system in which she conducted the experiment. She discovered, however, that a correlation of .6822 and .0940 resulted when she compared the Phonetic Perception Subtest of Byers' Test with Shorthand achievement. This was, however, one subtest of the battery. There appears to be general agreement in the literature that the use of published aptitude tests as a single predictor for success or failure in beginning shorthand is not recommended. It is possible, however, that scores from the published aptitude tests together with measures of other factors, not yet defined, could be used to forecast performance of an individual's potential to succeed in beginning shorthand. It is in this context that the present study is considered to be related to these efforts. 37Wright, Ellen M., "A Summary of Recent (1940-1962) Selected Findings in Shorthand Prognosis with Specific Reference to the Use of the Byers' First-Year Shorthand Aptitude Tests at the High School in Southington, Connecticut" (un ublished Master's thesis, Central Connecticut State College, 1963 , pp. 38-40. 37 Shorthand Studies Involving Multiple Research Factors in Relation to Achievement Many researchers have studied and compared the relationship between a number of factors and achievement in beginning shorthand. Among the factors which have been considered are English grades, scholastic achievement (GPA), foreign languages, reading ability, general intelligence, typewriting, and various personal character- istics. Frink38 noted considerable interest in prognostic measures among researchers and indicated that a great number of investigators found grade-point averages excluding English to be the best single predictor of success in shorthand of any factor studies. None of the factors, however, yield a high enough correlation to be considered the sole factor for use in prediction. Eyster39 recommended that a battery of tests be used to predict shorthand success rather than relying on a single prognostigating factor. He conducted one of the most extensive predictive studies based on multiple research factors. Eyster compared five predictive factors and shorthand achievement: (1) average English grades, (2) average of all high school grades exclusive of English, (3) mental ability, (4) scores on a stenographic prognostic test, and (5) subjec- tive trait ratings (work habits, personality, and character traits). 38Frink, p. 40. 39Eyster, Elven S., "Prognosis of Scholastic Success in Shorthand," The National Business Education Quarterly, XVII (December, 1938), pp. 31-34. .. m..——_ _-‘_._A______.._._ _._ 38 There were a total of 617 high school students which Eyster divided into three groups: (1) those he predicted would succeed in shorthand, (2) those he prediCted would have a 50-50 chance of succeeding, and (3) those he felt had little chance for success in shorthand. The students' teachers were not informed of the students' classifications. 0f the students whose success was predicted, 97.6 percent succeeded. 0f the students whose failure was predicted, 100 percent failed. Of the students predicted as having a 50-50 chance of success, 50.8 percent succeeded. Eyster's prognosis was quite accurate; however, he concluded that the factors selected for research were actually indices of general scholastic ability rather than indices of shorthand aptitude. 40 attempted to relate aptitude for modern foreign language, Lang vocabulary, linguistic ability, and general scholastic ability to dictation and transcription. Her findings indicate that the best single predictor of dictation and transcription success was foreign language aptitude. However, this relationship was not sufficient to serve as a single predictor of success. Foreign language aptitude testing has been a source of ideas in the search for shorthand aptitude measures. Foreign language aptitude testing has also drawn upon work in the shorthand prognosis search. For example, one of the subtests of the Modern Languages Aptitude Test is a modification of one of the subtests of the Turse Shorthand Aptitude Test. 40Lang, Mary Jane, "The Relationship Between Certain Psychological Tests and Shorthand Achievement at Three Instructional Levels" (Doctor's thesis, University of Missouri, 1960). .. _ u.—..-..-A_.—..._. —__ .. s‘rpvqr- ‘ .a. 39 Veon's4] research attempted to determine the relationship of certain learning factors found in particular foreign language aptitude tests to predicting shorthand achievement at the college level. The tests used in this research were: (1) American Council on Education Psychological Examination for College Freshmen, 1944 edition; (2) Iowa Placement Examination, New Series, Revised, Foreign Language Prognosis Test, Form A; (3) Luria-Orleans Modern Language Prognosis Test; and (4) Carmichael's Shorthand Learning Test, Semester I (the shorthand criterion of achievement). A total of 299 beginning shorthand students at George Washington University were given the tests during the 1945-1948 academic years. Veon found the American Council on Education Psychological Examination for College Freshmen, 1944 edition, correlated .5102 with the shorthand criterion. The Iowa Placement Examinations, Foreign Language Aptitude Test, Form M, yielded a correlation of .6374 with the shorthand criterion test. Symond's Foreign Language Prognosis Test showed an even higher correlation with the shorthand criterion, .7192, than the Iowa Test. The multiple correlation, however, was found to be .5421 which seemed to indicate that the combination of tests used by Veon was not particularly effective in shorthand prognostication. 41Veon, Dorothy H., "The Relationship of Learning Factors Found in Certain Modern Foreign Language Aptitude Tests to the Prediction of Shorthand Achievement in College" (unpublished Doctor's dissertation, Oklahoma Agricultural and Mechanical College, 1950). 4O Varaho,’2 in considering the effect of academic motivation as measured by the Michigan M-Scales, on first-and second-semester shorthand students, found that academic motivation was a learning factor in first-semester shorthand but not in second-semester. He concluded that the best predictors for first-semester shorthand was grade-point average, ninth grade English grades, tenth grade English grades, and mental ability. Varah also found that the Michigan M-Scale when used in combination with a test of mental ability, would tend to increase the probability of accurate prediction by an estimate of mental ability in predicting the achievement of eleventh grade girls in beginning Gregg Shorthand. The Word Rating List was found to be a significant predictor of shorthand achievement for first- and second- semester students of Gregg Shorthand. He concluded that the academic self-concept of the student as measured by the Word Rating List is also a factor in learning first- and second-semesters of Gregg Shorthand. To the extent that the educational cognitive style of an individual might be related to performance in shorthand, the present study is considered to be related to these efforts. Moskovis43 attempted to identify similarities and differences 42Varah, L..J., "Effect of Academic Motivation and Other Selected Criteria on Achievement of First and Second Semester Shorthand Students" (Doctor's thesis, Michigan State University, 1966). 43Moskovis, L. M., "An Identification of Certain Similarities and Differences Between Successful and Unsuccessful College Level Beginning Shorthand Students and Transcription Students" (unpublished Doctor's thesis, Michigan State University, 1967). 41 between successful and unsuccessful college beginning shorthand students. He suggested college counselors and business educators consider high school English grades, give students a critical thinking measure (such as the Watson-Glaser Critical Thinking Appraisal), identify good study habits and attitudes (possibly using the Brown-Holtzman Survey of Study Habits and Attitudes), and consider a measure of clerical speed and comprehension (name-checking subtest of the Minnesota Clerical Test) when advising college-level students into beginning shorthand. Moskovis, in his study, utilized five standardized tests: (1) the California Psychological Inventory, (2) the Brown-Holtzman Survey of Study Habits and Attitudes, (3) the Minnesota Clerical Test, (4) Watson-Glaser Critical Thinking Appraisal, and (5) Wellesley Spelling Scale. He also used skill achievement tests, student questionnaires, and institution records. Students who received a grade of A or B were classified as successful. Students who received a grade of D or F were classified as unsuccessful. He found that the successful and unsuccessful college-level beginning shorthand students were significantly different at the levels indicated by mean score in: nun-l 0 college major (.01) college English composition grade (.001) name checking (.001) 2 3 4. study habits and attitudes (.001) 5 spelling ability (.001) 6 critical thinking (.001) 42 7. capacity for status (.02) 8. sense of well-being (.05) 9. responsibility (.001) 10. socialization (.02) 11. communality (.01) 12. self-control (.05) 13. achievement via independence (.05) 14. achievement via conformance (.001) 15. intellectual efficiency (.Ol) 16. psychological-mindedness (.02) 17. grade-point average (.001) 18. shorthand theory (.001) 19. shorthand brief forms (.001) 20. shorthand reading (.001) A number of significant differences were identified between successful and unsuccessful college-level beginning shorthand students as related to the variables employed in this study. In many ways, the present study is related to this effort. In addition to measures of theoretical symbolic ability, such as were employed in Moskovis' study, the present effort also considers qualitative symbolic elements, cultural determinants, and modalities of inference as possibly being connected with performance in shorthand. I It is recognized by some business educators and persons outside the field of business education that the study of shorthand requires more than average intellectual ability. While above average 43 intelligence has been found to be necessary, it is not sufficient, by itself, for a prognosis of success. A high IQ is no guarantee of success; other factors must also be examined as they relate to possible shorthand achievement. Osbornef4. in 1943, did a multiple factor study whereby 139 second-semester high school students were given a battery of selected psychological tests. The test battery included: (1) the Otis Self-Administering Test of Mental Ability, (2) the Iowa Silent Reading Test, (3) the Institution of Educational Research Clerical Ability Test, (4) the Revised Minnesota Paper Form Board Test, and (5) the Gates Visual Perception Test. The Carmichael Shorthand Learning Test was utilized as the standard shorthand achievement test. Correlations were computed for the thirty variables obtained from the five selected psychological tests with the shorthand achievement test and with one another. The correlation between the shorthand criterion and mental ability (.3765) did show some relation existed. However, Osborne reported that not one of the correlations was high enough to be utilized as a predictor for shorthand success or failure. She cautioned that high mental ability does not assume superior shorthand achievement nor low mental ability preclude high performance. Osborne believes that no single factor could adequately predict shorthand success. 44 Tests and Shorthand Achievement" (published Ph.D. dissertation, Columbia University, 1943). Osborne, Agnes E., "The Relationship Between Certain Psychological E ‘M' -4 E1 44 45 examined many of the factors Osborne did Henrickson and supported her findings. She reported low correlations to predicting shorthand success. Since these studies measured psychological elements, as well as theoretical symbolic achievement, they are considered to be related to the present effort. 46 analyzed the correlations between student scores on Coleman subtests of the Michigan State University Entrance Test Battery Fe and final performance in beginning shorthand when two methods of instruction were used. He found that the correlations were not high enough to possess predictive value of grades of potential 1 shorthand students. i Doubleday's47 investigation included a study of: silent reading abilities on printed and handwritten material, rapidity of motor response, mental ability, purpose for taking shorthand, personal reaction to shorthand, vocational interest, amount of time and interest given to school activities, and a composite of teacher's grades and shorthand grades. 45Henrickson, Rosanne C., "The Differential Aptitude Tests for Verbal Reasoning, Numerical Ability, Abstract Reasoning, Space Relations, Mechanical Reasoning, and Clerical Speed and Accuracy as Predictors of Success in Shorthand" (unpublished Master's thesis,) University of Minnesota, 1963. . 1 46Coleman, Brendan G., "The Effects of a Tape Laboratory L,_ Instructional Approach Upon Achievement in Beginning Shorthand gggzges" (gnpublished Doctor's thesis, Michigan State University, , p. . 47Doubleday, Lewis, "A Study of the Factors Affecting Achievement in Shorthand" (unpublished Master's thesis, State University of Iowa, 1939). 45 The results indicated no significant correlations in the factors studied. Doubleday further concluded that the student of high intelligence had a better chance of success in shorthand than a student of low intelligence, however, there was not a great enough difference to predict success in individual cases. Lynch48 supported Doubleday's conclusion that the relationship between a student's grade point average and success in shorthand is high enough to justify using them for shorthand prognosticating. Evans49 also investigated certain specific factors in relation to college shorthand achievement. The high school and college records of a total of 335 students who had completed one year of shorthand and had received grades of A, B, D, or E were studied. . She concluded the following: (1) amount of high school shorthand did not affect college shorthand performance, (2) success in college shorthand appeared to be related to high school English grades, and (3) rank in high school appeared to have a definite relationship to successful shorthand performance on the college level. Hale50 administered standardized tests to 78 high school students measuring reading ability, word sense, spelling, vocabulary, manual dexterity, spacial visualization, abstract thinking, symbol 48Lynch, Aline, "Factors Related to the Achievement of the One Hundred-four High School Seniors in a First Course of Shorthand" (unpublished Master's thesis, University of Michigan, 1947). 49Evans, Ernestine, "Factors Related to Varied Achievement in Shorthand on the College Level" (unpublished Master's thesis, State College of Washington, 1941). 50Hale, Jordan, "A Factor Analysis of Shorthand-Transcription Ability" (unpublished Ph.D. dissertation, New York University, 1958). 46 manipulation, perseverance, memory, phonetics, typewriting speed and accuracy, study habits, speed of writing, and name and number comparison. He conducted a factor analysis to identify those factors that affect the shorthand-transcription process. The independent, uncorrelated factors he studied were: verbal, perceptual, manual dexterity, abstract thinking, and personal. The New York State Regents examination for transcription and typewriting was used as the research criterion. Three factors were identified by Hale as significant as determined through his factor analysis: psychomotor speed, verbal, and non-verbal with a spacial visualization-mechanical ability. He also reported that 54 percent of the total variance of the Minnesota Clerical Number Comparison Test could be attributed to the psychomotor speed factor. SUMMARY The investigations of multiple factors which have been identified here generally agree that a combination of factors should be used for selection or prognostic purposes regarding the study of shorthand. The factors generally concluded to be of prime importance were: mental ability, English grades, overall grade-point average, and personal trait ratings. Motivation apparently plays a part, also, in shorthand success. It was generally agreed that success in shorthand was not dependent on a single factor--trait or characteristic--but upon a combination or variety of factors. 47 It appears to be evident that further research is needed to find better predictors of shorthand achievement. Investigations are needed to determine which factors or combination of factors are necessary in order to be successful in beginning shorthand. Once these factors have been identified, we must develop programs and materials to assist the student who does not have these "successful" factors. The present study is concerned with a number of factors included in an individual's cognitive style. The objective is to determine whether there is a "collective cognitive style" of successful beginning shorthand students and a "collective cognitive style" of the unsuccessful beginning shorthand students. Once these profiles have been determined, personalized educational programs of instruction could be developed to enhance those factors found to be lacking in the profiles of unsuccessful shorthand students. CHAPTER III METHODS AND PROCEDURES To determine whether there exists a different "Collective Cognitive Style" of the successful beginning shorthant student and of the unsuccessful beginning shorthand student, data per- taining to full-time beginning shorthand students were collected from three campuses of Oakland Community College during the l972-1973 fall and winter semesters. Three types of data were collected: (1) those yielded by the Oakland Community College Cognitive Style Test Battery, (2) those included in student records that were considered to be pertinent to the study effort, and (3) those resulting from informal discussion with faculty and selected students. GENERAL PROCEDURES Population The study was conducted at three campuses of Oakland Community College. The College was established by the electorate of Oakland County, Michigan, on June 8, 1964. The area served encompasses almost nine hundred square miles and contains twenty-eight public school districts with thirty-nine public high schools. In addition, there are twenty-two non-public high schools in the district. 48 49 The college Opened in September, 1965, with a record community college initial enrollment of 3,860 students on two campuses. By Fall, 1971, Oakland Community College, with twenty-one extension centers and three campuses, served over 15,000 students. In 1971, the college received full accreditation from the North Central Association. Goddard said that "More than any other educational institution, the community college is responsive to the various social, economic, - technological, and political forces in the community. These forces have a very significant effect on the changes in curricular offerings "51 of the community college, Poland contended that "in the future, the curriculum will be determined by the institutional philoSOphy and i goals which have been based upon the needs of students, community, "52 Goddard listed five functions of a community and the society. college. Oakland, as a multi-campus community college, has developed a wide spectrum of courses and programs to meet the educational needs of its district and recognizes seven major objectives or functions of their college. The 1973-1974 catalogue lists the following: 1. Courses in the academic disciplines that provide a solid basis in liberal arts and sciences for students desiring to transfer to four-year institutions. 5lGoddard, M. L., "The Potential Role of the Junior College in Education for Business" (unpublished Doctor's thesis, Indiana University, 1962). 52Poland, Robert P., "Implications of Certain Social, Economic and Technical Trends on Business Curricula in the Public Community Colleges of Michigan" (unpublished Doctor's thesis, Michigan State University, 1962). 50 2. Courses in the Applied and derivative fields such as nursing, pharmacy, and engineering and career education programs for those desiring immediate employment opportunities upon completion of one or two years. Many of the courses in the career program may be transferred to four-year institutions. 3. General education experiences which facilitate the develop- ment of a broadly educated person--one who has a coherent sense of systems of knowledge and is able to think clearly, communicate effectively, make relevant judgements, distinguish among values and make appropriate applications of knowledge. To provide a framework for general education, Oakland Community College developed the Foundational Studies curriculum. 5* 4. Counseling and guidance programs that offer assistance in self-evaluation and professional Counseling in areas of admissions, education, career, student activities, financial aid, and community guidance. 5. A diversified program of comunity services, in addition h to the regularly scheduled day and evening classes, designed to meet the educational, cultural, and recreational needs of all members of the Community College district. 6. Developmental programs to assist students with high potential who require strengthening in basic areas prior to undertaking advanced education. 7. Research, development and evaluation acgivities relative to the improvement of teaching techniques. The additional functions or objectives listed by Oakland but not by Goddard, are numbers six and seven above. These objectives have been developed in the nine years since Oakland has been established. 1 There are a variety of instructional approaches at Oakland. Performance goals are clearly stated and media used for instruction include filmstrips, slides, audio and video tapes, and printed text materials. The following statement in regard to instructional approach appears in the 1973-1974 catalogue: 530ak1and Community College Catalogue 1973-1974 (Bloomfield Hills, Michigan: Oakland Community College, 1973), pp. 3-4. 51 "Oakland Community College has developed an individualized approach to learning based,on the premise that a student of normal ability will nut fail if the content of the course is presented in a fashion appropriate for his cognitive style. Currently, about 35 percent of the courses are designed to provide this personalized type of instruction. The keystone of this approach is a diagnostic testing program that measures a student's ability to acduire meaning through qualitative strengths as well as the more traditional theoretical methods. The * scores achieved on written diagnostic tests and on demonstrable performances, along with supportive data from personal interviews, are translated into elements of the student's cognitive map. The student's map becomes the basis for identifying the educational sequences, teaching media, and ' instructional techniques which will assist him in the mastery of essential skills. 'While each student remains responsible for the content of his course, he may "burst" into any of several instructional modes appropriate to his needs, abilities, and cognitive style. The environments for the "burst" activities are the Carrel Arcades, the Learning Resources Center (LRC) and the Individualized Programmed Learning Laboratory (IPLL),"54 At the present time, 1973, Oakland Community College consists of four campuses and two additional facilities in Royal Oak and Madison Heights yet to be developed. The Central Administration building of this multi-campus college is located on Opdyke Road in Bloomfield Hills, Michigan. Three campuses of Oakland Community College offering beginning Gregg shorthand classes for at least two years were included in this study. The campuses were: (1) the Auburn Hills campus located east of Pontiac near M-59 and I-75 Expressways (the main thrust of this facility is vocational/technical education); (2) the Highland Lakes Campus located in Union Lake, West of Pontiac 54Catalogue, p. 5 52 (the main emphasis in this, the smallest campus included in the study, is placed on the health related programs); and (3) Orchard Ridge, the largest campus, located off I-696 in Farmington, has the largest secretarial enrollment; because of its location, however, it places primary emphasis on Liberal Arts and general education programs. All three campuses included in this study offered at least one section of beginning shorthand (Gregg Shorthand Diamond Jubilee Edition), each semester of the study. A semester is defined as fourteen weeks of instruction plus one week of final examinations. Beginning shorthand standards within the college are defined as: three minute takes at 80 words per minute for a grade of A, at 70 words per minute for a grade of B, at 60 words per minute for a grade of C, and at 50 words per minute for a grade of D on new material with a 10 percent preview and a required accuracy rate of 90 percent. Performance below this level would earn an F grade.' The grade of I can be used in most cases where the student evidences sufficient ability, but might need more time to achieve a successful performance level. A mggk_of "X" may be assigned with or without student consultation if a student does not attend the class and section for which he is registered. This EECE.15 interpreted to mean "no credit." If a student wishes to receive credit for a course in which he has received an "X" he must re-register for the course. A mark of "W" is associated with student-initiated withdrawals. "'—‘1-‘- m- - ‘L‘vr P I 53 Five faculty members taught beginning shorthand. They were all vocationally approved and the range of teaching experience is seven to twenty years. Table 3.1 shows the enrollment of full-time beginning shorthand students, for which cognitive style maps were available, by campus: TABLE 3.1 TOTAL ENROLLMENT 0F FULL-TIME BEGINNING SHORTHAND STUDENTS BY CAMPUS Campus Number Percent of Total Auburn Hills 32 21.2 Highland Lakes 27 17.9 Orchard Ridge _22_ 60.9 Totals 151 100.0 Table 3.2 presents data regarding the distribution of grades and marks earned by the beginning shorthand students for which cognitive style maps were available. 54 NKN Ne 06 m 3 N N m min m may ¢N_m.: m_. N.FN Nm o.oop Fmp 4m mpzmoahm azx< 0000000004000 AMAAAWMA n n E U, v V x /\ l l C 31 ”j A L__ ___ R' (K) 66 The entire Cognitive Style Battery was administered to each of the students included in the study. In an effort to further clarify how the elements are determined, the "cut-off" scores for the different parts of the test battery areas are as follows: Cugnitive Style Test Cut-Off Scores Test Symbol Major Minor 1 T (VL) 30-50 20-25 2 T AL 7- 8 2- 6 3 T (V0; 24-40 18-23 4 T VL 36 different reading levels from 2.0 to 12.3 5 T (A0; 8-10 5- 7 6 T VL 35-50 23-34 7-8 0 Codes 15-24 8-14 9 I,A,F 21-30 14-20 10 M,D,R,L 37-70 22-36 Circle K - Five questions on Inventory 10 are used to determine this deductive inferential pattern. If three out of five inventory items are answered "usually," and the student has a major or minor orientation in T (VQ), the Circle K is included in his or her "map.“ English Placement Based on Test 6 Scores Raw Scores Recommended English Course 44-50 English 152 35-43 English 151 Minor T (VL) English 131 Negligible English 052 Methods. Empirical mapping was employed to interpret the individual maps of the students. Empirical mappings involve persons making judgments on a "makes sense" or "does not make sense" basis to 67 classify elements (e.g., persons, processes, and properties of a social system called "curriculum") into "logical" categories. This type of decision-making employs not only theoretical symbolic mediations (e.g., "words" and "numbers“) but demands qualitative symbolic aspects of reasoning (e.g., "picturing" the solution of a problem) as well. For example, in order to diagnose (map) the cognitive style ("9") of an individual, the "educational scientist" (e.g., teacher) must consider the individual's level of educational development in terms of the elements included in the Sets 1, II, and III, or S E, and H, respectively. Empirically, the diagnostician ("scientist") must decide which elements in the sets S, E, and H, respectively, are appropriate for inclusion in the sub-sets s, e, and h that form the individual's cognitive style. This approach demands the classification of the elements of S into two categories: 5 and s (not 5); E into the two categories: e and e (not e); and H into h and h (not h). The process involved here is called "empirical mapping." This type of mapping (diagnosing) can only be effected by a person (e.g., educational scientist) classifying the elements involved on a "makes sense," or "does not make sense" basis. If mathematical mapping of these elements were desired, there would need to be a logical and theoretical vehicle (e.g., a formula, or equation) available by which the decision could be derived. In this case, a person is not actually involved in the decision-making process associated with the classification. Under these circumstances, the decision is rendered by performing indicated operations on the logical (mathematical) function used to determine the desired outcome. 68 Mapping of cognitive styles is mainly empirical in nature, and as such, is dependent upon the judgments of persons (diagnosticians) ‘responsible for the process of classifying the elements of symbolic 'orientations, cultural determinants, and modalities of inference into a Cartesian product of three sets that represent the cognitive styles of the individuals under consideration. 57 technique suggests that the lowest 27 percentile Flanagan's of a frequency distribution of scores; measurements; or other quantifications; and the highest 27 percentiles of a distribution can be used to identify the "low" or "unsuccessful" group and the "high" or "successful" group of a given distribution, respectively. A slight modification of Flanagan's technique was employed in this study to determine the "collective cognitive style" of the "highly successful" and "unsuccessful" beginning shorthand student groups, respectively. For example, if the element of theoretical visual linguistic, T(VL), appeared as a major orientation (fiftieth through the ninety-ninth percentile) in 70 percent of the individual's cognitive maps of the group ("highly successful" or "unsuccessfu1") of students under consideration, the element would be included as a major in the "collective cognitive style" of that group. Once the collective cognitive style of the high and low or "highly successful" and "unsuccessful" students were determined, visual inspection of the maps was employed to determine differences. 57F1anagah, pp. 684-680 69 Informal discussions with students regarding attitude toward the subject matter and reasons for enrolling in the course were held. Discussions were limited to students from Orchard Ridge since location and access to students on other campuses was not possible. Further discussions with faculty teaching beginning shorthand at Orchard Ridge campus regarding the faculty member's impression of a student and how the faculty member's impression of the student agreed with the student's cognitive style map, were held. The discussions with selected students and teachers provided valuable supplemental information to this investigator in regard to those elements which may have appeared to be ambiguous. Hypotheses As cited in Chapter I, the three primary hypotheses tested were as follows: H1: There is a collective cognitive style which is characteristic of the group of "highly successful" beginning shorthand students. H2: There is a collective cognitive style which is characteristic of the group of "unsuccessful" beginning shorthand students. H3: There is a difference between the collective cognitive style of the "highly successful" and the "unsuccessful" beginning shorthand student groups. Procedures of Data Collection The following procedures were employed to collect the data used in the present study: 70 l. A request was made of the Data Processing Department of Oakland Community College to provide a printout of available cognitive style maps for students taking Shorthand Fundamentals in the l972-l973 fall and winter semesters at Auburn Hills, Highland Lakes, and Orchard Ridge campuses of Oakland Community College. 2. With this information, the researcher reviewed the maps in terms of grades or teacher evaluations. 3. The researcher then determined the upper quartile ("highly successful" students) and lower quartile ("unsuccessful" students). 4. The researcher analyzed the cognitive style maps of the identified sample to determine common elements which exist in the students' maps identified as "highly successful." The same was done with students' maps identified as "unsuccessful." 5. The cognitive style maps were then analyzed in terms of elements contained in the maps of "highly successful" students that were not in the maps of "unsuccessful" students. 6. Discussions were held with teachers regarding their impressions of selected students to determine the amount of agreement of the teacher's impression of the student and the student's map. 7. Informal discussions were held with selected students at Orchard Ridge regarding their attitude toward the subject matter and reasons for enrolling in the course. SUMMARY Students making up the sample in this study were the upper and lower quartiles of beginning shorthand classes at three campuses of Oakland Community College. The classes were taught in the fall and 71 winter semesters of l972-1973. The classes were composed of women who had no previous instruction in shorthand. The textbook used in the course was Greggfs Individual Progress Method. The Oakland Community College Cognitive Style Test Battery was administered to each student in the study upon entering the college. Maps of each student included in the sample were analyzed in terms of common elements for inclusion in a "collective cognitive style." The purpose of this chapter has been to describe the design of the study in terms of the source of data, the sample employed in the study, its selection, adequacy, and the method and procedures of data collection. In the next chapter, the analytical techniques employed in the study, analysis of data, and the findings of the study will be presented. The fifth and final chapter will present the summary, conclusions, implications, and recommendations emanating from the study. CHAPTER IV ANALYSIS OF DATA AND FINDINGS The background of the present study was provided in Chapter I. In Chapter II, the literature related to the present study was reviewed, and in Chapter III, the methods and procedures were presented. The purpose of Chapter IV is to analyze the data and present the findings. An analysis of "highly successful" and "unsuccessful" beginning shorthand students at Oakland Community College is presented in regard to the 27 selected variables in the cognitive style battery. The categories highly successful and unsuccessful are dependent on the final grade received in the beginning shorthand classes. Students who received a grade of A were classified as highly successful; students who received a grade of D or F, or a murk of X were classi- fied as unsuccessful. ANALYSIS OF DATA The process of empirical mapping can be described and analyzed by means of a miniature example including an educational scientist (e.g. teacher), diagnosing (mapping), selected elements (variables) of the cognitive style of an individual or individuals on the basis of test scores, inventory scores, observed behaviors, reported be- haviors, and other information pertaining to the individual or individuals whose "style“ is being mapped. For example, a miniature 72 73 sample of the first "style” (i.e. incomplete collective style) shown is that of the collective style of the "highly successful" student group: FIGURE 4.1 HIGHLY SUCCESSFUL STUDENT (Sample Map) [(11 lm |:1: .2— ._... T'(AL) T'(AQ) T'(VL) T'(VQ) I M D g=< Q'(V) /\ x < F") XSR' 1255 Q (CET) L__ ___. 1_w___. .L__ RDLV 7 or above In the S set, the highly successful student possesses a minor orien- tation in the four theoretical symbols of visual linguistics, auditory linguistics, visual quantitative and auditory quantitative. The student also possesses a minor orientation in qualitative visual and a major orientation in qualitative code ethic. Set E of the highly successful student profile contains a major orientation to "individuality" with a minor orientation in the "Family" and "Associates" cultural determinants. In Set H, the highly successful student possesses a major orientation in the "Magnitude" and the "Difference" mode of inference, respectively, each of which is combined with a minor orientation in the "Relationship" mode of inference. 74 The collective style also indicates that at least seventy percent of the highly successful students have an educational deve10pment of twelfth grade, and a reading level of seventh grade or above (RDLV 7 or above). The second "miniature" sample is that of the collective style of the unsuccessful student group: FIGURE 4.2 UNSUCCESSFUL STUDENT (Sample Map) IV) [111 |:: 12.. __ ._ __ T'(VL) T'(AL) T'(AQ) I' 9=< Q'(CET) x < A' ? x& L 5 9 Fl (L... ___. ______i RDLV 5 or above The unsuccessful student group's miniature collective cognitive style sample above possesses a minor orientation in theoretical visual linguistic, theoretical auditory linguistic, and the0retical auditory quantitative. The "style" also contains a minor in qualitative code ethic in S, the first set of the sample. The second set, E, shows the collective style of the unsuccessful student possessing a minor orientation in "individuality," "Associates,“ and "Family" cultural influences. A distinction exists between a 75 collective cognitive style of a group and that of an individual student's map. An individual student's map must include a major orientation in one, but no more than two, of the cultural determinants. Set H, the third set of the unsuccessful student group's collective style sample shows an "appraisal“ modality of inference. The educational deve10pment of the unsuccessful student style is tenth grade or above with a reading level of fifth grade or above (RDLV 5 or above). With reference to the highly successful student group's collective style, elements in which the "styles" differ, in this limited example, would be: FIGURE 4.3 DIFFERENCES 0F HIGHLY SUCCESSFUL STUDENTS (Sample Map) S E B. 12 __ 1.. _ T'(VQ) M D g=’ x R' R'> ' CH) '(CK) ' CKH) A' CP) (CS) (CT) /\¥ I 000000000 RDLV 7 or above The "style above illustrates the strengths of the highly successful 58Appendix B, Frequency Count of Scores, Oakland Community College Cognitive Style Test Battery. 78 beginning shorthand student group. The level of educational deve10p- ment is twelfth grade, the maximum level tested by the Oakland Community College battery. The reading level (RDLV), is seventh grade or above. In set S, the highly successful student group's style indicates: A minor orientation (twenty-sixth to forty-ninth percentile) in the four theoretical symbols. i.e., auditory linguistic, visual linguistic, auditory quantitative, and visual quantitative. A major orientation (fiftieth to one-hundredth percentile) in each of the qualitative symbolic codes of empathetic, ethics. proxemics, and synnoetics abilities. A minor orientation in each of the qualitative codes of visual, esthetics. histrionics. kinesics, kinesthetics, and transactional. In the second set, E, the "strengths" of the highly successful student group are exhibited with a major orientation in the "Individuality" cultural determinant combined with minor orientations in "Family" and "Associates," in that order of strength. "Magnitude" and "Difference" are the two major orientations of the highly successful student group in the third set, H, i.e., modalities of inference, each of which is combined with a minor orientation in the third set of "Relationship." The reading level indicated in the style is seventh grade or above. It should also be noted that the qualitative symbols of auditory, olfactory, savory, tactile and proprioceptiveness were negligible in the collective cognitive style of the highly successful shorthand student group. 79 The collective cognitive style of the unsuccessful beginning student group at Oakland Community College is illustrated in Table 4.2 below: TABLE 4.2 COLLECTIVE COGNITIVE STYLE MAP OF THE UNSUCCESSFUL BEGINNING SHORTHAND STUDENT GROUP §. 5. II ._a lo _. _, .1 AL) T'(AQ) T'(VL) L— T— CEM) I' (D II A 000000000 -l MAMAAAA A O O n n n n O m '0 § X I m m V vv—lm V “V X > V X A. l— RDVL 5 or above The strengths exhibited in set 5, symbols and their meanings, in the collective "style" above are: A minor orientation in each of the theoretical symbols of auditory linguistics, auditory quantitative, and visual linguistics. A major orientation in each of the qualitative codes of empathetic proxemics, and synnoetics abilities. A minor orientation in each of the qualitative codes of esthetics, ethics, histrionics, kinesics, kinesthetics, and transactional abilities. \ 80 In the second Set, E, none of the elements show a major orientation in the collective "style." The cultural determinant "individuality" had the highest percent of students in the minor orientation. The "Associates" cultural determinant ranked next in the number of students exhibiting a minor orientation in that element. The cultural determinant "Family" was the least prevalent of the three elements. The "Appraisal" modality of inference is the major orientation in the third Set, H, of the collective "style" of the unsuccessful shorthand student group. The "Appraisal" pattern involves the use of "Magnitude," "Difference," and "Relationship." with equal weight in coming to some meaning or conclusion. The level of educational deve10pment in the collective "style" above is tenth grade, and the reading level is at least fifth grade. The elements that were negligible in the collective "style" of the unsuccessful student group were qualitative code auditory, olfactory, savory. tactile, proprioceptiveness, and visual, respectively. The elements held in common by the highly successful and the unsuccessful student Minor Minor Minor Major Minor Minor Minor Minor Minor Major Major group "styles" were as follows: theoretical theoretical theoretical qualitative qualitative qualitative qualitative qualitative qualitative qualitative qualitative auditory linguistic auditory quantitative visual linguistic code code code code code code code code empathetic esthetic histrionic kinesics kinesthetics transactional synnoetics proxemics. 8] SET E F' Minor family orientation A' Minor associates orientation There are differences between the collective cognitive style of the highly successful beginning shorthand student group and that of the unsuccessful beginning shorthand student group. l. The cognitive style maps of each of thirty of the thirty-two highly successful students showed a twelfth grade level of educational development. 2. The "maps" of each of sixteen of the twenty-nine unsuccessful students showed a grade level of educational development below the twelfth grade, with the range from third grade to eleventh grade. This finding tends to agree with the literature reviewed in Chapter II which indicates that verbal, grade point average and mental ability are factors to be considered in regard to successful performance in shorthand. 3. The theoretical symbols, set S, display a difference between the highly successful and unsuccessful student groups in the elements of theoretical visual quantitative. It should be noted that the theoretical visual quantitative represents the ability to acquire meaning in terms of functional relations which express conditions of. equality or inequality. The successful beginning shorthand student group has a minor orientation in T'(VQ) in its collective "style" while the unsuccessful student group "style" does not display this element. 4. The highly successful student group's "style" exhibited a minor orientation in qualitative visual. That is, the ability to perceive meaning through pictures and observations of actual settings. 82 The “style“ of the unsuccessful student group did not exhibit this element. 5. Qualitative code ethic, dedication to obligations or duties, is exhibited as a mgjgr_orientation in the highly successful student group's "style," and a mjggr_orientation in the "style" of the unsuccessful student group. 6. "Individuality" was found to be the major orientation of the highly successful student group's "style" in set E, with a minor orientation in "Family" and "Associates" cultural determinants of the meanings of symbols. The unsuccessful student group "style" does not exhibit a major orientation in any of the three cultural determinants. The "style," however, does display a minor orientation in each of the elements, i.e., "Individuality," "Associates," and "Family," in that order. 7. The highly successful student group's collective style and the unsuccessful student group's "style" differ in the third set, H, in the following way: The "highly successful" group's style displays a major orientation in "magnitude" (categorical thinking that utilizes norms or classifications as a basis for making decisions), and in "Difference" (utilizes contrasts or comparisons of selected character- istics to understand), each of which is combined with a minor in "Rela- tionship." The unsuccessful student group's "style" exhibits a major in "Appraisal," which involves the use of all three of the above (M, D, and R) with equal weight. 83 8. The elemements held in common by the highly successful and unsuccessful students in Set S, symbols and their meanings, were: T'(AL) Minor theoretical auditory linguistic T' AQ Minor theoretical auditory quantitative T'(VL Minor theoretical visual linguistic Q (CEM) Major qualitative code empathetic Q CS) Major qualitative code synnoetics Q CP) Major qualitative code proxemics Q'(CES) Minor qualitative code esthetics Q'(CH) Minor qualitative code histrionics 0' CK) Minor wualitative code kinesics Q' CKH) Minor qualitative code kinesthetics Q'(CT) Minor qualitative code transactional 9 The elements held in common by the highly successful and unsuccessful students in Set E, cultural determinants for the meanings of symbols. were: F' Minor family A' Minor associates l0. Those elements that were negligible in the collective styles of both the highly successful and the unsuccessful students were: Q (A) qualitative auditory Q (0) qualitative olfactory Q 5; qualitative savory Q T qualitative tactile Q (P) qualitative proprioceptive ll. Interviews with highly successful students who did not exhibit characteristics or traits common to highly successful students indicated these students were highly motivated toward achievement in the beginning shorthand class. l2. Interviews with selected unsuccessful students whose maps contained elements uncommon to the unsuccessful group, elicited the following responses: 84 Parental pressures caused them to enroll in the beginning shorthand class. Student did not wish to work in an office. Student preferred another career choice. l3. Discussions with faculty members concerning their impressions of the above students indicated the following: Student did not appear to be interested in the subject matter. Student did not appear to be sufficiently motivated toward achievement in the subject. Student had a poor attitude. The study was originally designed to determine a collective cognitive style of the highly successful student group and of the unsuccessful student group for each campus of Oakland Community College. That design called for a comparison of the elements in these campus "styles“ in terms of similarities and differences. A total College collective cognitive style would then be determined for each of the two groups. When all the "maps" were collected and the sample defined (upper quartile and lower quartile), it was found that the sample from each campus was not adequate in size to develop a valid collective cognitive style for each group by campus. A collective cognitive style for the highly successful and the unsuccessful beginning shorthand student groups for the College was then determined utilizing all members included in the sample. The sample represented the pro- portional number of students enrolled in the beginning shorthand course of each campus found in the defined population. CHAPTER V SUMMARY. CONCLUSIONS, IMPLICATIONS, AND RECOMMENDATIONS SUMMARY A knowledge of the characteristics and traits possessed by highly successful and unsuccessful beginning shorthand students will assist educators in develOping programs of instruction to assist students. A variety of instructional programs are needed to meet the different educational cognitive styles of students. This study was undertaken in an effort to determine the characteristics and traits that the highly successful and the unsuccessful student possesses and their differences. Need for the Study A survey of the literature revealed that well over 200 studies have been conducted in the area of shorthand pr09nosis. Many of these studies were prompted by the high failure and dropout rates in beginning shorthand. The literature further revealed that success in shorthand was not dependent on a single factor but upon a combination or variety of factors. There is a need to identify those characteristics and traits which are indicative of the highly successful and unsuccessful beginning shorthand student. Purpose of the Study There were three primary purposes for this study. They were: 1. To provide information about the types of students who might have a high probability of achieving successfully in shorthand, 85 86 2. To provide students with information about themselves, 3. To provide information regarding the methods and instructional strategies that may need to be employed to accommodate the different "styles" of students. The educational cognitive style maps of both the upper quartile and the lower quartile students were obtained and collective cognitive styles were ascertained employing a modification of Flanagan's technique. Visual inspection was employed to determine differences between the successful and unsuccessful student groups. Hypotheses Tested The hypotheses which this study tested are restated as an introduction to the procedures used. H]. There is a collective cognitive style which is characteristic of the "highly successful" beginning shorthand student group. H2. There is a collective cognitive style which is characteristic of the "unsuccessful" beginning shorthand student group. H3. There is a difference between the collective cognitive styles of the "highly successful" and the "unsuccessful" beginning shorthand student groups. Procedures Used A request was made for the available cognitive style maps of the students enrolled in beginning shorthand at Oakland Community College in the fall and winter semesters of 1972-l973. The maps were reviewed 87 in terms of teacher evaluations and the sample, upper and lower quartiles, was established. The individual cognitive style maps were analyzed to determine the common elements existing in the students' maps identified as "highly successful" and "unsuccessful." A modification of Flanagan's technique of "estimating the product-moment from the data at the tails of the distribution" was employed to determine those elements which would be included in the collective cognitive style of the highly successful and the unsuccessful student. The process of visual inspection was employed to determine the differences and similarities which existed between the high and low or highly successful and unsuccessful. Consideration was given to faculty and student comments made during the informal discussions which took place with faculty and students regarding attitudes toward the subject matter and reasons for enrolling in the course. The teacher's impression of a student's characteristics and traits were noted in terms of its agreement with the student's map. Findings The cognitive style maps of 6l students (32 highly successful and 29 unsuccessful) were included in this study effort. These students represented the upper and lower quartiles according to teacher evaluations of the students taking beginning shorthand in fall and winter of l972-l973 at three campuses of Oakland Community College. 88 An analysis of the maps and a collection of the elements held in common determined the collective cognitive style for the highly successful and the unsuccessful student. It was possible, therefore, to accept the first two hypotheses and conclude there was a collective cognitive style that was characteristic of the highly successful student group; and there was a collective cognitive style that was characteristic of the unsuccessful student group. Employing visual inspection of the collective "styles," differences between the highly successful student group style and the unsuccessful group style were ascertained. The third hypotheses could then be accepted. Similarities between the two groups were also determined by visual inspection of the collective cognitive "styles." The number of elements that were common to both groups was greater than the number of elements that were different. Informal discussions with selected highly successful and unsuccessful students revealed that motivation was an important part of achievement in beginning shorthand. Students who were not successful expressed a lack of interest and desire to achieve in beginning shorthand. CONCLUSIONS The conclusions of the study are presented in the form of either an affirmation or a negation of the respective hypotheses stated in Chapter I. 89 The first hypothesis stated: "There is a collective cognitive style which is characteristic of the "highly successful" beginning shorthand student group." This hypothesis was accepted. The collective style of this group appears as follows: TABLE 5.] COLLECTIVE COGNITIVE STYLE OF HIGHLY SUCCESSFUL BEGINNING SHORTHAND STUDENT GROUP 5 E II 12 T'(AL) T'(VL) T'(Ao) T'(voT— CET) J> x( M o 9 =4 F' > X <: ‘>. RDLV 7 or above The second hypothesis stated: "There is a collective cognitive style which is characteristic of the "unsuccessful" beginning shorthand student group." This hypothesis was accepted. The collective style of this group is illustrated below: 90 TABLE 5.2 COLLECTIVE COGNITIVE STYLE OF UNSUCCESSFUL BEGINNING SHORTHAND STUDENT GROUP .5. E II 10 ____ ___, __. "1' T'(AL) T'(AQ) T'(VL) Q (CEM) 12:55: 1- 3‘2EI >' x 4i A' >' x ‘< L :> ' CKH) CP u ' CT RDVL 5 or above “a I 00000000 The third hypothesis stated: "There is a difference between the collective cognitive styles of the ”highly successful" and the "unsuccessful“ beginning shorthand student groups." This hypothesis was accepted. The differences are presented in Table 5.3 below: TABLE 5.3 DIFFERENCES, REFERANT HIGHLY SUCCESSFUL STUDENT GROUP IV: 5. fl RDVL 7 or above 91 TABLE 5.4 DIFFERENCES, REFERENT UNSUCCESSFUL STUDENT GROUP |m Im II 10 ___. -—T .__ ,L_-—- 9 =4Q'(CET) F X i1} x12? RDVL S—or above The differences with respect to the highly successful student group's style exhibits a theoretical visual quantitative element in the first set,.S. This element indicates the ability to acquire meaning in terms of numerical symbols, and in terms of functional relations which express conditions of equality or inequality. This element appears to be a significant one in the process of learning shorthand outlines. For example, the shorthand outlines for s, f, and v are I, ) respectively. The shbrthand outlines for a and e are 0 0. When letters are combined into words (e.g., vase9) ) it appears to be necessary to have the ability to recognize proportional differences and combinations of shorthand letters into words. This ability to see and interpret is strengthened by the qualitative visual, Q'(V) that the highly successful group's "style" also exhibits, different from the unsuccessful group's "style." The theoretical visual quantitative and qualitative visual elements do not exist in the collective style of the unsuccessful student group in either a major or a minor orientation. 92 Qualitative code ethics (dedication to obligations or duties), appears as a major orientation in the first set, S, of the collective "style“ of the highly successful student group. Beginning shorthand is a skill requiring organized practice on a regular basis. The highly successful student appears to have this commitment. The un- successful student group's collective "style" does not exhibit a major orientation in this element. It does include, however, a minor orientation in this element, i.e., qualitative code ethics. This finding would imply the unsuccessful student group might not "try so hard" to achieve successfully in shorthand performance as the "highly successful" group does. In the second set, E, cultural determinants for the meaning of symbols, "individuality" is the major orientation for the highly successful student group, with a minor in "Associates" and "Family." The program of instruction utilized in the study was Gregg's Individual Progress Method. This appears to indicate that students with a major in "Individuality" might prefer to learn or study on an independent basis. With a major in both "Individuality" and qualitative code ethics, it would appear that with the program utilized, the student I n- -.~.__ I . 1 might have a greater probability for successful performance that would those showing "minors" or "negligible" orientations in these elements. a." The unsuccessful student exhibited a minor in "Individuality," "Associates," and in "Family," in that order. During the interviews with members of the unsuccessful student group whose "styles" included 93 only some of the elements which might have indicated highly successful achievement in shorthand performance, the students indicated they were “taking shorthand because their parents insisted that they must have a marketable skill." It might be concluded that a student with a "minor" I (Individuality) might well acquiesce and take shorthand because of parental pressures. However, with a minor orientation in qualitative code ethics (sign of relatively little commitment) and the existence of a “minor" in the “Family" cultural determinant, the student might not perform successfully in shorthand. In the third set, H, the successful student group's "style" exhibits a major orientation in "Magnitude" (categorical thinking utilizing I classifications) and in "Difference" (utilizes contrasts in the thinking process), combined with a "minor" in "Relationship" (ability to synthesize). The "style" indicates the ability to employ the use of rules, to see contrasts (proportions), and then put them together (synthesize). The "style" of the unsuccessful student group exhibits the "Appraisal" modality of inference. This mode of inference involves all three modes (M, D, and R) with equal weight. This particular type of person tends to be slow in the process of drawing a conclusion. In relation to the need for developing speed in beginning shorthand, the "Appraisal" mode appears to indicate difficulty for the individual showing that element in his or her style. —\"i"*--- ”n l-umafi-n._-- .. 94 IMPLICATIONS There are a number of implications which follow from the conclusions of the study. l. Certain elements, visual and individuality, of cognitive style seem to predispose one to do well in shorthand. Possession of these elements by a particular student would tend to increase the probability of success in shorthand. 2. Students having difficulty in shorthand could possibly improve their chances of succeeding in shorthand by augmenting their cognitive style to include more of those elements associated with students who are successful in shorthand. 3. It appears that one could establish a relationship between certain educational cognitive style elements and various abilities and success in shorthand which could assist in the identification and encouragement of future shorthand students, and which might enable educators and guidance counselors to interpret more realistically a student's relative success in shorthand than is currently possible. 4. The construct of educational cognitive style as used in the present study appears to be valuable in that it allows one to look at educational phenomena from frames of reference which have not been used before. This allows one to analyze educational phenomena from points of view which differ from any which have been previously used. 5. The cognitive style construct used in the study appears to be useful as a conceptual framework that can aid in explaining phenomena in business education. Its usefulness in the study in terms of success in shorthand offers great hope of its usefulness in other areas of study. 95 6. The results of the study point out the importance of cognitive style in the educational process. It may well be that: (l) greater communication occurs when a student having a particular cognitive style works with a teacher having a similar cognitive style, (2) a student relates better to instructional materials that match his cognitive style, (3) the educative process should provide a means for mapping the cognitive style of students to determine goals which are consonant y with the student's cognitive style and to identify aspects of the student's cognitive style that might be augmented and/or modified. Educational experiences should be provided to develop certain kinds of processes and cognitive skills in order to develop elements through , instruction and practice. Instruction can probably be designed to develop certain cognitive style elements. Cognitive styles probably are developed at an early age and guide students through many situations. Ways of accommodating the various styles should be provided for in the educational process. Consequently, it seems necessary to remain flexible in the classroom. The same method may not always be optimal as the content changes, and it will probably not be optional for all students. 3 By presenting and seeking information in various ways, and perhaps several j ways. one can increase the range of students of varying style who "get the point." At the same time, the necessary variety of presentation will keep the teacher from settling into a cognitive style presentation which may be unlike that of most of his students. Information about the dif- ferent cognitive styles of students may help a teacher to be more 96 effective with his individual students and may enable the teacher to lead a student to more significant educational and thinking experiences than is currently possible. 7. Cognitive style will probably affect the instruction that results from educational treatment and probably can enter into the evaluation of educational procedures either by being a consequence of a procedure or as a factor in its affectiveness. 8. It would be desirable to have knowledge of the development and background producing a particular cognitive style in order to better understand the cognitive characteristics of an individual student than is currently possible. 9. Differences among students in sty1g§_of perceiving and conceptualizing are probably as real as are differences in general intellectual ability and educational achievements. Knowing a student's cognitive style should become as much a part of a school's measurement program as the 1.0. test. All of the foregoing implications consider various possibilities for use of cognitive style that hold considerable promise for research . .,. ”$7.... deve10pment, and implementation in a wider spectrum of educational fir: '-' -. .9- A settings. RECOMMENDATIONS Among the recommendations that arise from the results of the present study are the following: 97 l. Add the fourth set, memory-concern functions, to the cognitive style map, in addition to the symbolic, the cultural determinants, and the inferential processes. The fourth set is concerned with the influence and function of memory-concern as a factor of cognitive behavior. This fourth set is presently being develOped. 2. The instrumentalities used in the study measure observed behavior, not necessarily that which motivated that behavior. Attempts should be made to assess certain pertinent psychological aspects of behavior, and to incorporate the findings into the procedures used to determine cOgnitive styles. If one can get at the psychological motivation behind observed behavior, then one can determine whether or not that behavior is a true manifestation of the state of affairs that gave rise to the behavior. In assessing cultural determinants, one might observe behavior that is indicative of I, the major individuality determinant. However, the observed behavior may be motivated by a desire on the part of the subject to compensate for a weakness in this regard, rather than being a true manifestation of a major individuality determinant. “ 3. Attempt to measure negative cultural determinants. The J'D-‘L A‘— existing procedures for assessing the family, associates, and individ- uality determinants do not measure the negative aspects of these cultural determinants. For example, does a subject who exhibits a major individuality determinant (50 to 99 percentile) automatically have a negative associates determinant. 98 4. Attempt to arrive at a more refined measurement for assessing cultural determinants than currently being used (e.g., major-above the 50 percentile; minor-from 25 to 49 percentile). 5. Relate the cognitive style elements associated in this study with successful shorthand students to the tests used in the literature to associate the corresponding characteristics with successful shorthand students. This approach would allow additional instrumentalities to be incorporated in determining cognitive style maps for students. 6. Include the factor of attitude toward shorthand in future studies which attempt to relate cognitive style to success or lack of success in shorthand performance. 7. Study the consequences of grouping students with teachers possessing cognitive styles similar to those of their students. Would these students achieve higher grades in given subjects than they would if they were grouped in any other fashion? 8. Examine the consequences of having students use instructional materials (e.g., texts, visual aids, audio tapes) developed by persons possessing cognitive styles that match or come close to matching the students' styles. Would these students achieve higher grades in given subjects than they would if they used materials developed by persons with cognitive styles disjunct to those possessed by the students? 99 9. Conduct similar investigations at the high school level with students at various achievement levels. What findings would occur and what types of conclusions and recommendations would result if this study were replicated with samples drawn from other grade levels? BIBLIOGRAPHY BOOKS Conant, James B. The Education of American Teachers. New York: McGraw-Hill Book Company, Inc.,T196O. Dixon, W. J. and F. J. Massy, Jr. Introduction to Statistical Anal sis. 2d ed. New York: McGraw-HillTBook Company, Inc., I957. Guilford, J. P. Nature of Human Intelligence. New York: McGraw- Hill Book Company, Inc., 1967. Newcomb, Theodore, Turner and Converse. Social Psychology. New York: .Holt, Rinehart, and Winston, Inc.,81965. Tonne, Herbert A., Estelle L. P0pham, and M. Herbert Freeman. Methods of Teaching Business Subjects. New York: McGraw-Hill 860k Company, Inc., 1965. ARTICLES, MONOGRAPHS, AND SERVICES Blanehard, Clyde I. "Results of a Study of the Validity of the Hoke Prognostic Tests of StenOgraphic Ability," The American Shorthand Teacher, X (January, 1930), 196. Eyster, Elven S. "Prognosis of Scholastic Success in Shorthand," The National Business Education Quarterl , XVII (December, 1938), 31-34. Flanagan, John C. "General Considerations in the Selection of Test Items and a Short Method of Estimating the Product-Moment from the Data at the Tails of the Distribution," Journal of Educational Psychology, (1939), 674-680. Gagne, Robert M. "The Acquisition of Knowledge," Psychological Review, Volume 69, (1962) 355-365. Guttman, Louis. "An Outline of Some New Methodology for Social Research," Public Opinion Quarterly, 18 (Winter, 1954-1955) 393-404. Hosler, Russell J. "Aptitude Testing in Shorthand," Journal of Business Education, XXII (May, 1947), 25. lOO 101 Jack, Melvin C. "Can we Predict Success in Shorthand?" The Balance Sheet, XXXIII (January, 1952.) 217-219. Jessup, E. "A Prognostic and Achievement Test in Shorthand." Journal of Commercial Education, LVIII (June, 1928). 173-174. Lambrecht, Judith J. "Aptitude Testing in Shorthand," Business Educa- tion Forum, (October, 1972). 17-24. Oakland Community Colle e Catalo ue. Bloomfield Hills: Oakland Community CoTlege ress, (l . 3-4. Turse, Paul L. "Prognostic Studies in Business Education," National . Business Education Quarterly, XXXXV (Winter, 1966-1967), 53. I q). U.S., Department of Labor, "Occupational Outlook for Stenographers, Secretaries, and Typistsf“ Occupational OutTook Report Series, T966067 Washington, U.S. Department of Labor, Government Printing , Office, (1966). : Wasser, Lawrence. The Educational Science of Cognitive Style: An I Introduction. OakTand"Community College Press, (1971). DISSERTATIONS Blanzy, James J. "Cognitive Style as an Input to a Mathematics Curriculum System: An Exploratory Study in the Educational Sciences." Unpublished Doctoral Dissertation, Wayne State University, (1970). Byers, Edward L. "Construction of Tests Predictive of Success in First-Year Shorthand." Unpublished Doctoral Dissertation, Boston University, (1958). Coleman, Brendon G. "The Effects of a Type Laboratory Instructional I Approach Upon Achievement in Beginning Shorthand Classes." ‘ Unpublished Doctor's Thesis, Michigan State University, (1964). i l Doubleday, Lewis. "A Study of the Factors Affecting Achievement in I Shorthand." Unpublished Master's Thesis, State University of I Iowa, (1939). 1 Evans, Ernestine. "Factors Related to Varied Achievement in Shorthand on the College Level." Unpublished Master's Thesis, State College of Washington, (1941). Fragale, Marvin J. "A Pilot Study of Cognitive Styles of Selected Faculty Members and Students in a Community College Setting." Unpublished Doctoral Dissertation, Wayne State University, (1969). 102 Frink, Inez. "A Comprehensive Analysis and Synthesis of Research and Thought Pertaining to Shorthand and Transcription 1946-1957." Unpublished Doctoral Dissertation, Indiana University, (1961). Goddard, M. L. "The Potential Role of the Junior College in Education for Bgsiness." Doctor's Thesis, Bloomington: Indiana University, 1962 . Hale, Jordon. "A Factor Analysis of Shorthand-Transcription Ability." Unpublished Doctor's Dissertation, New York University, (1958). Hendrickson, Rosanne C. "The Differential Aptitude Tests for Verbal Reasoning, Numerical Ability, Abstract Reasoning, Space Relations Mechanical Reasoning, and Clerical Speed and Accuracy as Predictors of Success in Shorthand." Dissertation, Unpublished Master's Thesis, University of Minnesota, (1963). Hill, Joseph E. "The Educational Sciences." unpublished Manuscript, Wayne State University, (1968). - Hill, Joseph E. "The Educational Sciences." Unpublished Manuscript, Wayne State University, (1971). Hoogasian, Vaughn. "An Examination of Cognitive Styles Profiles as Indicators of Performance with a Selected Discipline." Unpublished Doctoral Dissertation, Wayne State University, (1970). Kessinger, E. “A Prognostic Study in High School.Shorthand." Unpublished Master's Thesis, Louisiana State University, (1936). Lambrecht, Judith J. "The Validation of a Revised Edition of the Byer's Shorthand Aptitude Test." Doctor's Thesis, Madison: The University of Wisconsin, (1971). ‘ Lang, M. J. "The Relationship Between Certain Psychological Tests and Shorthand Achievement at Three Instructional Levels." Doctors Thesis, Columbia: University of Missouri, (1961). Lynch, Aline. "Factors Related to the Achievement of the One Hundred- Four High School Seniors in a First Course of Shorthand." Unpub- lished Master's Thesis, University of Michigan, (1947). Maedke, Wilmer O. "The Relative Prognosis Value of Selected Criteria in the Prediction of Stenographic Success or Failure in Selected High Schools in Illinois." Unpublished Doctoral Dissertation, Northwestern University, (1957). McAdam, Glenn F. "Personalizing Instruction Through the Educational Sciences of Cognitive Style and Teaching Style." Un ublished Doctoral Dissertation, Wayne State University, (19718 103 Moskovis, Michael L. "An Identification of Certain Similarities and Differences Between Successful and Unsuccessful College Level Beginning Shorthand and Transcri tion Students." Doctor's Thesis, Michigan State University, (1967 . Osborne, Agnes E. "The Relationship Between Certain Psychological Tests and Shorthand Achievement." Published Doctor's Dissertation, Columbia University, (1943). Poland, Robert P. "Implications of Certain Social, Economic and Technical Trends on Business Curricula in the Public Community Colleges of Michigan. Doctor's thesis, East Lansing: Michigan State University, (1962). Schroeder, Arlan. "A Study of the Relationship Between Student and Teacher Cognitive Style and Student Derived Teacher Evaluations." Unpublished Doctoral Dissertation, Wayne State University, (1969). Shuert, Keith L. "A Study to Determine Whether a Selected Type of Cognitive Style Predisposes One to do well ianathematics." Unpublished Doctoral Dissertation, Wayne State University, (1970). Takasugi, Dorothy. "The Relationship Between Certain Psychological Tests and Other Selected Factors with Shorthand Achievement." Unpublished Master's Thesis, University of Southern California, (1961). Terrill, Chester J. "The Value of the Hoke Prognostic Test of Steno- graphic Ability as a Means of Selecting Shorthand Students." nguinshed Master's Thesis, New York State College for Teachers, * 927 . Varah, L. J. "Effect of Academic Motivation and Other Selected Criteria on Achievement of First and Second Semester Shorthand Students.“ Doctor's Thesis, East Lansing: Michigan State University, (1966). Veon, Dorothy H. "The Relationship of Learning Factors Found in Certain Modern Foreign Language Aptitude Tests to the Prediction of Shorthand Achievement in College." Unpublished Doctor's Dissertation, Oklahoma Agricultural and Mechanical College, (1950). Warner, James L. "An Analysis of the Cognitive Styles of Community College Freshmen Enrolled in a Life Science Course." Unpublished Doctoral Dissertation, Wayne State University, (1970). Wasser, Lawrence. "An Investigation into Cognitive Style as a Facet of Teachers' Systems of Student Appraisal." Un ublished Doctoral Dissertation, The University of Michigan, (1969? “T-fi _ . . 104 Wood, Ethel H. "Correlation of Prognostic Test and Will-Temperament Tests with Actual Results in Gregg Shorthand." Unpublished Master's Thesis, Washington State College, (1928). Wright, Ellen M. "A Summary of Recent (1940-1972) Selected Findings in Shorthand Prognosis with Specific Reference to the Use of the Byers' First-Year Shorthand Aptitude Tests at the High School in Southington. Master's Thesis, New Britain: Central Connecticut State College, (1963). Wynn, Charles M. "The Educational Sciences: A Bird's Eye View." Unpublished paper, DakTand'Community—ColTége, T1973). APPENDICES APPENDIX A 1 05 Ed. Sciences Manuscript J. E. Hill APPENDIX A PROBLEM SET ON DISCRIMINATIVE POWER OF SELECTED TYPES OF TEST ITEMS ASSOCIATED WITH SAMPLING CERTAIN COGNITIVE STYLE ELEMENTS Background Information The basic purpose of a test, inventory, orrating procedure is to place or categorize the sub- Iects under study on a defined continuum according to the amount of the characteristic under study which their responses seem to protray or that they exhibit in behavior under defined condi- tions. The instrumentality, i. e., the instrument or the technique, which can perform this function successfully is said to have high discriminative power. It should be noted that the dis- criminative power of an instrumentality is greatly affected by the reliability and validity of the instrumentality. In other words, the characteristic of discrimination is more apt to be associated with a valid and reliable instrumentality than it is with one which is not. The rationale for the discrimination power of items included in a data collection instrumentality is based on the assumption that a different quality or magnitude of response should be expected from individuals, or groups of individuals, according to how they might differ in terms of the variable, or variables, under consideration. For example, pupils of superior ability in mathematics should answer a difficult item on a mathematics test correctly more frequently than students with little mathematical ability . Example. A procedure for determining the discriminatory power of a test item is illustrated * by a modified example taken from Greene,Jorgensen, and Gerberich: An experimental test was given to a class of 100 pupils having a normal range of ability in the subiect. The tests were corrected and scored. On the basis of these scores, the pupils were divided into three groups. 1'Greene, H. A., Jorgensen, A. N., Gerberich, J. R. Measurement and Evaluation in the Secondary School. David MacKay Company, Inc., New York: 1955. 106 Ed. Sciences Manuscript J. E. Hill The 27 percent of the pupils making the highest scores constituted the superior group; the 27 percent making the lowest scores formed the inferior group. The 46 percent of the class in the middle were not considered in computing the index of discrimination. The use of the 27 percent comprising each of the extremes followed a proposal made by Kelley and further exploited by Flanagan [John C. Flanagan, "General Considerations in the Selection of Test Items and a Short Method of Estimating the Product-Moment Coefficient from the Data at the Tails of the Distribution," Journal of Educational Psy- chology, 30:674-80, December, 1939] for this purpose. The next step involves an item count for all the items in the test showing the percent of pupils in the F. superior group and the percent of pupils in the inferior group that answered the respective items correctly [it should be noted here that some students in each of these groups might not respond to the iterfl’f A summary of a brief sampling of items from a typical test is given in Table I. TAB LE I Discriminative Power of Test Items in Percent of Success by Superior and Inferior Groups ‘51:...” Superior Group Inferior Group Index of Item High 27% Low 27% Discrimination 1 12 4 .23 12 6 4 .08 23 ' 8 14 -.13 44 10 18 -.15 55 24 13 .15 97 52 12 .46 I40 92 40 .59 This table indicates that Item I was answered correctly by 12 percent of the superior and 4 percent of the inferior pupils [N . B. that the percent of students not responding to the item in each group is not shown, and that this situation-has an influence on the value of the Index bTDiscrimination] .* The value of the Index of Discrimination [also called "The Validity Index"] * can be found by the two x two table method of computing the O (phi) coefficient. Item I shows great difficulty and a limited power to discriminate between good and poor achievement. The fact that the item is answered correctly by such a small proportion of all pupils (average 8 percent) indicates that its difficulty is great. Item 44 is correctly answered by a smaller percent of superior pupils than of inferior pupils. This condition is shown by the negative discrimination ,.-.. index (-.15). The negative value of the index indicates that the item is at fault or the wrong facts have been taught in the subiect. The item should probably be eliminated from the test. Items 97 and 140 with positive indices of .46 and .59 are probably good enough to retain in the test. *Inserta information . 107 Ed. Sciences Manuscript J. E. Hill This method of determining the discriminative power of test items is widely used in the critical analysis of test items for standardized tests. . . . . The value of the Index of Discrimination associated with a given item can be calculated by means of the ¢(phi) coefficient, providing the number of persons not responding to an item is known, or if the number responding incorrectly to the item is known, along with the number responding correctly to it. A two x two table and its associated general form of the phi co- efficient could be shown as follows: ITEM I Group Response Superior Inferior Total Correct a b a + b Incorrect c d c + d Total a+c b+d a+b+c+d ad - bc ¢ = J(a+c)(b+d)(a;b)(c+dl To illustrate how the table might appear, and how the value of O is calculated, suppose that on a given Item "X" the data were arranged as shown: ITEM I x 2323 Response Superior Inferior Total Correct 25(students) 9(students) 34(students) Incorrect 2(students) 18(students) 20(students) T°I°I 27(students) » 27(students) 54(students) r 108 Ed. Sciences Manuscript J. E. Hill Then the value of ¢would become: 450 - 18 _ 432 16 ¢ = 25 18 - 9 2 = _ = 7I2‘T7W27 (120“);= 27768—0 (2"5—‘TT7 (26. 2.67%.613 Biserial Correlation Although a number of statistical methods have been devised for use in determining the discriminative power of an item (e. g. , phi coefficient), the use of the biserial correlation IIb coefficient" is usually regarded as the standard procedure in item analysis. The value of the biserial r shows the correlation of an item with total score on the test, or with scores on some independent criterion. The adequacy of other methods, most of them being summary in nature, is Iudged on the basis of the degree to which they yield results that approximate those obtained by the "biserial" method. Garrett” outlines a method for determining the discriminative power of an item (validity index), which he describes as "much favor " by test constructors, and, "one of the best among several methods." The procedure, somewhat modified in terms of the number of examinees employed, and in the descriptions associated with each step, is as follows: _S_t_ep_l_. Assume that there are 100 examinees, and arrange the test papers in descending order of the test scores, i. e. , place the paper with the highest score at the top, ~. and the one with the lowest score at the bottom of the pile. I" . M According to F lanagan's technique, count off the top 27% of the test papers ' (since n = 100, count off the top 27 papers), and the bottom 27%, i. e., the bottom 27 papers. It should be emphasized here that Flanagan has "proved" hm that the discriminative power of an item is most accurately determined when t *See page 375, Henry E. Garrett, Statistics in Psychology and Education, Longmans, Green and Co., Fifth Edition, New York: 7958. * *Llfl, pgs . 365-367 . 109 Ed. Sciences Manuscript J. E. Hill item analysis is based on the top and bottom 27% of the distribution, re- spectively, rather than any other percent of the respective ends of the distribution . S3332: Disregard the 46 papers associated with the "middle" of the distribution of scored papers. Step}; Count the nunber in the top group who responded correctly to the item being analyzed; and the number in the bottom group who responded correctly. Con- vert these numbers to percents. For example, if 24 of the 27 students in the top group responded correctly to the item, the percent would be: 24/27.\\’.90. M Correct these percents for chance success, i. e. , correct the difficulty index* for chance success, by the formula: W P . 1E. . c N - HR ' where Pc denotes the percent of the students who actually knew the right answer, i. e. , did not "guess" the answer; R indicates the number responding correctly, W the number responding incorrectly, N the number of examinees in the group under consideration, HR the number of examinees who did not respond, and K denotes the number of options or choices associated with the item under consideration. To illustrate this point, suppose that in the top 27% of the sample of 100 examinees, i. e. , suppose that among the 27 students forming the top group, as in Step 4, 24 respond to the item correctly, 2 respond incorrectly, *The value of the difficulty index of an item is the arithmetic average of the percents correct in the upper and lower groups. For example, suppose that on a given item, 90% of the upper group show a correct response, and 30% of the lower group respond correctly. The approximate value of the difficulty index for the item for the tethgroup is: .9o+.3o =1.20 ___ 60 .2 2 ' 110 Ed. Sciences Manuscript J. E. Hill and 1 does not respond, then: R = 24; W = 2; N = 27; HR = 1; and if there are four options associated with the item in question, K = 4. Under these circumstances, the value of the "corrected" difficulty index would be: 2 P = 24-“ = 24'2/3 =33—°—3§ =.897\.9O c ‘77— T 26 \ Since this value is the same as the approximate percent found for the number with 9f "‘9 I°P 9V°UP P055109 in Step 4, in this case, correcting this percent for ; chance success was of dubious value. I m Employing Table 51 (attached) along with the value of percent success in each h of the respective groups, i. e. , "tap" and "bottom, " the appropriate value of the biserial r (the discriminative index or the validity index for the item) may be found in the cell formed by the intersection of the appropriate column and row. For example, suppose the corrected percent of correct responses for the upper group was: .90; and for the lower group was: .30; the value of the dis- criminative index or validity index would be: This = .63. M To find the value of the difficulty index of the item, compute the simple arith- metic average of the two percents, i. e. , add the percent of the top group to that of the bottom group and divide that sum by 2 (see footnote regarding cal- culation of value of difficultv index). _———.—-.——-- ‘I—auzr—IMIH, ‘ J . 111 I" . TABLE 5| "' Normalized biserial coefficientsf of correlation as determined from proportions of correct responses in upper and lower 27 per- cent of the group "amour or connect nanome- sx nu men 21 on can! i o: os to N is :2 26 so so so 4: 46 so ss so a: so 7o 14 to a: as oz ooissosrssusisssosi6366601012 rs 7511 rosoazos os ooniozosisssous‘rsosssssssissssssn737673 to oo or is 21 as so as as n as 48 s: so 51 so as as so 11 74 En 00 oz 12 1s 22 21 at so as a 45 so 51 ss s1 so as or to zts ooosnlszozszsszsssossstsssssssosssr :2 ooosioisiszsztsissssszsswszsssoss Eu oooso9uis22 2630333141 «as: 5660 330 oo 04 09 13 11 21 as :9 ss 31 so u «.9 ss 51 so «pawnsunzszosssrusswss so wososrslszozsnssstrzusi 543 ooosooiziszozszssssssss has 00 or or i: is 21 25 so 34 39 4s so ooosos 1311212631 3642 so 00040313112221 :3: so so as 09 is is 23 23 so r362 00040914192531 so so 04 09 IS so 21 .10 oo 05 to is 22 I 574 on as n is ; to no as 12 ' 3.: on or . goo oo 4 so C £3: ~ g l ozosiouiszzzssossssszsssossssszsosoinsure I ' This table is abridged from J. C. Flanagan's table of normalized biserial coeffi- cients originally prepared for the Cooperative Test Service. It is included here with the generous permission of Dr. Flanagan and the Educational Testing Senice of Princeton, New jersey. This version is taken from Merle W. Tate, Statistics in Educa- tion (New York: The Macmillan Co., 1955), p. 364. Reproduced by permission. f Decimal points are omitted. III the pro rtion of correct responses in the lower 27 percent exceeds that in the upper, enter IE: table with the lower 27 percent proportion at the top and attach a negative sign to the coeflicient. - r ‘HMAA_ 112 Ed. Sciences Manuscript J. E. Hill APPENDIX A PROBLEM SET ON THE RELIABILITY OF MAPPING "STYLE" ELEMENTS Under the assumption that an individual's "educational" cognitive style, teaching style, administrative style, or counseling style can be modified or augmented, i. e. , these "styles" as defined in the Educational Sciences are dynamic not static in nature; the problem of deter- mining the reliability of the instrumentalities, i. e., instruments and techniques (e. 9., tests, inventories, and empirical mapping),requires a particular approach. Basically, there are three theoretical models of reliability of measurement: (1) the concept of true and error scores, (2) the eclectic concept of true scores and parallel tests, and (3) the concept of domain sampling. These three basic concepts of reliability are associated with one or more of I93 methods usually employed to estimate the value of the reliability coefficient of an instrumentality under con- sideration.* In order to accommodate the dynamic nature of the "styles" defined in the Educational Sciences, it is necessary to employ the concept of domain sampling and the method 9: inter- correlations among the elements of a sample or test to determine the reliability of the instrumen- talities employed to produce these "styles." In addition to the "reliability coefficient," an index of relationship, as an indicator of the reliability ofa domain sample (e. 9., inventory), the "standard error of measurement" also provides another means of assessing the reliability of a "sample." Considerations Regarding Rel iabilityéloeffi cients The method of intercorrelations among items can be employed to yield a precise and exact value of the reliability coefficient associated with each domain of the orientations of each "See pages 23-25 of chapter on mapping. A i I 113 Ed. Sciences Manuscript J. E. Hill -2- element of the respective "styles" (e. g., cognitive, administrative) of the individual, and thereby, to yield the value of the reliability index of the mapping process employed by the diagnostician regarding the "elements" under consideration. Formula (I), the Kuder-Richardson formula, can be used to determine the value of the reliability coefficient, rxx 2 S _ K I xi [xx—K-I I- 52 (I) x Where K denotes the number of elements in the sample; "z " denotes "the summation of;" Sfi indicates the value of the variance for each element; and 5: the value of the variance of the total number of measurements (e. g. , scores, ratings) included in the sample. The following miniature example illustrates how the value of the reliability coefficient for a sampling of the domain "qualitative code histrionics, Q(CH) - ability to play a role to produce a particular effect on other persons," was determined from the ratings (0 through 9) by four diagnosticians of the behaviors of the five individuals under consideration. (Assume that the individuals were observed by the diagnosticians at the some time and in the same situ- ations.) (See next page .) 114 Ed. Sciences Manuscript J. E. Hill -3- Individual (n = 5) 2 l 2 3 4 5 ZXZ_(_&0_ 2_ n . .. 2 2 2 2 2 s.- Dtagnostrcran x x x x ~ x x x x x x xI n-I 2 207-———(3” A 4169816367495252_ 5_ sA- 4 -3.7 2 ”1-99). B 5255254166363 952-‘5—13 [VT—T— T 2 173-(29) c 4166366367496362_ ‘5—_ 5c ‘—T— -I 2 7 I90-£1Q.I_ o 5258646367494le2= 5:25 K___4 o 4 4 Z x 18 28 22 27 18 ZQXI = 113 K=l )2 ($02 182= 324; 282= 7B4; 222= 484; 272- 729; 182 = 324 2(sz2= 2645 2 2 2 (gm _(ns) _12,769 '. 23(le ' n 3645 T3645 ‘3'— =2645-2553.8=9l.2 = Sx n-I 4 4 4 T ”'8 {Sit =3.7+ 1.3+ l.2+2.5=8.7 7.52 rxx K-I S2 4-l 2278- 3 ' 3 ' X r =4(.206) = .824 xx If the elements of a domain sample are in the form of items included in an inventorydesigned to elicit dichotomous responses (e. 9., "yes" or "no," "true" or "false"), then the value of the ‘4‘ b“h‘k*-—_I —,' . -o.-—.-_._. ‘___. 115 Ed. Sciences Manuscript J. E. Hill -4- reliability coefficient can be calculated by means of formula (2), a modification of formula (I) designed to accommodate those instrumentalities involving dichotomous variables in the measurement process: _ K I;"fimi 'xx "Terr ‘ ' ___—2'- (2) n(n-I)Sx In formula (2), rxx' K, and S: are as previously defined in (I), n indicates the number of individuals being tested or considered, p denotes the proportion of the individuals responding to an item in terms of the "desirable characteristic," where the desirable characteristic is the response (e. g. , "yes" response) associated with the characteristic (variable) under study, and q denotes the proportion of individuals responding to an item in favor of the "other characteristic" (e. g. , "no" response). Dichotomous variables are quantified by assigning a score of " 1" to each response associated with the desirable characteristic and a score of "0" to each response associated with the other characteristic. To illustrate, suppose the "desirable characteristic" was a "yes" response, and that the "other characteristic" was a "no" response. If from the responses of: n = 100; individuals to a given item there were found to be 60 "yes" and 40 "no" responses, then each of the 60 "yes" responses would be assigned a score of I, and each of the 40 "no" responses would be assigned a score of "0." The proportion p of the sample responding in favor of the desirable characteristic would be: p = 60/100 = .60; and the proportion q of the sample responding in favor of the other characteristic would be: q = 40/100 = .40. It should be noted that: P+q=L that is: .60 + .40 =1, and therefore: q = l - p {-‘JJ 116 Ed. Sciences Manuscript -5- J. E. Hill The following miniature example illustrates how formula (2) might be applied to find the value of the reliability coefficient for an inventory composed of ten items (each of which calls for a "yes" or a "no" response, with "yes" (X = I) being the desirable characteristic) admin- istered to eight individuals. Indivifial (n = 8) Item A B c o e r c H npi(X=l) nqi(X=0) (npi)(nqi) I 1 o I I o o o I 8(4/8)=4 8(4/8)=4 16 2 I I I o o o o I 8(4/8)=4 8(4/8)=4 I6 3 o o I 1 o 1 o I 8(4/8)=4 8(4/8)=4 16 4 1 1 o o o o o I 8(3/8)=3 8(5/8)=5 15 5 I 1 1 I 1 I o I 8(7/8)=7 8( l/8)=l 7 6 o I I I I I 1 1 8(7/8)=7 8(1/8)=I 7 7 o I 1 I 1 I I 1 8(7/8)=7 B(I/8)=l 7 8 o 1 I I o I 1 I 8(6/8)=6 8(2/8)=2 : 12 9 l o l 1 o o 1 1 8(5/8)=5 8(3/8)=3 ' 15 10 o l I 1 o 1 1 1 8(6/8)=6 8(2/8)=2 12 7 9 8 3 6 5 10 £(ZX) '= 53; 27,- 123:2(npi)(nqi (ZX)2 52:25; 49 81 64 9 36 25 100 ;(zx12=389. 2 {np.nq. (rt-115: =}:2 -———‘z§f° ; I; K'f 1- ——-2' ' n(n-I)S 2 (8-05: = 389 1513:1- 752 = 389 - 23:1 =389 - 351.1 =37.9; __10 9 (“123 - ‘0 _10 _ (816779)) '9' (I- -406)- -§( . 59.26. ...-_~.ao *3”.‘ I- c_‘.-I. m- a 117 Ed. Sciences Manuscript J. E. Hill -6- Estimating the Effect Upon Reliability of Lengthening an Instrumentality. Averaging the results of five different "samplings" (e. g. , observation-assessment periods, inventories) of a domain of the orientations of a given style element should lead to informa- tion that is more trustworthy than that accruing to a single sample. By the same token, the mean of 10 "assessments" should be more dependable than the mean of the results associated with the five different samplings. Increasing the length of an instrumentality or averaging the results obtained from several different applications (samplings) of the instrumentality (or samples parallel to it) will increase the reliability of the instrumentality." A good estimate of the effect of increasing the length of an instrumentality or repeating its application a number of times can be obtained by employing the Spearman-Brown prophecy formula: nr 'nn = Ifinfxl )rxx (3) Where rnn denotes the correlation between "n" forms of an instrumentality and "n" alternate forms [or the m (arithmetic average) of n forms correlated with the r_n_egg of n other formg; and rxx indicates the value of the reliability coefficient associated with the given (or origindl) imtrumentality. To illustrate the use of formula (3), suppose that the value of the reliability coefficient of a team of four diagnosticians involved in mapping the orientation in qualitative code histrionics, Q(CH), of each of 25 persons, based M93391 6__t observations, is 'xx - .80. *In fhe mafhematical context of the formulae involved, averaging the. scores from "n" appli- cations of an instrumentality yields the same result as increasing the length of the instrumentality "n" times. It should be noted again that the term "instrumentality" pertains to both instruments and techniques (e. 9., rating of an individual by an observer). It is in this context that the phraseology "increasing the length of an instrumentality" might pertain to increasing the length of time of a given observation period, or increasing the number of periods, as well as adding items to a given instrument. ti 118 Ed. Sciences Manuscript J. E. Hill -7- What would be the effect on the value of the reliability coefficient for the team, if instead of having the 25 individuals being mapped participate in one set of "designed" activities, they were to participate in f_ou_r_s.e_ts_ (parallel forms) of these activities. Since this approach would, in effect, quadruple the "length" of the domain sampling, i. e., base the "mappings" of the diagnostic team on IE?! 21s of observations of "designed" activities (considered to be parallel forms) instead 080232, the values of: n = 4, and rxx = .80, would be substituted in‘formula (3), and the value of rm would become: _4x.80 _ 3.20 3.23%.94 '46? - W - m - Quadrupling the instrumentality's (the diagnostic team's ratings of the individuals under con- sideration) "length," therefore, increases the value of the reliability coefficient from .80 to .94. It should be noted here that predictions of increased reliability by the Spearman-Brown formula are valid only when the items or set of behaviors added to the instrumentality pertain to the some aspects of the variable under consideration, are of approximately the some difficulty, and are comparable in other respects of the "designed" items or set of behaviors commising the original instrumentality. It is difficult to satisfy these conditions, not only in terms of the content or materials to be employed, but in terms of motivation, fatigue, and, sometimes, the integrity of the individual under consideration. When the conditions for prediction are satisfied, however, empirical evidence indicates that an instrumentality may be increased by five or six times its original size (length) and the prophecy formula yields a close estimate of the "expected" (exper- imentally determined) results. If increases greater in length than five or six are employed, the prophecy formula tends to "over predict ," i. e. , yield a higher estimate of the value of the reliability coefficient than those found in situations employing the actual number of items or sets of behavior involved. 119 Ed. Sciences Manuscript J. E. Hill -3- Applying the Prephecy Formula to Ratings Formula (3) may be used to estimate the value of the reliability coefficient for ratings issued by two or more diagnosticians, paired comparisons and other iudgments, as well as test scores. Suppose that in rating the behaviors of 25 individuals in Q(CH) - qualitative code histrionics, the ratings (range 0 to 9) of two diagnosticians (both assumed to be of equal experience, and to be equally well-acquainted with the individuals under consideration) show a reliability coefficient (correlation) of: rxx = .70. If the ratings by the two diagnosticians were averaged, how reliable would the means of these two sets of ratings be? Employing formula (3), it is found that: 2 x .70 1.40 '2u=WTTF7o'- . ="""1.7o If instead of two diagnosticians there had been three diagnosticians whose ratings showed an % .82 average correlation of: 'xx = .70,* then the mean value of the prophecy formula estimate of the averaged ratings of the three diagnosticians would be: _ 3x.70 __2.10__ ‘ f3m-1—W - o 0 -m -‘ .875 The confidence that can be placed in these estimates depends upon how well the assumptions Wy, in order to "average" correlation coefficients that are "product-moment" in form, or that are close approximations of it, each coefficient must be first translated into the form: 2 = l/2 In -:-:—TF; where "In" denotes the "natural logarithm," i. e., log to the base "e"; then the 2 values are averaged arithmetically. The "antilog" of this mean value (2) is found from a "natural logarithm" table, and this value is employed in the equation: e22 = }-:—:—; which in turn is solved for the average value of rxx' If, however, the values of the correlation coefficients to be averaged are not too widely divergent, a simple arithmetic average of the rxx's involved, provides an adequate estimate of the mean value of 'xx l ‘I 120 Ed. Sciences Manuscript J. E. Hill -9- underlying the use of the formula are met, i. e. , whether the diagnosticians are "equally experienced," are "equally well-acquainted with the individuals under consideration," and are equally skilled in the observation techniques being employed. It should be noted that in most cases, increased estimates of 'xx such as these must be taken as approximative in nature. Under these circumstances, the values of rxx that are found can be useful but are at. exact. Lengthening the lnstrumentality to Attain a Desired Degree of Reliability Formula (3) may also be used to determine how many times an instrumentality should be lengthened or repeated to yield a particular standard of reliability. To illustrate this point, suppose that a diagnostic team has a reliability coefficient: rxx = .80, in mapping the orien- tations of qualitative code empathetic, Q(CEM) - the ability to identify with the role of the other person, based upon a: set of observations. How many parallel sets of observations are needed to increase the value of this reliability coefficient for the "team" to .90? Substituting the values of rxx = .80, and r"n = .90 in (3), and solving for "n," it is found that: 90: .80n ___ .80n = .80n ‘ ‘ lilo-1535 HEM-.30 .§6+.§0n ? (.20+.80n).90 = .80n .18+.72n .80n .mn = .18 _ , Z n = 2.25, or rounding to the next whole number, 3. ¢___ If to: parallel sets of observations are employed instead of one, and providing all other conditions can be satisfied, e. 9., the group of individuals being observed will not become fatigued or bored, the value of the reliability coefficient for the mapping of the orientations on(CEM) would be .90. 121 Ed. Sciences Manuscript J. E. Hill -10- The Standard Error of Measurement Recognizing the fact that variable or chance errors are included in measurements ob- tained by, or assigned to, and individual from or by a given instrumentality, and that there- fore these measurements are divergent from what might be termed their "true" values, the standard error of measurement provides a vehicle for reflecting the magnitude of these effects. The magnitude of the standard error of measurement can be found by formula (4): em = Sx I - ''xx (4) Where gem denotes the standard error of measurement, Sx indicates the value of the standard deviation of the measurements yielded by the instrumentality, and rxx the value of the reliability coefficient of the instrumentality. It should be noted that the standard error of measurement provides a measure of the error made in estimating the value of the "true" measurement of an individual by an "obtained" rneasurement.‘II Since the standard error of measurement takes into account the variability within the group regarding the measurements yielded by the instrumentality *fiJ. The difference between the standard error of measurement (Sam) and the standard error of estimate: Sest = Sle - r ; where Sy denotes the value of the standard deviation of the Y distribution of Y-values (e. g. , reading scores), and r the value of the coefficient of correlation between the X (e. 9., history test scores) and the Y values. If X scores were to be predicted from Y, the standard error of estimate would take the form: ‘Se = l - r ; where Sx denotes the value of the standard deviation of the distribution of X-valjg, and r is as previously defined. The standard error of estimate is used with prediction, by means of the regression equation connecting the variables, the most probable values of the dependent variable (e. g. , Y) which can be obtained from a given value of the independent variable (e. 9., X). The standard error of estimate can be used to predict the range of "most probable" scores that an individual might ttain on Test A (e. 9., mathematics) when his score on Test 8 (e. 9., reading) and the correlation between the set of paired scores (mathematics and reading) is known. 122 Ed. Sciences Manuscript. J. E. Hill -11- as well as the reliability of the instrumentality itself, it is actually a better way of expressing the reliability of the instrumentality than that provided by the reliability coefficient. The following example illustrates how formula (4) can be employed. Suppose the reliability coefficient on the listening test associated with the measurement of theoretical auditory linguistic ability T(Al.) is .84, the mean for the papulation of scores associated with the test is 75 and the population standard deviation is ,9' lfan individual earns a score of 82, what is the standard error of this measurement (score)? What is the individual's true score? According to formula ‘4'): S =5 -r em x xx =9 ”=9 W=9(.4) =3.6 and, if the population of scores associated with the test is assumed to be distributed normally, then approximately two-thirds of the scores will occur within 1- one standard deviation of the mean. Under these circumstances, it can be expected that approximately one-third (1 /3) the entire group forming the population will earn scores that are in error more than 13.6 points, and two-thirds (2/3) will be in error this amount (i 3.6 points) or less. In order to find the true score of the individual, formula (5) must be employed: _ * x.°-rxxxI +(i - rxx) M (5) Where 3; denotes the value of the estimated true score (measurement) on the instrumentality, X 1 indicates the score (measurement) obtained on the instrumentality by the individual under consideration, M indicates the value of the mean (arithmetic average) of the distribution of measurements (e. 9. scores) associated with the instrumentality, and rxx denotes the value of the reliability coefficient for the instrumentality. In this particular example, the individual's true score would be: *The syFEol'O (infinity) denotes ”true" scores. 123 Ed. Sciences Manuscript J. E. Hill -12- Y“: .84(82) + (t-.84)75 = .84(82) + .1605) = 68.88 + 12.00 = 80.88%81 It should be noted that the individual's obtained score, 82, is within the range of one standard error of measurement (1- 3.6) of the t_rr_:_e score, 81. It should be noted that when the value of the reliability coefficient is high, or relatively (as in this example) little is gained from computing h the value of the estimated t_rt£ score, i. e. , the Md score is sufficiently accurate for use in the mapping process. When the value of the reliability coefficient is relatively low (e. g. , .40 :rxx 2.50), then the value of the estimated true score should be calculated. Concluding Remarks A question that is frequently raised is: How large, or of what magnitude, should the value of the reliability coefficient be in order to be considered satisfactory? The required magnitude of a reliability coefficient depends upon the nature of the instrumentality,the size and variability of the group being measured, and the purpose for which the instrumentality was administered. For example, if the problem to be solved is that of differentiating between the mean scores of two school grades of children, where a relatively narrow range (highest score less the lowest : score) of values probably exist, a reliability coefficient of: .50 3 [xx '5 .60, is satisfactory. . If, on the other hand, the instrumentality is to be used to yield data from which individual diagnoses are to be made, i. e. , to distinguish one person from another in a relatively precise manner, then the value of its reliability coefficient should be at least .80, and preferably .90 or higher. It is in this context that most constructors of standardized intelligence, aptitude, or educational achievement examinations strive to develop values of reliability coefficients of at least .90 between alternate forms of their tests. The value of the reliability coefficient of an instrumentality is also affected by the vari- ability of the group, and it is in this context that the value of the standard deviation of the _ 124 Ed. Sciences Manuscript J. E. Hill -13- distribution of scores yielded by the instrumentality should be given along with the value of its reliability coefficient. The method employed (e. 9., split-halves) to calculate the value of the reliability coefficient should always be reported (in the case of mapping "style" elements it is the "intercorrelation among sample elements," i. e., the Kuder-Richardson approach), as well as information about the group and that which pertains to the procedures that were employed as in administering the instrumentality. Without this information comparisons of results are difficult '5 if not impossible. It should be re-emphasized that the value of a reliability coefficient of an instrumentality administered to a group that is wide in its range of development (e. 9., a mathematics test ’ designed for a particular level of educational development administered to a group of students from several school grades) cannot be compared directly with that of a reliability coefficient of a relatively "parallel" instrument administered to a group of students covering a relatively narrow range of educational development. In short, the reliability of an instrumentality is influenced by the variability of the group in terms of the aspect(s) which the instrumentality was designed to measure. In a statistical sense, the greater the heterogeneity of the group, the greater the variability of the measurements yielded by the instrumentality, and the higher the value of the reliability coefficient. If the value of the reliability coefficient of an instrumentality, based upon data yielded by a group of individuals showing a wide range of development in the dimensions covered by that instrumentality, is known, the value of the reliability coefficient of the some instrumentality can be estimated for a group showing a narrow range of development, assuming the instrumentality is considered to be approximately equally effective over both (the wide and the narrow) ranges. Formula (6) shows the relationship among standard deviations and reliability coefficients ob- toined during different stages when the instrumentality is considered to be equally effective 125 Ed. Sciences Manuscript J. E. Hill -14- throughout both ranges: S Jl-r £=____ww (6) Sw Jl-r nn Where Sn and Sw denote the standard deviations of the measurements yielded by the instrumen- tality for the group of students of narrow range and group of wide range, respectively; and rm 3 and 'w denote the values of the reliability coefficients for the instrumentality based upon the measurements yielded by the instrumentality for the narrow range group and the wide range group. To illustrate how this formula might be used in mapping orientations of "style" elements, .. suppose that the instrumentality (e. g. , listening test) designed to measure theoretical auditory quantitative, T(AQ), ability (and eventually "orientation") is administered to a group of indivi- duals covering a range of levels of educational development from 6 through 12, inclusively. The value of the reliability coefficient based upon the measurements yielded by the instrumen- tality for this group is found to be: rWW = .91, and the standard deviation is: SW = 12.00. if, when the instrumentality is administered to a group of individuals showing a level of educa- tional development of 9, the standard deviation of the group's measurements is found to be: S" 34.00; assuming the instrumentality is iust as effective in measuring individuals at the single level of educational development (9) as it is throughout the range of 6-12, estimate the value of the reliability coefficient of the instrumentality for the "narrow range" group. Applying formula (6): S -r __n_= w .W Sw Jl-r 126 Ed. Sciences Manuscript J. E. Hill -15- I- 5F:— nn Jl - n: 3(.3)= .9 squaring both sides of the equation: l-r nn r =.19 ft" .81 Under these conditions, i. e., rw = .91, SW =12.00, and Sn =4.00, a reliability coefficient w of: 'xx = .19, based upon data yielded by the group of narrow range in the dimensions covered by the instrumentality, indicates as much measurement (e. 9. score) dependability as an rgx = .91 for a group in which the range of development is six times as wide. Although this condition, estimating reliability of an instrumentality administered to a group of narrow range from data yielded by the instrumentality when administered to a group reflecting a much greater range, is the one usually found to exist, it is possible, providing the assumption of equal effectiveness can be made, to estimate the value of the reliability coefficient for the wide range group from data yielded by a group showing a narrow range of development in the dimensions covered by the instrumentality. In short, formula (6) can be used to make estimates for the value of the reliability coefficient for either group. Other factors affecting the reliability of an instrumentality are: timed sequences (e. g. , speed tests), obiectivity, item difficulty, and differing discriminitive power of items included in the instrumentality. The topics of discriminative power and item difficulty must be taken into account by procedures designed to consider these matters (see problem set on "discriminative power of selected items"). If these problem areas are monitored appropriately, the reliability of an instrumentality is increased. In similar fashion, the problem area of abiectivity must be 127 Ed. Sciences Manuscript J. E. Hill ~16- covered by procedures designed to take this factor into account (see problem set on obiectivity) . The important relationship between reliability and timed sequences (e. g. , speed tests) to be taken into account is that in many cases, the examinees will not attempt all the items, and since the items included in speed tests are usually of a lower level of difficulty, frequently a spuriously high value of a reliability coefficient for odd-even halves of an instrumentality 128 Ed . Sciences Manuscript APPENDIX A PROBLEM SET ON VALIDITY OF MAPPING PROCESS The validity of an instrumentality depends upon how authentically it measures what it is designed to measure. An instrumentality is considered to be valid when the behaviors which it is designed to measure correspond to the same behaviors measured by or obiectively defined by a source independent from that which created the instrumentality. An example frequently used to clarify the concepts of reliability is that of a clock being set forward 15 minutes. If the clock is "reliable," i. e., consistent, it will maintain that amount of "fast" time, but the time it displays will not be valid relative to the standard of time for the region in which it is located. The validity of any measuring instrument can be found by comparing the information which it yields with so-called "standard measures of that information." Since independent standards, i. e., criteria, are difficult to establish in mental measurement, the validity of such an instru- mentality cannot be estimated as accurately as can the validity of an instrument designed to measure a physical aspect of the world. Validity is a relative, but yet specific, term. An instrumentality is valid for a specific purpose, or a particular situation, it cannot be generally valid. This situation is exemplified in numerous ways. When industrial concerns use an individual's performance on a general aptitude test as a predictor for successful performance in a routine clerical iob, they frequently discover that this type of instrumentality does not provide a valid measure-of the skills that : might be necessary to satisfactory performance in that type of job. ' Content and "Face" Validig Content validity is employed in connection with standardized educational achievement .mw tests. This type of validity is described in verbal terms and is based upon the consensus of iudgments of educators regarding the degree of knowledge that an individual of a given age or given leve‘l of educational development should possess in such subiects as history, English, mathematics, or geography. For example, a test of geography is iudged to be valid if it consists 129 Ed. Sciences Manuscript J. E. Hill -2- of questions that cover the subiect matter considered by this specialization. Content validity is appropriate for those situations in which: (1) competent professionals are available, and the items employed cover a wide spectrum of topics in the specialization to which the test pertains, and (2) adequate standardization groups are available. " Face validity," less rigorous in terms of the standards of judgment employed in content validity, is used to describe the validity of an instrumentality that ms to measure that which it was designed to measure. Judgments of face validity are useful during the initial stages of item analysis. Face validity should be used little more than as a first step in assessing the potentiality of an item for inclusion in an instrumentality, it should not be used as the final information accorded the validity factor of an instrumentality. Predictive Validity and Concurrent ValiditL The validity of certain types of instrumentalities can be determined on the basis of the cor- relation of values (e. 9., scores) yielded by the instrumentality and some independent criterion. A criterion may be an abiective measure of performance, or a iudgment of the excellence of the performance. For example, intelligence tests were first validated against performance of examinees in different levels of school grades, teachers' ratings of students' aptitudes, and other "substantial" indices of ability. Personality, attitude, and interest inventories are usually validated by comparing their predictions to actual outcomes. A high degree of correlation between an instrumentality and a criterion is evidence of validity if: (a) the criterion was created independently from the instrumen- tality, and (b) both the instrumentality and the criterion are reliable. The value of the index of reliability can be used as an approach consonant with the concept ofdomain sampling which must be employed in connection with the mapping of elements of a phenomenon as dynamic as "style." The value of this index of relationship indicates the degree 130 Ed. Sciences Manuscript J. E. Hill -3- of correlation between scores obtained by sampling the domain in question and their theoretically "true" counterparts found in that domain. In the context of a domain, an individual's true score can be defined as the mean (arithmetic average) of a very large number of samplings of the domain (theoretically possessed by that person) by the same instrumentality under standard conditions. The correlation between a set of obtained scores and their corresponding true counterparts can be found by employing formula (I): . rx ~= wxx (I) Where rx ”denotes the value of the correlation between obtained and true scores; and rxx the value of the reliability coefficient of the test. The coefficient tune is termed the index of re- liability. This index indicates how well obtained scores agree with their theoretically true values. The index also indicates the highest correlation the instrumentality (domain sample) is capable of yielding in its present form. By squaring both sides of the equation, it is found that: 2 l'xce" rxx and that the value of the reliability coefficient of an instrumentality indicates the proportion of the variance of the obtained scores which is determined by the variance of the true scores of the domain. For example, if the value of the reliability coefficient-for a given instrumentality (domain sample) is found to be .81, then 81 percent of the variance of the score obtained by that domain sample (the instrumentality) is attributable to their counterpart true scores in that domain. Moreover, since rxw= m = .90, it is known that .90 is the highest value of correl- ation which this instrumentality (domain sample) is capable of yielding in its present form. It should also be noted that if the value of the reliability coefficient of an instrumentality is as *The syEol'fifinfinity) denotes true scores. {We __ 131 Ed. Sciences Manuscript J. E. Hill -4- low as .25, and, therefore: rxm= .25 = .50, it is undoubtedly a waste of time to use the instrumentality in its present form. Lengthening it, or otherwise improving it, is indicated. An instrumentality whose index of reliability is only .50 is yielding poor estimates of the domain it was designed to measure. It is in this type of context that the index of reliability is considered a measure of validity. , , Validity and the Length of an lnstrumentality ‘r -.—.-——"h( 3 Not only is the reliability of an instrumentality (domain sample) increased by lengthening it, or averaging the results of n parallel forms of it, but its validity is increased as well. In this context, the lengthened instrumentality provides a better sampling of the domain, i. e., it provides a better measure of its criterion, than the original form did. The value of the validity coefficient of an instrumentality lengthened n times, or the average of the results yielded by n parallel forms of it, can be found from formula (2): nr r = cx A (2) cnx ‘./Fl+n(n-1)rxx Where rcnx denotes the value of the correlation between the criterion c and either n parallel forms of instrumentality x, or instrumentality x lengthened n times; 'cx is the value of the correlation between the criterion c and the original instrumentality x; rxx is the value of the reliability coefficient of instrumentality x; and n indicates either the number of parallel forms of instrumentality x, or the number of times it is lengthened. The use of the formula is illus- trated by the following example. If the value of the correlation between the instrumentality and its criterion, c, i. e., its validity coefficient, is: r‘:x = .50, the value of its reliability coefficient is: rxx = .75, and the instrumentality were to be tripled in length, what would be the estimated value of the validity coefficient for the lengthened form of the instrumentality, i. e. , what would be the estimated value of rcnx? Substituting values appropriately in formula (2), it is found that: 132 Ed. Sciences Manuscript J. E. Hill -5- r = 3(.50) = 1.50 = 1.50 _1.5o\. 55 ‘3” +3 -1)(.7 3+6 .75) [7.5 2'74 Tripling the length of the instrumentality, or averaging the results of three parallel forms of the instrumentality, or averaging the results of three administrations of the some instrumentality would increase the value of its validity coefficient from: r‘:x = .50, to rc3x§ .55. It should be noted that the effect of tripling the length of the instrumentality on its reliability would be from r = .75 to: xx 3.(75) 2.25 _2.25 .90 'aIII" Wit—1754 T175" 2.755 15" The increase in the reliability of the instrumentality which accompanies its increase in validity demonstrates the high degree of relationship which exists between these two factors. If it were necessary to determine how many times its original length on instrumentality had to be in order to attain a given level of validity, i. e., to attain a validity coefficient of a specified value, then formula (3) could be employed: _ cnx xx . (3) where r , r , r , and n are defined as in formula (2). To illustrate the use of formula (31, cnx cx xx suppose that the listening test associated with the measurement of the domain of theoretical auditory linguistic, T(AL), has a reliability coefficient of the value: rxx = .70, based upon the scores of 300 individuals. The value of its correlation with its criterion (a standardized listening test) is: rcx = .80. How many times its present length must the test be if the value of the validity coefficient for the lengthened form is to be: rcnx = .90? Employing formula (3), it is found th t: a |r‘.__( 90) :(1.- 70) _ .81(.30) ___ .243 ___ .243 ( 80) 2-(. 90) 2.( 70) “4301-715 .64-.567 . 1573-3 3 133 Ed. Sciences Manuscript J. E. Hill -6- or rounded upward: n = 4. This result indicates that the listening test must be 4 times its present length if the value of its validity coefficient is to be increased from .80 to .90. The value of the reliability coefficient for the lengthened test would become: nr XX l.rtn = l'l(n-1)rxx = 4(.70) = 2.8 =2.8~90 "fl! IRE-11(.705 1+§.l IT" ' The fact that the value of the validity coefficient for the lengthened test would be: rc3x = .90; and its reliability coefficient would be: r411= .90 is coincidental to the conditions of the problem. In other words, the value of the validity coefficient of a lengthened test is not necessarily equal to value of its reliability coefficient. This type of result depends upon the circumstances of the values of: r , r , of the original test, and the "desired" value for r cx xx cnx Construct Validifl Predictive, concurrent, content, and construct validity, respectively, have somewhat different purposes and goals. These different types of validity are needed because of the kinds of questions that are raised by the different situations in which measurement is used. Although these types of validity have different purposes, they are not entirely different logically. To be sure, in meaning and in their implications they show considerable concurrence. In a general sense, predictive, concurrent, and content validity are concerned with the question: To what extent does the in- strumentality measure the aspects (of the individual) it was designed to measure? On the other hand, construct validity deals with the question: What aspects are measured by the instrumen- tality? It is generally agreed among test constructors that predictive, concurrent, and content - __ 4_4+.___.f- -. '1 134 Ed. Sciences Manuscript J. E. Hill -7- validity are but special cases of construct validity. Therefore, construct validity can be ex- pressed in terms of the statistical descriptions of empirical relationships associated with predictive and/or concurrent validity, and in terms of the verbal iudgments and descriptions associated with content validity. Many of the elements of "style" are intellectual constructs rather than directly observable behaviors. These elements, in their respective theoretical frameworks, are considered to be related in various ways, and to various degrees, with other characteristics of the individual under consideration, and to have certain gff_e<_:_t_s_ mm the individual's behavior; while at the same time themselves being amenable to modification by certain types of treatments. Consequently, if an instrumentality is to provide valid measures of a "style" element, i. e., of a constructed trait, it must necessarily provide information that reflects the posited "relationships," the "effects," and "modifications." Obviously, the greater the agreement between the fished and the obtained relationships, effects, and modifications, and the greater the number of these agreements, the more valid the instrumentality yielding this information is considered to be. If the theoretical framework of an element of "style" is not sufficiently developed sq that expected relationships, effects, and modifications can be specifically detailed, then possible pertinent correlates, effects, and modifications must be probed by the instrumentality. Investi- gation of these "observable," possibly pertinent, relationships, effects and modifications by the instrumentality provides information that can be used to further specify the "element," and to perhaps broaden the scope of the nature of the characteristics considered to be involved. The determination of construct validity, then, is a dynamic and continuous process, i. e. , a program that employs information from a variety of sources. It is inductively inferential in nature rather than deductively conclusive. No M validity coefficient can demonstrate construct validity, nor can an analysis of the nature and content of the instrumentality involved provide 91! the - 51.5... L' 135 Ed. Sciences Manuscript J. E. Hill -3- needed information. A program of information composed of predictive, concurrent, and content validity descriptions, supplemented by results from experimental and other types of systematic investigations to ascertain whether the hypothesized relationships, modifications, and effects that occur could form the construct validity of an instrumentality designed to yield data pertaining to a given element of "style." Therefore, although it is mainly based upon obiective inform- ation and quantitative data, the construct validity of an instrumentality is essentially determined and evaluated‘by a process of subiective iudgment. It should also be noted that the degree of construct validity possessed by an instrumentality cannot be expressed by any single quantitative index such as a validity coefficient, but must be presented in verbal terms. The notion of construct validity can be illustrated by describing how one of the instrumental- ities designed to measure the domain of the cognitive style element, qualitative code synnoetics, Q(CS) - knowing oneself, was developed and validated. On the basis of the definition - "knowing oneself," it was agreed that the domain was a composite one with many possible sub-domains. (In the interest of time and space the discussion will be limited to the development of an instrumen- tality to sample the domain "synnoetics" in terms of performances demanding the use of certain motor skills.) A calibrated device called the "electric eel" was constructed. The “eel" was a piece of sheet metal, cut in the form of an irregular sine curve, with calibrations from 0 through 50 pointed at intervals from the "tail" to the "head" of the figure. The sheet metal forming the figure was electrically charged. The individual being "tested" was given an instrument in the form of a metal loop attached to a wooden handle. Whenever the metal loOp was in contact with the sheet metal, a bell—buzzer would be activated. 1 After the subiect was given one "trial run," that individual was asked to predict the highest score (calibrated score) that he or she believed he would attain in three test trials. After the 136 Ed. Sciences Manuscript J. E. Hill -9- first trial, the subiect was permitted to revise his original estimate. After the second trial, the subiect, once again, was permitted to revise his earlier estimate. The degree to which the predicted score was congruent with the performance score provided a first approximation to the amount of self-knowledge, regarding such matters, the individual possessed. When the nature and content of this type of performance test is examined, it is apparent that it does provide a means of assessing, in broad terms, the individual's self-knowledge of the level of the motor skills required by the instrumentality. As a further means of validating the measurement of the "synnoetics" element construct, it was hypothesized that scores yielded by this instrumentality should be related to performance on iobs where this "style" element is important. It was found, for a relatively small sample group of examinees (airline pilots), that their scores on the electric eel were not only high (and were accompanied by highly accurate self-assessments), but when paired appropriately with scores yielded by similar items included in one of the screening examinations for pilots used by the airline which employed them, the correlation was statistically significant. Anticipating that the degree of "synnoetics" an individual seemed to possess would have certain effects on other aspects of his behavior, it was hypothesized that individuals doing well on this type of performance test, i. e. , showing a "major" in "synnoetics," would perform similar tasks associated with their occupations at a higher level of competency, and enioy those tasks more, than would those individuals who showed less self-knowledge (minor, or negligible, orientation) regarding the types of tasks involved in the test. Based upon samples of nursing students, it was found that those with maior orientations in synnoetics, based upon the "eel" test, passed the unit regarding ministrations involving the hypodermic syringe at higher levels of achievement than did those students with minor, or negligible, orientations. Generally, the students with these "maiors" in "synnoetics" reported, more frequently, on 137 Ed. Sciences Manuscript J. E. Hill -1o- enioyment in performing the tasks associated with these instructional units, than did those students with "minors," or "negligibles," in qualitative code synnoetics. It was also hypothesized that practice (a treatment) should affect the performance of an individual on this instrumentality, thereby improving his self-assessments regarding the motor skills involved in the test. Allowing practice time showed clearly that the individual assessed his limitations much more accurately than he did prior to the practice session. Examining the relationships between the scores on "synnoetics" yielded by the electric eel test, and those associated with certain tests of qualitative proprioceptiveness (e. 9., the Bass test, the Nottus test) also helped to further specify and clarify the nature of the "synnoetics" elements. In a somewhat different dimension, the effect of qualitative code empathetic, Q(CEM), on that of qualitative code synnoetics, also provided helpful information. Some persons showing a maior orientation in qualitative code empathetic, when confronted with a problem calling on that aspect of empathy that pertains to sympathy, demonstrate clearly that this influence tends to reduce their capability in the realm of self-knowledge, i. e., synnoetics. For example, many persons who cannot swim, when given a choice of saving a drowning child by going to the rescue (with full knowledge that this choice means that the child and person will both drown), or by trying to find a swimmer to save the child, under the constraint that the probability is high that the child will drown before help can be found, will choose the former and put his or her faith in providence. The purpose of the example was to illustrate the program 9! constnrct validity which was employed, and is continuing to be employed in the development of instrumentalities to sample the domain qualitative code synnoetics. The inductive inferential nature (as opposed to con- clusive information) of construct validity should also be evident from the example. 138 Ed. Sciences Manuscript J. E. Hill -1]- Concluding Remarks Validity and reliability are, in a sense, measures of the efficiency of an instrument. Re- liability measures the consistency of the results yielded by the instmmentality, and is based upon information yielded by that instrumentality itself. Validity, implies evaluation of the instrumen- tality in terms of external and independent criteria. One of the greatest difficulties associated with the validation of an instrumentality is that of determining authentic criteria. Of necessity, criteria must be approximate and indirect, because if authentic criteria were readily available the measurements they provide could be used instead of those yielded by the instrumentality under consideration. In order to be valid, on instrumentality must be reliable. Thus, a highly reliable instrumen- tality is always a valid measuring device of some aspect or function. For example, if the value of the reliability coefficient of an instrumentality is: rxx = .90, then the value of its index of reliability is- $.76 = .95. This result means that the instrumentality correlates .95 with true measures of the domain it samples (or of itself, if the "true score, error score"concept is employed), where these true measures constitute the criterion. In this context, on instrumentality may be theoretically valid, but show little or no correlation with any other criteria. It is in consider- ation of such situations, that the construct validity of an instrumentality must be viewed as a continuous process involving a wide variety of possible criteria and their accompanying statistical O and verbal descriptions . 139 Ed. Sciences Manuscript J. E. Hill APPENDIX A PROBLEM SET ON OBJECTIVITY OF PROCESS OF MAPPING "STYLES" In the chapter on mapping it was noted that by the very nature of the information with which the "mapper" of a "style" must deal, in addition to the variability accruing to his own cognitive style, i. e. , the way he views the world, the process of mapping must be relatively subiective. It was also noted that in order to reduce this element of subiectivity, the three Principles for determining maior, minor, and negligible orientations of "style" elements, re- spectively, must be employed throughout the mapping process. In addition to these constraints, data from standardized tests, inventories, and scales should be employed with the Principles in the mapping process. It was further noted that when standardized instruments were not avail- able and locally-constructed instruments had to be employed, certain principles pertaining to: the obiectivity of test items, the obiectivity of questionnaires, the obiectivity of rating scales, and the objectivity of observational methods need to be employed to help reduce the subiectivity of the mapping process. Although obiectivity is given a high priority in the mapping process, the mappings employed in the Educational Sciences are not intended to be impersonal diagnoses. To the control-y, trans- actions between the subiect and the diagnostician are encouraged so that the resulting mappings of measurements and data yielded by the observations involved can become more valid, reliable, and meaningful than they otherwise might be. In consideration of the fact that the diagnostician, and therefore his mappings, might well be suspect not only in terms of their reliability and validity, but in regard to their obiectivity and discriminative power as well, it becomes important to be aware of four main sources of error that tend to be Operating during the processes of diagnosing and mapping, respectively. Without regard to ranking in importance to these processes, the sources of error in question are: 140 Ed. Sciences Manuscript J. E. Hill -2- Conditions-Time Bias Error. Many behaviors which should be covered by the set of test measurements and/or observations of the individual under consideration may be inadequately sampled (or even omitted), because they occurred too infrequently (or not at all) during the period of observation, or during the testing situation. This source of error can be minimized by increasing the number of "sampling situations," and including a variety of conditions under which observations can be made. In addition to these provisions, lengthening the interval of the observational periods has also proven to be beneficial. Classification-Bias Error. This source of error is a function of the diagnostician's tendency to interpret several relatively similar behaviors and certain results from mea- suring instruments in terms of a particular profile of "style" elements, thus assigning a disproprotionate weighting to that aspect of the individual's map. For example, if an individual is observed to behave somewhat different from the "usual" mode of behavior of subiects being observed in a given situation, some "mappers" place increased weight- ings on the "D" (difference) modality of inference, while the weighting might better , be assigned to the subiect's "individuality" (I). In any event, this type of error can be minimized by the diagnostician following a pattern of accommodation, and a "wait and see" attitude, while continuing to consider all the possible profiles. of elements that might "explain" the behavior. Role Expectation-Bias Error. Behaviors which certain individuals tend to display during periods of observation and testing are those which they perceive as expected of them at those times. Role expectation errors can be minimized by adiusting when and where observations are made, and by adiusting the setting of the testing period and the type ‘1 141 Ed. Sciences Manuscript J. E. Hill -3- of instrument to be used. Of paramount importance to the reduction of this type of error is the rapport which the diagnostician can establish with this type of individual. Cognitive Style-Bias Error. This source of error includes a wide variety of possible response sets on the part of the diagnostician, such as the "halo effect," or at the opposite extreme the "cognitive dissonance effect." The reliability, validity, obiect- ivity, and discriminatory powers of a mapping process may be seriously affected by an intersection (e. 9., maior match) of the respective cognitive styles of the diagnos— tician and the individual he is considering. Under these circumstances, the behaviors of the individual might well fulfill the expectancies imposed upon them by the cognitive style of the diagnostician, and a halo effect on the mapping process could result. In similar fashion, but at the apposite extreme, a disiunction (e. 9., negligible match, or "low minor") of the "styles" of the diagnostician and the individual, respectively, could lead to each person witnessing cognitive dissonance regarding the other, and the map of the individual's "style" being distorted accordingly. The effect of cognitive style bias can be minimized by the diagnostician beihg acutely aware of his own "style," and adiusting his mapping processes accordingly, or even disqualifying himself for the mapping of the individual under consideration. In the field of psychoanalysis this type of bias is dealt with by the psychiatrist undergoing analysis and being completely informed of the results prior to being allowed to practice. In some cases, psychoanalysts have disqualified themselves for the treatment of certain patients because of their knowledge of themselves, and the probable existence of certain factors with which they might not be able to deal obiectively. In many ways the analysis of diagnosticians, and their processes of mapping, is analogous 142 Ed. Sciences Manuscript J. E. Hill -4- to attempts to determine the accuracy of his social perceptions, sensitivity to others, self- assessment, and other similar types of factors. It is generally. agreed that most such attempts have not yielded valid results. Recent developments in the study of these factors have led to the examination of the access involved instead of the accomplishments evidenced by indi- viduals in these dimensions. These types of approaches are consistent with those employed in the examination of the mapping goes in the Educational Sciences, instead of an overt con-l centration on the accomplishments of the diagnostician regarding these factors. Obiectivity of Diagnosticians. The process of mapping should yield verifiable, reproducible information that is more than the idiosyncratic "mappings" of a given diagnostician. The process of mapping, nonetheless, essentially depends upon a diagnostician (or team of diagnosticians) exercising iudgments pertaining to the appropriate orientations of elements of "style" for an individual based upon the results of observations made of that individual under certain conditions, or measurements of his abilities yielded by appropriate tests, scales, or inventorying devices. Team of Two Diagnosticians. A highly acceptable method for determining the obiectivity of a given diagnostician is to compare the orientations of the elements of "style" he maps for a given individual for agreements and disagreements with those mapped for that individual by an experienced diagnostician, where the orientations mapped by both diagnosticians are based upon the same set of data and observation periods. Under these circumstances, a measure of the obiectivity of the diagnostician and his process of mapping could be the percent of agree- ment between the orientations of the elements of "style" mapped for the individual by the novice and the experienced diagnostician, respectively. Following a method described by Bernstein,*let Px be defined as the probability that the *See Allen L. Bernstein, "An Estimate of the Accuracy (obiectivity) of Nominal Category Coding," Michigan-Ohio Regional Educational Laboratory (MOREL) Monograph Series Number One, Detroit, Michigan, October, 1968. 143 Ed. Sciences Manuscript J. E. Hill -5- novice diagnostician will map the orientations of the elements of an individual's style " correctly." Let PY be defined as the probability that the exerienced diagnostician will map the orientations of these elements of style "correctly." The necessary assumptions that Px and Py are constant and independent, and that the narnber of categories (orientations) to be considered is constant, i. e., a "maior," or a "minor," or a "negligible" orientation for each of the elements to be mapped are the only "categories" to be considered, can be satisfied. Under these circumstances, the question becomes: "From the set of paired elements (paired "orientations of the elements mapped by the novice and experienced diagnosticians, respectively). how can the estimate of the value of Px and that of P), be calculated ?" By definition: and A denotes the ratio (percent) of agreement in the set of paired elements yielded by matching the mappings of orientations of "style" elements mapped by M diagnostician X with those mapped by the experienced diagnostician Y. . If “’1' P2, P3, . . . ., Pn) is defined as the set of probabilities associated with the set of orientations of "style" elements (1, 2, 3, . .. ., n) mapped by each diagnostician, then estimates (pl, p2, p3, . . . ., pn) of (P1’ P2, P3, . . . ., Pn) can be used to define the probabilities of the n elements under consideration.* For example, suppose diagnostician X mapped orientations of five elements of "style" for a given individual, with the following probabilities being deter- mined by the percent of agreement between his "mappings" and previously known information“ *It mould be noted that the capital letters (P1, P2, P3, . . . ., Pn) denote rameters of the dichotomous population of "successful classification" (mapping), and "unsuccessful classifi- cation" of orientations of "style" elements; while the lower case letters (p1, p2, p3, . . . . , pn) indicate estimates, i. e., statistics, of their counterpart parameters, derived from samples, or "guesstimates . " "Suppose the diagnostician is asked to map elements of "style" for the map of an individual whose stvle has alreadv been manned and is well-verified. ‘ ‘ ' 144 Ed. Sciences Manuscript J. E. Hill --6- regarding these "elements," pI = .72, p2 = .67, p3 = .77, p4 = .80, and p5 = .85; then these values can be uwd as estimates of (PI , P2, . . . . , P5) for n = 5 elements of "style." The probabilities associated with the possible results for diagnostician X and Y, respectively, can be found by means of the following expression: (l’x + Qx) (PY +Qy) = PxPy + pry + Pny + Qny (1) An analysis of the interpretations of the products on the right side of equation (I) would show: Nature of Agreements Outcomes Probability and Disagreements Diagnostician X and Diag. Y "correct" PxPy X and Y agree Diag. X "correct," Diag. Y "incorrect" Pny X and Y disagree Diag. X "incorrect," Diag. Y "correct" QxPy X and Y disagree Diag. X and Diag.Y "incorrect" Q Q X and Y agree on the x y same "incorrect" element, or X and Y disagree, but each maps an "incorrect" element Now, if A = ratio (percent) of agreement in the set of elements yielded by matching the "mappings" of "novice" X with those of diagnostician Y, then: A = PxPy + QnyK Where K denotes the fraction of the events in the set associated with the probability represented by the product: Qny, i. e., the probability indicating that diagnostician Xand diagnostician Y have selected the same incorrect "element" in their respective mappings. For example, when: P =.7,andQ =I-P =1—.7=.3, X X X P =.8,andQ =1-P =_1-.8=.2, y y y then: A = (.7)(.8) + (.3)(.2)K, ize‘fi—ik A.-_‘-Mh-.d———‘ 145 Ed. Sciences Manuscript J. E. Hill -7- or: A = .56 + .06K, is the value of the ratio (percent) of agreement in the set of paired "elements" of the respective diagnosticians' mappings. Bernstein has shown that the value of K can be estimated in a variety of ways, and has provided the following table of values of P for different values of A and K.‘ I - TABLE OF VALUES OF P, FOR VALUES OF K AND A .A K== I .5 .3 0 .95 .974 .975 .975 .975 .90 .947 ' .948 .949 .949 .85 .919 .920 .921 . .922 .80 .888 .891 .893 .894 .75 .854 .860 .865 .866 .70 .816 .828 .832 .836 .65 .774 .793 .799 .807 .60 .724 .755 .764 .775 *Values in the Table are found by a formula derived as follows: A = PxPy + QnyK' can be written as: A = PxPy + (l - Px)(l - PY)K. When the two diagnosticians X and Y are considered to be properly trained and approximately equally experienced, it is reasonable to assume: Px = P), = P. If this assumption is true, then A becomes: A = P2 + (l°P)2K; or: P2 + K - 2PK + 1‘./KZ )- P2K = A. Solving this quadratic equation for the value of P yields: P = K 1 1+: K . Be- cause with trained diagnosticians, the values ofA should be 2 1/2, and P should be >1/2, the smaller of the two quadratic roots is excluded. Using the larger root, P becomes: 1>= K+./’TT'T—A +1< -1< 1+K 31' 146 Ed. Sciences Manuscript J. E. Hill .3- It should be noted that the extreme values, K = 1 and K = 0, yield values of P a little dif- ferent from each other until the value of A is as low as .70. Even in this case, the difference is but: .836 - .816 = .02. Under these circumstances, it is a reasonably safe procedure to choose K = 0, and estimate the value of P by the formula: p = .G‘ Team of Three Diagnosticians. In general, it is desirable to use a third experienced 1 Ti diagnostician, thereby forming a "team" approach to mapping, whenever possible. Under :--, these circumstances, the assumption that: P)( = Py = P, is not necessary. With the addition of a third experienced diagnostician 2, it is possible to provide three sets of paired mapped ', .4 orientations of elements of "style," one associated with X and Y, one with X and Z, and one 1 with Y and Z. The probability set associated with the possible independent outcomes for three diagnosticians can be found from the equation: ’W‘tQ)(P+Q)(P+Q)=PPyP +PXPQ WQPtQPP WPQQKQPQKDQPt-QQQ =1 (2) xy e x y e If it is arbitrarily assumed that agreement between the orientations of a given "style" element mapped by any two of the three diagnosticians constitutes a "correct" mapping of that element, then the sum of the first four terms of the right side of equation (2) is the probability that the re- sults of the given observation or measurement will be mapped into the "correct" orientation of the "style" element under consideration. ’ In a fashion similar to the case of mdiagnosticians, it is possible for flee diagnosticians to agree on (or map) "incorrect" mappings, and therefore a factor K must be applied to all -_ terms of the equation m the fi__rs_t £921. If we employ the value: K = l, the percent of agreement between the diagnosticians will become 1.00 (or 100%), which is a highly unlikely event for a set of mappings. If it is assumed, however, that K = 0, and Px - Py = P2, then the following results calculated by formulas (1) and (2), respectively, occur as shown in the Ed . Sciences Manuscript table below: 147 -9- J. E. Hill 11 - TABLE OF VALUES FOR A, FOR VALUES or P, AND K = o Problem of Agreement of Three Diag. under Problem of Agreement of two Assumption that Agreement of two Diag.'s P Diag. on "Correct" Mapping Mappings Determine "Correct" Mapping .90 A=PP =P2=(.90)2=.81 A=P.PP+PPQ+PQP+QPP =P3-3PzQ= ‘1 xy xyzxyzxyz xyz 1 .903 - 3(.90)2(. 10) = .972 .80 .64 .896 l . .70 .49 .784 l , .60 , .36 .648 Example 1. (Three Diagnosticians). If M is defined as the percent of "correct" mappings, under the assumption that agreement by at least two of the diagnosticians on a mapping determines a "correct" mapping, and if the condition K = 0 is assumed to exist, then: M=PPP +PPQ +PQP +QPP xyx xyz xya xya Since this is a single equation involving three variables, one of its solutions will occur uhder the condition of: Px = Py = P2. Assuming this condition exists, the following cubic equation results: M =93 - 3920 This equation is the same as the one employed in the "Table of Values for A." (a) Suppose that three trained diagnosticians are to employ the data yielded by their ob- servations of an individual in seven different settings to map the "orientations" (a "maior, " or a "minor," or a "negligible") for each of five different "style" elements. Assuming that the "correct" mapping of an orientation of a style element is defined as that orientation of the given 148 Ed. Sciences Manuscript J. E. Hill -10- element that is mapped by at least two of the diagnosticians, and assuming that the probability of mapping the "correct" orientation of a given element for each of the diagnosticians is the same, i. e., Px = Py = P: = P, and that the value of that probability is: P = .83, find the probability (obiectivity index) that all three diagnosticians (the total team) will map the orien- tation of that element correctly. Solution. Employing the equation: M = P3 - 3920, the probability of all three diagnosticians mapping the "correct" orientation would be: M = (.83)2 - 3(.83)2(.17)§‘:.572 + .351'1‘.923. (b) Suppose the same conditions exist as in (a), but P = .90. Solution. Reading from the second column of Table II in the row where P = .90, the probability of all three diagnosticians mapping the " correct" orientation would be: M = .972. Example 2. If the M of agreement on correctly mapped orientations of style elements between diagnosticians X and Y is defined as A, that between X and Z is defined as B, and the "percen_t giagreement" between Y and Z is defined as C, then the following three equations may be written: PP =A;PP =B;andPP =C. xy xz yz Multiplying the expression for A by the one for B, it is found that: 929? =A8, xyz or: 2 AB A 571715-525: Y! W In similar fashion, it is found that: 149 Ed. Sciences Manuscript J. E. Hill -11- These expressions permit the substitution of an estimate of the value of PX, Py' and P2, re- spectively, under the assumption that K = 0. Suppose that the percent of agreement between X and Y on "correctly" mapped orientations of "style" elements, included in a pilot study sample of "mappings," is found to be: .80. Then, the estimated value of A would become: a = .80. In similar fashion, suppose the percent g_f agreement between X and Z is estimated to be: b = .75; and, finally, the "agreement" between Y and Z is estimated as: c = .70.* 2_(.80)(.75) . 2___(.80)(.70) . 2=(.75)(.70) _ Px “___—.70 'Py. 7—5—.7 ”“5 P; ___—.80 ' . ____ (.80(.5 . = .80).70). = .75(.70) . °'- 1’. JT '1’, “.75 ”NP. T' Then: and: px = J .8571 ; py = \/ .7467 ; and p2 = 7.65625; or: $3.925; py\\.864; and p}.810. These results indicate that diagnostician X would have the highest probability (pf-.925), and therefore would probably be the most objective of the three diagnosticians, in mapping the "correct" orientation of a "style" element based upon a given set of data, diagnostician Y the next highest probability (py\\ .864), and objectivity, and diagnostician Z the lowest probability (p21). .810), and the least objective, of the three. It should be noted, however, that since the three probabilities range in value from an estimated p = .810 to p = .925, the objectivity of a mapping of an orientation of the "style" element under consideration by any of these three diagnosticians would be rated as, "relatively high." *It ghould be noted that a, b, and c, respectively, are statistics from the pilot study sample whose values estimate those of their counterpart parameters A, B, and C which are characteristics of the population from which the sample was drawn. 150 Ed. Sciences Manuscript J. E. Hill -12- These methods provide a good approach to determining the objectivity of diagnosticians engaged in the process of mapping orientations of "style" elements associated with the Educational Sciences. It should be emphasized at this point that the use of three diagnosticians tends to im- prove the process cg estimating t_h£ objectivity elf diagnosticians. In addition to this advantage, the three member team approach to mapping also increases the probability of increasing the reliability, validity, and discriminative power of the process. _l- ~ e '- b __44'1 APPENDIX B "‘ ' .‘1 V-aaxl mZOHMH>HG ON dammmmUUDMZD ID ZNwSBNQBH I H JDmmmNUUDm t m e , 151 APPENDIX B _iN Fro mozmm mOZ-Qm m02§ Ao~HmOHHoz< Ao>vh m>HH A4m0kHo3< H hum . onhamamhmHn >uzm=ommu V rl‘fi'lu‘hle‘: L‘- .r.’ll.'1t P x 8 _umonmmza om .Summuuunmzzua zmmzhmmzHua Jaummmuunmum X 1552 NN¢MFFNF¢NNF w av » F a a - -> ». m-omz>m - hum - zo-ezm-eem-o >uzm=omme 153 *- «83 .. monwH-fia ow Nana? EU:- ADhmmmUUDWZD ID X. mOHCZ we 318191193N WDZQK ZmWZBNQZH t H up ¢Oh¢2 mm QCFOB «mum .N~1_ mwzmm >au<fi umzhu mu mmoou m>Hhuzmaoumu moHHmozz>m .I‘lu‘ . . .‘ AbmmmmUODm t m N 3 9 all I 9 I 8 1 3 154 $3:- ROOH I mZOHmH>HQ ON mm mm mwz92m: mam ZmuzammZH H .anmmmmuupm 1 m 155 woo—umonmH>Ho ON .% ‘4 Nathan-1.2.. . r E T..-”1111A hit-INN.” .zmazmmz- 1- 25883 - m 8 mm 2: .x . H: a 2 «one: HH HN CH— .6sz a? .u «M .2909 N NNIHN H N m Nu m . e m m HH m m m on m m m m- m H o N e H 2 w m N H Wu. H N H HH 3 e H oH H H H m N m H e . m 1.1 H. _ Z _ H H _ To . a H .w l @0243 Can—HHS. HHuHc -uzm: mm... .1. H r ... infin-Inr. r illllllrlt r. i..lt1ut h H2:.s.monmH>Ha om aammwmuupmzp 1b zuuseunzu 1 H .apmmmuuapm 1.m «a en en mange; _a mosz_ mq NM film-HOB H m .H om-m~ H m m 8.5“ 15v. .w. N e~-m~ oH euumm IMMANI om e mH-mH HHue H mH-eH mH-~H 1HH-oH m-w lmql m4 Mrw ml Arc HOPVN N"1 m _UONIH 1.565 HTGIDITBEN 318191193N 318191193N ._ H D H m mwz-me DOM—g Mr .H <.H. >«H4H <9 m>HHhH4<=oH>HozH mHzHHzHnH ON >.H .H (H. .. .r all}! iitllrrlrr -slsa-tlfs.l1 1; .HDnHmmMUODmZD ID ammammZH 1.. H .HDmmmflUODm I w «a . . xv <2 N 11. DJ 11 I DJ I 8 ..l 3 3253qu .5 mmH: .Eooz HHH hum 1 onhamHmhmHo >uzmzcmmm MICH INllfllllflljlflljjjjfljjjijjfljfljflmflafim.s