l {Michigan fin—v, r1; aw?"- .,,;( it. .: L?§:.H 3A ’af 1 Univemi i}; ._._‘.___.,.,_. .4‘___ A PLACE IN RETURN BOX to remove thls checkout from your record. TO AVOID FINES return on or before date due. DATE DUE DATE DUE DATE DUE (v i995 3 PER] 2 in)!” EEC S '1 W6 FEB 1 4 2010 ‘ 4.: _ 72' 03 u 9 N 'w 09:1? 1993 Wan 3 2m MSU Is An Affirmative Action/Equal Opportunity Institution cm ”3'9 .1 A MODEL FOR EVALUATING AND CHOOSING AMONG WIDELY USED ASSESSMENT INVENTORIES OF COGNITIVE/LEARNING STYLE: AN EXPLORATORY DELPHI STUDY By Diane Genshaw VOLUME IL A DISSEERTATION Submitted to Michigan State University in partial fquiIlment of the requirements for the degree of DOCTOR OF PHILOSOPHY Department of Teacher Education 1991 ABSTRACT A MODEL FOR EVALUATING AND CHOOSING AMONG WIDELY USED ASSESSMENT INVENTORIES OF COGNITIVE/LEARNING STYLE: AN EXPLORATORY DELPHI STUDY BY Diane Genshaw The purpose of this research is to add to the existing body of knowledge about learning/cognitive style assessment inventories. Very little research has been done in which inventories of cognitive/learning style have been evaluated, compared, and contrasted. Practitioners often have a difficult time choosing among the different inventories of cognitive/learning style when selecting an inventory that best meets their instructional needs. The methodology of the study was a Delphi technique utilizing descriptive statistics to analyze the findings. Three Delphi rounds were sent to 41 experts in the field of cognitive/learning style. After each round data were analyzed and sent back to the experts in order for them to arrive at consensus. The results of the study indicated that: 1. Experts were able to identify 30 published inventories which they perceived as being widely used. 2. Experts were able to identify and rank a group of elements that they perceived as important elements to differentiate effectively and efficiently among inventories of cognitive/learning style. Experts were able to identify and rank in order of importance 42 elements or characteristics of cognitive/learning style inventories. Experts were able to identify and rank reasons for using cognitive/learning style inventories and their disappointments with cognitive/learning style inventories. The Delphi methodology is an appropriate technique for this type of study. There is some confusion about the definition of cognitive/learning style, and how cognitive/learning style should be measured. A model has been built, using the data from the experts, that will aid practitioners in selecting a cognitive/learning style inventory that could best suit their instructional needs. Copyright by Diane Genshaw 1991 DEDICATION This dissertation is dedicated to my son: IHARC GENSHAW Who has taught me that to dream a dream is more than half way to achieving a reality II ACKNOWLEDGEMENTS I wish to express my sincere respect and appreciation to Dr. Peggy Riethmiller, chairperson of my doctoral committee. Dr. Riethmiller provided valuable advice, direction, and assistance. She generously gave of time to help me complete this project on schedule. Also I wish to thank Dr. Castelle Gentry. He gave unstintingly of his time and expertise to help me run the Delphi Survey. I wish to thank Dr. Ben Bohnhorst and Dr. Casmer Heilman for their suggestions to improve and strengthen this dissertation. Without the strong support of my committee, this dissertation would have never been completed. I extend my grateful appreciation and love to my family and friends for their constant support and assistance. I especially want to thank Karlene Rabidoux who helped to proofread and edit this dissertation, and provided invaluable encouragement during crucial periods of this project. Additionally, I want to thank Ann Harlan who listened when I needed a friend and made valuable suggestions. I wish to thank my mother and father who started me on the road to my education, and believed that I could accomplish this task. Finally, and most importantly, I wish to thank my son who has been with me through all the ups and downs of writing this dissertation. vi TABLE OF CONTENTS List of Tables .... ...... . ............................. x LiSt Of Figures ......OOOIOOO ....... O OOOOOOOOOOOOOOOOOO Xii CIIAMER 1: INTRODUCTION 0 O O O O O O O O O O O O O O O O O O O O O ........ 1 Statement of the Problem ..... . ................. 2 Elaboration of the Purpose of the Study . ....... .. 4 Need for the Study ....... ........ ....... ....... .. 5 Relevance of the Study to the Field of ........... 7 Education Theoretical Base for the Study ....... ...... ...... 10 Research Questions ................ ............. .. 15 Primary Research Question ................... 15 subqueStionS 0..........OOOOOO...0.0.0.000... 15 Limitations of the Study ..... .... ..... . ......... . 16 Organization of the Study ... ................... .. 17 summary 0 O O ..... O OOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO 18 CHAPTER 2: REVIEW OF THE LITERATURE .. ..... . ......... . 20 IntrOduction O O O O O O O O O O O O O O O O O O O O O O O O O I O ........ O O 20 The Delphi TeChnique O O C O O I O O O O O O O O O I O O O O O O O O O O O O O 2 0 Definition. 0 O O O O O O O O O O O O O O O O O O O O O O 0000000000 O 21 History, Purposes, and Philosophy ........... 22 The Process of the Delphi Technique ......... 27 Format of the Initial Round ................. 29 Disadvantages, Limitations and Problems of .. 31 of Using the Delphi Technique Advantages and Applications of Using the .... 33 Delphi Technique Elements of Cognitive/Learning Style and the .... 36 Inventories That Measure Cognitive/Learning Style Definitions of Cognitive/Learning Style ..... 35 Cognitive/Learning Style .. ............. ..... 38 Problems and Issues Addressed in the . ....... 41 Literature Concerning Cognitive/Learning Styles Cognitive/Learning Style Assessment . ..... ... 42 Inventories Summary ......................... ........ . ...... .. 52 CHAPTER 3: DESIGN OF THE STUDY ....................... 53 Outline of Research Design ....................... 53 Research Questions ........................ ....... 54 vii Primary Research Question ... ............... . Subquestions ................... ........ ..... Population and Sample .......... ........ . ....... .. Description of Population ................... Sampling Procedure .......................... Instrumentation ..................... ........ ..... Reliability and Validity of Instruments ..... Design ........................................... Round 1 Delphi .............. ..... ........... Round 2 Delphi .............................. Round 3 Delphi .............................. Data Collection Procedures ....................... Resources Required .......................... Training Required ........................... Plan for Analyzing Data .......................... Statistical Treatment of Data ............... Decisions Rules for Interpreting Data ....... Presentational Format Used to Present ....... Findings summary O.......OOOOOOOOOOOOOOOOOOO0.00.0.0....... CHAPTER 4: DATA ANALYSIS AND FINDINGS ................ Data Collection and Analysis ... .................. Data Collection.............................. Data Analysis .......... ......... ............ and Findings by Question .... ..... .. Data Analysis Question Question Question Question Question 1 2 3 4 5 Summary of Findings Across All Questions ......... Question 1 Question Question Question Question 2 3 4 5 summary 0.00.0000.........OOOOOOOOO0.00.00...0.... CHAPTER 5: SUMMARY, CONCLUSIONS, RECOMMENDATIONS, ..... AND REFLECTIONS Introduction ......................... ........... . summary Of the Study ...OOOOOOOOOOOOOOOOOOO0...... Results, Conclusions, and Recommendations by ..... Question QueStionloooooooooooooooooooooooooooooooooo Question 2. Question 3 .................................. Question 4 viii 54 55 57 57 62 65 65 66 66 68 7O 71 71 72 73 73 76 76 76 79 79 79 80 81 81 96 101 117 125 141 141 144 144 148 155 164 169 169 169 172 172 175 177 179 questions0.00.00.........OOOOOOOOO0.0.0.... 181 A Model for Evaluation and Choosing Among .........187 Widely Used Assessment Inventories of Cognitive/Learning Style The Flow Chart .............................. 225 The Matrices ................................ 225 The Inventory Information Sheet and the ..... 228 Definition Information Sheet Opinions, Conclusions, and Reflections of the ... 229 Experts Reflections ..................... ................ . 241 Summary .......................................... 245 Appendix A ........ .................................... 248 Appendix B ............................................ 255 Appendix C ................... ..... .......... ......... . 271 Bibliography ................................ ........ .. 284 ix Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table 10 11 12 13 14 15 LIST OF TABLES Summary of Experts' Demographics and . ....... 59 Response Rates Results of Part Round 1: Widely Used ........ 83 Cognitive/Learning Style Inventories Usage and Familiarization by the Experts ... 86 of Widely Used Inventories of Cognitive/ Learning Style Most Frequently Used Inventories of . ........ 90 Cognitive/Learning Style by the Experts Ranked Inventories as Classified by Age ..... 92 Groups Response Rates for Round 2 by Block ......... 103 Points and Ranks for Subelements of ........ . 104 Cognitive/Learning Style Ranking of Cognitive and Technical .......... 114 Elements Convergence to Consensus: Cognitive and ..... 115 Technical Elements Ranked List of Characteristics of .. ......... 121 Cognitive/Learning Style Inventories Ranked List of Reasons to Use ............... 132 Cognitive/Learning Style Inventories Ranked List of Disappointments With ......... 136 Cognitive/Learning Style Inventories Rankings of Categories and Elements of ...... 145 Cognitive/Learning Style Inventories Agreement Across all Elements in Round 3 .... 149 Convergence to Consensus of Cognitive/Learning Style Inventories Percentage Mean, Median, and Mode of ........ 149 Agreement Across All Elements in Round 3 Convergence to Consensus X Table Table Table Table Table Table 16 17 19 20 21 InterQuartile Ranges and Ranks of .......... 151 Characteristics of Cognitive/Learning Style Inventories Interquartile Ranges and Ranks of Reasons ... 156 to Use Cognitive/Learning Style Inventories Interquartile Ranges and Ranks of ........... 160 Disappointments With Cognitive/Learning Style Inventories Convergence to Consensus Round 3 for ........ 165 Reasons to Use and Disappointments With Inventories of Cognitive/Learning Style A Model For Evaluating and Choosing ......... 189 Among Inventories of Cognitive/Learning Style Experts' Ranks and Comments On Elements ..... 231 in Cognitive/Learning Style Inventories xi Figure 1 Figure 2 Figure 3 LIST OF FIGURES Information Processing Model by ............. 8 Charles Letteri Use of the Inventories by Age Ranges ........ 97 Flow Chart for Choosing an Inventory ........ 188 CHAPTER 1 INTRODUCTION The purpose of this dissertation was to add to the existing body of knowledge on cognitive/learning style assessment inventories. Additionally, using results from a Delphi Study conducted with experts in the field of cognitive/learning style assessment inventories, a model was built to aid practitioners in selecting an inventory to meet their instructional needs. How students learn has been an educational question for as long as there have been teachers and learners. Every great educator has recognized and tried, to some degree, to accommodate differences in learning. Aristotle used association and mnemonic devices. Plato used dialogue with questions and answers to stimulate the learning process. The Greeks recognized differences in learners by classifying personalities as sanguine, choleric, melancholic and phlegmatic (Cornett, 1983, p. 7). More recently Jerome Bruner pondered if there may be innate ways of organizing material. Bruner asked the basic questions of education "...What shall be taught, when and how?"(Bruner, 1960, p. 2). Bruner also felt that schooling should help each student to achieve their optimum intellectual development. This is not to say that the pace or the content of courses need be identical for all students...Careful 1 .zv arc. "tt“ n:.‘ ...§| 2 investigation and research can tell us wherein differences must be introduced. One thing seems clear: if all students are helped to the full utilization of their intellectual powers, we will have a better chance surviving as a democracy in an age of enormous technological and social complexity (Bruner, 1960, p. 9-10). Gagne stated the obvious when he said, "The central purpose of any program of education is to promote learning" (Gagne, 1975, p. 1). He stated that, in order to promote learning, the teacher must be the manager of education and must effectively deliver instruction to the student by using any medium (oral, written, etc.) in order to arrange conditions so every student learns what he is intended to learn (Gagne, 1975, p. 4). One way to optimize learning is to adjust instruction to match learners' cognitive/learning styles. Teachers could measure students' styles by using cognitive/learning style assessment inventories. They are considered by many to be powerful tools in aid of the instructional process. This research investigated widely used cognitive/learning style assessment inventories. Review of the literature, expert opinion using the Delphi Technique, and surveying the actual inventories were the methods used in this investigation. STATEMENT OF THE PROBLEM The problem this researcher addressed was the difficulties which practitioners have in making informed ichoices among the many assessment inventories of 3 cognitive/learning style available. In this study, cognitive/learning style was defined as: "The characteristic cognitive, affective, and physiological behaviors that serve as relatively stable indicators of how learners perceive, interact with, and respond to the learning environments" (NASSP, 1979, p. 4). In this research, the term cognitive/learning or learning/cognitive style was used because practitioners and experts in the field use one or both terms, and there is very little consensus as to what is the best term to describe the phenomenon. Although distinctions have been made among learning, cognitive, and educational styles, in this research, these terms are used interchangeably. Experts, and others using the inventories, tend to use the term with which they are most familiar. In 1981 the NASSP and St. John's University of New York cosponsored a conference for practitioners and scholars interested in cognitive style, learning style and brain behavior. James Keefe, writing about this conference, referred to the area they were studying as cognitive/learning style (Keefe, 1988, p. 2). The task force formed as the result of this conference reviewed 40 broad elements of cognitive/learning style and brain behavior. From the initial list of 40 elements, 20 were chosen to be included in an assessment instrument which was developed by the NASSP group titled Learning Style Profile. The preceding NASSP definition was also developed at that time. 4 Although Keefe used the term cognitive/learning style to describe what was being reviewed in the 1981 conference, he and the NASSP group that developed the assessment instrument chose to use the term learning style to describe the instrument. In this research, the broader term cognitive/learning style assessment inventory will be used. In this way the duality of terms used by practitioners and researchers will be addressed. ELABORATION OF THE PURPOSE OF THE STUDY Very little research has been done which has compared, contrasted and evaluated information about the different assessment inventories of cognitive/learning style. In order to bring some order, clarity, definition, and unity to the area of style assessment inventories it is this researcher's aim to bring together information about several of the most widely used instruments. Although models have been developed to clarify the construct of cognitive/learning style, only very limited models have been developed which evaluated assessment inventories of style. The goal of this research is to give the practitioner a practical tool to use to select a specific cognitive/learning style assessment tool for a specific need. Examples of some different uses for cognitive/learning assessment inventories are aiding a teacher in selecting 5 materials that would most effectively present a reading unit to learning disabled students, giving incoming university students insights about how they learn best so they could adjust their own learning experiences, and aiding an administrator to determine supplemental materials that might benefit the majority of the students in a district to learn more effectively. It is likely that the same practitioner might select different inventories for each one of these uses. To reiterate, the aim of this research is to provide a means to assist the practitioner in this selection process. This study was designed to evaluate several popular cognitive/learning style assessment inventories, as well as provide a model which practitioners could use to guide their evaluation of style assessment inventories. NEED FOR THE STUDY Research in this area is needed because assessment instruments of cognitive/learning style are very different. They vary in content, purpose, and how results are measured and evaluated. One of the reasons there has been so much confusion is that there have been two major lines of research into cognitive/learning style. One line of research has been by psychologists, and another line of research has been by educators. The two groups have developed their own terminology and definitions to describe cognitive/learning 6 style (Blakemore et al., 1984). Psychologists frequently have used the term cognitive style to describe their research about style, and educators frequently have used the term learning style to describe their research about style. Style characteristics measured by psychologists and educators through style assessment inventories often overlapped, but the terminology to describe the characteristics of style was often different. For example, assessment inventories developed by psychologists which measure style often measured style factors called field dependence/independence, reflective/impulsive, complexity/simplicity and analytic/nonanalytic. On the other hand, assessment inventories developed by educators often measure style factors which deal with preferences for different kinds of social structure, physical environments, and personality characteristics which influence learning. Guilford (1980) summed up much of the difficulty with the research on cognitive/learning styles and with the instruments used to measure styles. He felt what was referred to as cognitive (learning) style was actually a whole range of different kinds of dimensions: some had to do with ability, some with higher order functions and others with preferences. Because of the variance and confusion about the definitions and ‘terminology in style assessment instruments, the practitioner is at a real disadvantage when trying to decide which assessment instrument to use. 7 RELEVANCE OF THE STUDY TO THE FIELD OF EDUCATION "The central purpose of any program of education is to promote learning" (Gagne, 1975, p.1). Learning occurs when the learner is presented with information, receives the information and somehow stores the information for further use. In his model of Information Processing Charles A. Letteri graphically represents this process as presented in Figure 1 (Letteri, 1988, p. 24). According to this model, after the presentation of the stimulus, the information goes through the learner's unique cognitive processing process. "The learner is central to this cognitive definition of learning. The responsibility for engaging the learning process belongs to the learner alone. The student must understand the learning process and how to control and direct it."(Letteri, 1988, p. 23) The Letteri Model diagrams the background and characteristics learners bring to the learning situation which influence what will be learned and how it will be learned. The concept of cognitive/learning styles is one way psychologists and educators have tried to define the background and characteristics that the learner brings to the learning encounter. Cognitive/learning style assessment inventories have been developed in order to diagnose individual style. Keefe (1988, p. 2) contends that diagnosis is the most neglected function of schooling and that style is the least understood I'Igure 1. Information Processing, General Operations Charles A. Letteri, I982 ATTENTION LEARNING AWARENESS-MONITORING I I I RECOGNITION > T I SHORT j PERCEPTION PERCEPTION FILTER - ' I..ONG ST'MULUS " ' MODE '7' MEMORY SYSTEM TERM TERM MEMORY \ WORKING /’ MEMORY . ------ “"3” MMS‘TEIISEL I EXTANT : MEMORIZE ------ " REHEARSAL: DATA l TRAN ma \ - -.- ICOONITIVIa l S M x $539" STRUCTURE) - ' LEARN --- ‘~ , REHEARSA : . OUT OF “ ,4 L ASSIMILATI: . SYSTEM ,’ LONGJI'ERM DIFFERENTIATE \ I STORAGE AND ASSOCIATE \ / RECALL \ COGNITIVE CONTROL DIMENSIONS IANALYZE—COMPARE—FOCUS—NARROW—TOLERANT—SHARPEN—COMPLEX. ETC. I 9 element of diagnosis. This may be true, but it is not for the lack of assessment inventories. Using only 7 reference materials on cognitive/learning styles, ( Blakemore et al., 1984; Cornett, 1983: Curry, 1987; De Bello, 1988: Keefe, 1988: NASSP, 1982; NASSP, 1979) this researcher counted 56 published inventories on styles. There were also "in house" unpublished inventories (Cornett, 1983). Because of the large numbers of cognitive/learning style assessment inventories, the lack of available research about how to choose appropriate inventories, and the confusion about what is cognitive/learning style, practitioners may have often found it difficult to select inventories for their educational needs. In this research, experts in the field of cognitive/learning style assessment inventories have contributed information used to develop a model to assist practitioners in choosing an assessment inventory which best suits their instructional needs. If the best assessment inventory is chosen, then presumably more accurate data about students' learning preferences will be obtained, and more optimal learning situations might be structured. This research is relevant to the field of education because it might aid practitioners in selecting an assessment inventory to help them structure a more optimal learning environment for individual students. 10 THEORETICAL BASE FOR THE STUDY Learning is a term used frequently in this study. It is beyond the scope of this research to make explicit the foundations of the theory of learning. However, it is important to the understanding of this study to outline the definition and theory about learning used by this researcher. There is no accepted single definition of what learning is. Some theorists are interested in overt behaviour, others in inferred mental processes. Some focus on the stimuli which are said to lead to molecular behaviour, others on the shaping and control of patterns of behaviour which constitute action in the environment. Some are concerned with strategies for teaching while others seek to facilitate self-discovered learning. Thus we are faced with an area of knowledge with very wide and flexible boundaries (Jones, 1982, p. 2). However, there are three criteria that psychologists have traditionally used to define learning. These criteria are: (1) there must be some change in behavior,(2) the change must be relatively stable, and (3) the change must result from experience (Zanden & Pace, 1984). Gagne (1970) restated these criteria in his definition of learning. "Learning is a change in human disposition or capability, which can be retained, and which is not simply ascribable to the process of growth" (3). Futhermore Gagne stated that the change of performance was the indicator that learning has occurred and change could be measured by looking at the subject's performance before and then after being placed in a learning situation. These criteria for determining and defining learning are the ones used in this study. Flowing from the criteria and definition of learning 11 described above was the definition of learning style developed by Keefe and Languis and adopted by the NASSP Task Force. ...the composite of characteristic cognitive, affective, and physiological factors that serve as relatively stable indicators of how a learner perceives, interacts with, and responds to the learning environment. It is demonstrated in that pattern of behavior and performance by which an individual approaches educational experiences. Its basis lies in the structure of neural organization and personality which both molds and is molded by human development and the learning experiences of home, school, and society (Keefe & Languis, 1983, p.1). The concept of cognitive/learning style was developed because of the recognition that learners came to the learning task with different ways of knowing and learning. That learners filter information using their own perceptions has been known for a long time. Bruner and Postman (1949) in their article about perception, cognition, and behavior concluded, "As we have thought about the matter, it becomes increasingly clear that a thoroughgoing psychology of perception imbedded in a thoroughgoing general psychology must inquire into all the conditions... how, in short, the organism by perceiving comes to adapt to the external, distal stimulus" (p. 29). Schlesinger (1953) of the Menniger Foundation outlined how important individual differences were in an early study of cognitive organization. His conclusion from this study was: Many studies have documented the growing conviction that perception in general can be influenced by motivational and situational factors. It is the primary purpose of this paper to show that the search for such generalized relationships bypasses an important source of variables 12 which condition these relationships. This sources is the person himself and the way he is organized to perceive-or more generally, his cognitive organization (p. 354). Gardener (1953), who was also from the Menniger Foundation, described individual differences of organizing and experiencing stimulus which he labeled as the individual's style. Piaget believed, "Cognitive acts are seen as acts of organization of and adaptation to the perceived environment" (Wadsworth, 1971, p. 9). Even though psychologists disagree about what constitutes learning and which learning theory is correct, one of the things they all do agree upon is the existence of individual differences in learning (Dubin & Okun, 1973). Bentov (1977) summed up the filtering process in his theory of relativity. The theory of relativity emphasizes the notion that no matter what we observe, we always do so relative to a frame of reference that may differ from someone else's, that we must compare our frames of reference in order to get meaningful measurements and results about the events we observe (p. 3). The terms cognitive or learning style(s) were developed to define the concept of individual differences in learning and knowing. Elements of learning style appeared in the research literature at early as 1892....Even before 1900 Cattell and Jostrow attempted to relate differences in perceptual mode to general intelligence and learning performance without success. Vernon, Eysench, and others described perceptual topologies such as analyzers vs. synthesizers and color vs. form reactors. The term 'cognitive style' was coined by Allport in 1937 to refer to a quality of living and adapting influenced by distinctive personality types. In the 1940's Thurstone and later Guilford identified factors of perceptual speed and flexibility.... C. :o. I... an 0' a“. V" I. I";‘ ‘v- 5 13 Specific research on cognitive styles was greatly expanded after World War II at Brooklyn College, the Menniger Foundation, and the Fels Institute (Keefe, 1979, p. 4-5). In 1954 Witkin (1977) was doing research on field dependence/independence, and by 1960 several psychologists and some educators were researching cognitive/learning styles. As new constructs of cognitive/learning style were identified, cognitive/learning assessment inventories were developed. The assessment inventories were used to decide if the construct existed and/or the parameters of the construct, and were also developed as a tool for practitioners to determine cognitive/learning style differences among learners. Most of the authors of the inventories formulated a rationale for cognitive/learning style and the cognitive/learning style construct they developed. As part of constructing a model for practitioners, this researcher examined some of these rationales and has presented them in subsequent parts of this work. In this study a model was constructed to help practitioners choose a cognitive/learning style assessment inventory. For the purposes of this study, the meaning and function of the word model was based on writings by Paul C. ‘Nutt (1984) and Leslie J. Briggs and Walter W. Wager (1980). .According to Briggs and Wager, "The word 'model' refers to a particular organized set of procedures for actually carrying out:a problem-solving process for a particular purpose" (p. 4)- In this study a flowchart and a set of matrices were developed as the organized set of procedures for carrying out 14 the problem-solving process of choosing an appropriate cognitive/learning style assessment inventory. According to Nutt (1984) the reason a model is used is to give the planner (in this study the practitioner) pictures of several possible solutions (cognitive/learning style assessment inventories). The model aids in the conceptualization process by offering the planner various ways to view the problem. The planner uses the structure to pose competing ways to deal with the problem and to draw out the implications of each.....The model helps the planner list, merge, and define key elements that must be considered. The model permits the planner to detail key functions so an assessment of workability can by made. Finally, models provide a coordinational device. They allow the planner to illustrate options, gain sanction, and communicate preliminary ideas to key parties (Nutt, 1984, p. 170). Additionally, according to Nutt, models could reduce costs by acting as a surrogate for pilot programs. In this study, if teachers and administrators used the model, and thus were able to narrow their choices down to one or two appropriate cognitive/learning style inventories then they might be able to avoid the expense of buying all the inventories or the wrong inventory. Nutt used the term nova models to designate new representations of solutions to problems. A nova model is a custom made model to fit a specific situation rather than a lristorical model which is a model built on the practices of «others. Nova models are built from the basic questions that undergird the problem. The model in this study was a nova model. This researcher started with basic questions such as, what is cognitive/learning style, and then asked for responses for the questions until consensus was achieved. A 15 model which fits the specific need of choosing a cognitive/learning style assessment inventory was developed from this consensus. A crucial element when developing a nova model is to use creativity techniques such as the Delphi instrument (discussed in Chapter 2). A Delphi methodology was used in this study to stimulate creativity and develop new solutions. RESEARCH QUESTIONS ima esea c uestion What process might practitioners follow to reliably select cognitive/learning style assessment inventories that effectively match their needs? Sill-mm Which cognitive/learning style inventories are perceived as being widely used? What elements of cognitive/learning style inventories are perceived to differentiate effectively and efficiently among widely used inventories? Are some elements in widely used cognitive/learning style inventories perceived as more important than other elements? What is the perceived range of elements of cognitive/learning style inventories in terms of 16 their respective importance? 5. What elements are perceived as important for a model that assists practitioners in selecting an appropriate cognitive/learning style inventory? LIMITATIONS OF THE STUDY Limitations of the study are related to the population, the methodology, and the procedures of the study. The population was made up of experts in the field of cognitive/learning style assessment inventories. It was not a randomly drawn sample. It was drawn from people who have published and given presentations about cognitive/learning style assessment inventories. They are made up mostly of university professors. Another limiting factor about the sample was that the respondents had to agree to participate in a study that required 30 to 60 minutes of their time for three separate rounds over a span of several months. Some of the experts contacted chose not to make that kind of commitment. Finally, because the sample was drawn from a population of experts the pool was limited in size (Round 1=36 experts, Round 2=33 experts, Round 3=29 experts). The Delphi methodology, which was used, may be open to semantic difficulty in communication. The open ended questions posed in each round tend to make it difficult to formulate unbiased next round questions. Because opinions are what were being sought, quantification of responses may In 17 leave some results open to various interpretations. A further limitation is the number of missing cases. Each succeeding Delphi Round required more of the lreespondents' time because each questionnaire was longer and more complex. The number of returned results diminished from round to round. Also it took approximately seven months from the initial phone contact to the mailing of Round Three. This time frame went from February 1990 to September 1991, Which covered two academic years. Some of the experts lost interest or moved during this period of time. Lastly this Delphi was several pages in length. Because of its length, it is quite possible that the last questions Inay not have received the same kind of attention as the first questions received . ORGANIZATION OF THE STUDY The study is organized as follows: Chapter 1 states the problem and purpose of the study. It also details the need for the study, and how the study adds to the field of Education. Research questions are Presented. The scope and limitations of the study are given. Chapter 2 reviews the literature on the Delphi technique, cognitive/learning styles, and cognitive/learning s‘Izzyle assessment inventories. Chapter 3 details the design of the study. It includes the research population and the sample, the instrumentation, 18 tzlre data collection, the procedures, and the plan for aarialyzing the data. Chapter 4 reports the data analysis and the summary of 1:11e findings for each research question. Chapter 5 presents the study conclusions. It includes a lorrief summary of Chapters 1 through 4. There is a presentation of the findings, the conclusions, and recommendations growing out of the study. SUMMARY In this chapter, the researcher stated the problem for this study. The stated problem was: What process can practitioners follow to reliably select cognitive/learning Style assessment inventories that effectively match their liteeds? The given purpose in this study was to develop a tool practitioners could use in order to effectively choose among Widely used assessment inventories. Additionally, a need for ‘tlie study was established by demonstrating the confusion in 'tlie literature about the meaning of cognitive/learning style ahd how to measure it. Relevance of the study to the field of education was 1“Lighlighted by defining certain goals of education and by Presenting the concepts included in the Letteri W rmat' Processin . The theoretical base of the study I‘esides in learning theories, cognitive/learning style theories, and model theories which were used in this l9 zreasearch. Additionally, a brief history of the development of the concept of cognitive/learning style was presented. The research questions were also presented. Limitations Irealated to the population, the methodology, and the zozrocedures of the study were cited. Finally the organization of the 5 chapters in this study were described. CHAPTERZ REVIEW OF THE LITERATURE INTRODUCTION The first part of this literature review investigates the Delphi technique. This is the methodology of this research. It undergirds and structures the research. Areas of the Delphi technique that are discussed are: definitions of the term Delphi technique: history, purposes, and philosophy of the technique; the process of the technique; disadvantages, limitations and problems with the technique: and advantages and applications of the technique. The second Part of the literature reviews the elements of cognitive/learning style and the inventories that measure Style. Areas that are explored are: definitions of Cognitive/learning style, applications of cognitive/learning Style to learning situations, and assessment inventories of cognitive/learning style. WISE-.13 The methodology of this research is the Delphi tectlnique. In this section of the literature review the Delphi technique will be examined. Specific areas of 20 21 examination are: (1) the definitions of the technique, (2) the history, purposes, and philosophy of the technique, (3) the process of the technique, (4) disadvantages, limitations and problems of using the technique, and (5) advantages and applications of using the technique. D E' 1!. According to Linstone and Turoff (1975) it is difficult to find one definition of the Delphi technique because it has been applied in such varying situations. However, below are several definitions that have been developed by experts using and writing about the technique. Delphi may be characterized as a method for structuring a group communication process so that the process is effective in allowing a group of individuals, as a whole, to deal with a complex problem.... To accomplish this structured communication there is provided: some feedback of individual contributions of information and knowledge; some assessment of the group judgment or view: some opportunity for individuals to revise views: and some degree of anonymity for individual responses (Linstone & Turoff, 1975, p. 3). Delphi is a group process which utilizes written responses as opposed to bringing individuals together. ... it is a means for aggregating the judgments of a number of individuals in order to improve the quality of decision making....Delphi is essentially a series of questionnaires (Delbecq & Gustafson, 1975, p. 83). The Delphi Technique ... replaces direct debate by a carefully designed program of sequential individual interrogations (best conducted by questionnaire) interspersed with information and opinion feedback derived by computed consensus from the earlier parts of the program (Helmer, 1966, p. 17). Delphi ... operates on the principle that several heads are better than one in making subjective conjectures ... and that experts ... will make conjectures based upon rational judgment and shared information rather than merely guessing, and will separate their own hopes and personal motivation from considered judgment in the process (Weaver, 1972, p. 6). 22 Delphi is a survey approach that pools judgments without discussion. The term is somewhat whimsically drawn from the mythological Greek oracle of Delphi. A delphi survey systematically solicits and collates judgments to form a synthetic group. A series of questionnaires is used (Nutt, 1984, p. 106). From these definitions some common elements of a Delphi can be extrapolated. They are: (1) it is a group process, (2) it is a communication process, (3) there is a feedback loop, (4) there is usually some degree of anonymity for individual members, and (5) it is a decision making process. 'sto r ses and Philoso The first use of the Delphi technique was in "Project Delphi" which was an Air Force sponsored Rand Corporation study beginning in the early 1950's using expert opinion. The objective of this study was to achieve consensus by experts using questionnaires with controlled opinion feedback. However, work in 1964 by T.J. Gordon and Olaf Helmer brought the Delphi technique from the defense community to the notice of the outside community. The Gordon and Helmer study was a Rand Report on long range trends :mainly in the areas of science and technology (Linstone & fruroff, 1975). Since 1964 the uses for the Delphi technique Ihaye increased. The Delphi has been used in a variety of sitmations for a variety of purposes (Delbecq & Gustafson, 11975: Linstone & Turoff, 1975). For example, Sweigert and Schabacker (1974) used the Delphi technique to help to establish consensus about educational goals, the Michigan 23 Developmental Disabilities Council (1989) used the Delphi technique to help to establish consensus on the quality of family support of families with handicappers, Operach (1988) used the Delphi technique to build alternative speech and language models for elementary children, and Helmer (1966) used the Delphi technique for a long-range forecasting study. Some of the purposes of a Delphi study outlined by Weatherman and Swenson (1974) are: (1) a forecasting probe, (2) a strategy probe, and (3) a preference probe. Linstone and Turoff (1975) describe some of the Delphi technique applications. They are: Gathering current and historical data not accurately known or available, Examining the significance of historical events, Evaluating possible budget allocations, Exploring urban and regional planning options, Planning university campus and curriculum development, Putting together the structure of a model, Delineating the pros and cons associated with potential policy options, Developing causal relationships in complex economic or social phenomena, Distinguishing and clarifying real and perceived human motivations, and Exposing priorities of personal values, social goals (p. 4). Linstone and Turoff (1975) and Mitroff and Turoff (1975) .investigated the Delphi technique in relationship to five Futilosophical theories. Since one of the philosophical theories does not relate directly to the Delphi technique, 24 this researcher will examine four of the theories. According to Mitroff and Turoff, (1975) the characteristics of the Lockean Inquiring System philosophy are: Truth is experiential, ie., the truth content of a system (or communication) is associated entirely with its empirical content. A model of a system is an empirical model and the truth of the model is measured in terms of our ability (a) to reduce every complex proposition down to its simple empirical referents (i.e., simple observations) and (b) to ensure the validity of each of the simple referents by means of the widespread, freely obtained agreement between different human observers....In sum, the data input sector is not only prior to the formal model or theory sector but it is separate from it as well. The whole Lockean IS (inquiring systems) is built up from the data input sector (p. 21). The Lockean model relates to the Delphi technique when the researcher gathers data from the participants based on their experiences and then tries to achieve consensus from the participants. The Helmer (1966) forecasting studies done for the Rand Corporation using the Delphi technique are examples of using the Lockean model. This has been the most common model for using the Delphi technique. According to Mitroff and Turoff, (1975) the characteristics of the Kantian Inquiring System philosophy are: Truth is synthetic, i.e., the truth content of a system is not located in either its theoretical or its empirical components, but in both. A model of a system is a synthetic model in the sense that the truth of the model is measured in terms of the model's ability (a) to associate every theoretical term of the model with some empirical referent and (b) to show that (how) underlying the collection of every empirical observation related to the phenomenon under investigation there is an associated theoretical referent...Theory and data are inseparable....Kantian IS are.the epitome of multimodel, synthetic systems. On any problem, they will build at least two alternate representations or models of it (p. VI- "= we NU. . C O .: a ‘ I I... .‘ Q‘anu I I 'v—vaot a ".VA‘ No..." (I'D ()nn. : 'f! r’) (‘1’ 'r1 H- (I) 25 27-27). The Kantian model relates to the Delphi technique when the researcher uses alternate scientific theories and gathers data integrated from several disciplines and from these components builds more than one model or alternative. An example of a Delphi study that used the Kantian model is the Adelson and Aroni study (Linstone & Turoff, 1975) of "Differential Images of the Future." This is not a very common philosophical base for the researchers using the Delphi technique. Hegelian, or Dialectical Inquiring Systems, according to Mitroff and Turoff (1975) have these characteristics: Truth is conflictual: i.e., the truth content of a system is the result of a highly complicated process which depends on the existence of a plan and diametrically opposed counterplan. The plan and the counterplan represent strongly divergent and opposing conceptions of the whole system. The function of the plan and counterplan is to engage each other in an unremitting debate over the true nature of the whole system, in order to draw forth a new plan that will, one hopes, reconcile (synthesize, encompass) the plan and the counterplan....The data input sector is totally meaningless and only becomes meaningful, i.e., 'information', by being coupled to the plan and counterplan....Finally, it is also assumed that on every issue of importance, there can be found or constructed a plan and a counterplan; i.e., a dialectical debate can be formulated with respect to any issue (p. 29-30). Debate and conflict with reconciliation as the end product, hopefully, would be indicators of a Delphi study run using the Hegelian philosophical underpinnings. This researcher found only one Delphi study using this philosophical base Iflhich was a policy Delphi by Turoff (1975). Singerian-Churchmanian Inquiring Systems, according to 'TO'A ... U. V a up“ ‘I III In 26 Mitroff and Turoff (1975) have these main features: Truth is pragmatic: i.e., the truth content of a system is relative to the overall goals and objectives of the inquiry. A model of a system is teleological, or explicitly goal-oriented, in the sense that the 'truth' of the model is measured with respect to its ability to define (articulate) certain systems objectives, to propose (create) several alternate means for securing these objective, and finally, at the end of the inquiry, to specify new goals (discovered only as a result of the inquiry) that remain to be accomplished by some future inquiry. Singerian inquiry is thus in a very fundamental sense nonterminating though it is response oriented at any particular point in time; i.e., Singerian inquirers never give final answers to any question although at any point they seek to give a highly refined and specific response....The system forms an inseparable whole....The designer's psychology and sociology are inseparable from the system's physical representation (p. 33). Large studies that are ongoing and changing dealing with holistic concerns and including the designer or designers of the study as part of the study would be indicators of this type of Delphi study using the Singerian-Churchmanian Inquiring System. Using a computer search, this researcher found no Delphi studies that have used this philosophical base. However, the potential to use it with the Delphi technique exists, if cost and time limitation could be addressed. The issues that might be addressed using a Delphi study that incorporates this philosophical might be holistic issues (i.e. environmental concerns) with strong ethical considerations . In summary, in the early 1950's the Delphi technique was «developed by the Rand Corporation using a grant from the Air Force. However, the work of T. J. Gordon and Olaf Helmer brought the Delphi technique to the notice of a more general I. I" it 5“ I :AAA‘ UCVU‘ I I .I‘ p 50...! I n "IQI‘ p H.‘.. 0a..- - hut.‘ I C . F... a .-.“ n ‘A‘ l‘..‘ I... p o I ll' ((1 27 audience. The Delphi technique has been used for a variety of purposes and applications. Finally, the Delphi technique, according to purpose and application can have a variety of philosophical underpinnings. Four of these philosophical underpinnings (Lockean, Kantian, Hegelian, Singerian), as they related to the Delphi technique were examined. T e es e e h' Tech ' e Several authors have outlined the process of the Delphi Technique (Helmer, 1966: Nutt, 1984; Weaver, 1972; Linstone & Turoff, 1975: Delbecq & Gustafson, 1975: Sweigert & Schabacker, 1974: Hopkins, 1972, and Moore, 1987). Delbecq and Gustafson (1975) developed a 10 step process plan for implementing the Delphi technique. The 10 steps are: (1) develop the Delphi question, (2) select and contact the respondents, (3) select the sample size, (4) develop the Questionnaire #1 and the test, (5) do the analysis of Questionnaire #1, (6) develop Questionnaire #2 and test, (7) do the analysis of Questionnaire #2, (8) develop Questionnaire #3 and test, (9) do the analysis of Questionnaire #3, and (10) prepare a final report. Although these are fairly common procedures for doing a Delphi study, :modifications do exist. For example, the number of rounds or questionnaires may vary from 2 to 4 (Helmer, 1965: Delbecq & (Sustafson 1975), and the way data is collected can range from using the telephone (Linstone & Turoff, 1975) to using cassette tapes (Delbecq and Gustafson, 1975) . The Delphi OAR!“ 'fi'out RA.F” .~‘. . o gyana h u u.~vv o .‘lA ‘ U u a I C \i “‘ U ...-V F. Au v“ CA D O Q 5 U W V 1'. 28 technique has also been used in "real time" studies during computer conferences or meetings (Moore, 1987). In the preceding paragraphs some of the elements of the process of the Delphi technique are discussed. Selection of the Panel of Experts Helmer (1975) outlines his reasons for using and relying on expert opinion to arrive at predictions and decisions. Reliance on the intuitive judgment of experts is thus not just a temporary expedient but a necessary ingredient of futures research, for such experts are ~ needed in all phases of the effort. They are called upon (1) to supply judgmental data about the future, based on their intuitive, though often theoretically unstructured, insights into real-world phenomena: (2) to construct ad hoc models or to judge the suitability of existing models; (3) to apply their expertise as role players in simulation games, and (4) to use their imagination and inventiveness to design the instrumentalities and long—range strategies that result in appropriate action programmes for dealing with the problems of the future (p. 6). As criteria for the selection of experts for the panel, Helmer (1966) gives knowledge about the subject area, and reliability that is determined by how accurate their predictions have been in the past. Nutt (1984) adds the criterion of motivation as important in the selection process of experts. Weatherman and Swenson (1974) add the criteria of representativeness of the panel and independence of responses. Helmer (1966) divides experts into 2 groups which are ‘the specialists and the generalists. The specialists can Provide information or predictions. The generalists can Provide problem-formulation, model-structuring, or D AOA on; _‘ Duvet: s ‘ "a In! ~ uvoo-v. "A"! I. i up I O hav- l..~‘ u." -"'-I. .- “...! O A RA 6‘, ‘t o '1»; i . - 9.. 29 preference-evaluation among alternatives. According to Helmer, experts' performance can be improved significantly by grouping them with other experts in their field and allowing them to interact anonymously. Sample size varies from study to study. Delbecq and Gustafson (1975) states that in a homogeneous group of experts 10 to 15 participants might be enough. Whereas, in a study where there are various groups, the researcher might need several hundred people. It is their contention, however, that a well chosen group of 30 usually is sufficient to generate the data needed. Nutt (1984) believes that 10 to 15 is a manageable panel size. Weatherman and Swenson (1974) reported that most panels are made up of less than 50 members, but report that the Cyphert and Gant Study had 400 participants. In summary, Delphi studies rely on experts on the panel. These experts can be specialists or generalists. Important criteria in the selection of the panel members are: (1) they are knowledgeable about the subject area, (2) they are motivated to participate in the study, (3) they are representative of the expert population in their area of expertise, and (4) their responses are independent of one another. Sample size varies with the study being conducted. frhe most common sample size is under 50 with 30 being average . O t e on The Helmer study (1964) using the Delphi technique used 30 questions and a questionnaire. Weatherman and Swenson (1974) recommend open-ended questions for the initial round. Moore 1987) details Delphi studies using questions and a questionnaire for which questions are derived from a literature search. Delbecq and Gustafson (1975) suggest that in the initial round, respondents should be asked to respond to a broad question. According to Weaver (1970), respondents in the initial round should be asked to generate several specific statements of events. Although several techniques are possible in the initial round, a questionnaire consisting of several questions or a single question is the most common way to start a Delphi study. Number of Rounds In the Helmer (1964) study and the Operach (1988) study using the Delphi technique, there were 4 rounds. Nutt (1984) recommends 5 rounds as minimal to achieve results. However, for a simple form of a Delphi study, one that only requires pro and con analysis, 3 rounds are minimal. Uhl (1975) claims convergence to consensus after 2 or more rounds. Moore (1987) describes conditions where the rounds would vary from 3 to 5 rounds. Sweigert and Schabacker (1974) contend that it takes 3 or more rounds to produce reliable, convergent results. Weatherman and Swenson (1974) and Delbecq and Gustafson (1975) suggest 3 rounds. Young (1977) states, using the results of a computer search, that the majority of Delphi studies use 3 rounds. 31 Qisadvanteges, LimiEations and Probiems of Using the Delphi 19921119119 One of the major disadvantages to using the Delphi technique is the time it takes (Moore, 1987). The minimum amount of time to allow for a Delphi study according to Delbecq and Gustafson (1975) is 44.5 days. Moore (1987) and Linstone & Turoff (1975) also state that another disadvantage or problem of using the Delphi technique could be the bias and honesty of the monitoring team. The lack of face to face communication is of concern to Moore (1987). Linstone and Turoff (1975) also believe the process can limit communication. While the written word allows for emotional content, the Delphi process does tend to minimize the feelings and information normally communicated in such manners as the tone of a voice, the gesture of a hand, or the look of an eye. In many instances these are a vital and highly informative part of a communication process (p. 7). Delbecq and Gustafson (1975) state that the Delphi process can cause "a feeling of detachment from the problem-solving effort [and cause] ...communication and interpretation difficulties among respondents"(p. 35). Uhl (1975) had concerns about how permanent convergence to consensus was because of a study he ran a year after he had conducted a study where there was convergence to consensus. In the second study he ran, his conclusion was that the changes in opinion were only temporary. Weatherman and Swenson (1974) had concerns about distortions in results dueato the selection of participants, and that the studies 32 that tested the assumptions of the technique have produced ambiguous results. Weaver (1970) states: It is therefore crucial that these tools heavily emphasize the explanations upon which the plausibility of the forecast rests. An intuitive forecast which carries with it no explanatory quality may be correct, but it would be trivial. That is the singular weakness of Delphi (p. 269). Delbecq and Gustafson (1975) believe one of the disadvantages to using the Delphi technique is the high motivation it requires of the respondents. The technique is greatly influenced by the commitment and interest of the participants. 1 Linstone and Turoff (1975) list these reasons a Delphi might fail: Imposing monitor views and preconceptions of a problem upon the respondent group by overspecifying the structure of the Delphi and not allowing for the contribution of other perspectives related to the problem. Assuming the Delphi can be surrogate for all other human communications in a given situation. Poor techniques of summarizing and presenting the group response and ensuring common interpretations of the evaluation scales utilized in the exercise. Ignoring and not exploring disagreements, so that discouraged dissenters drop out and an artificial consensus is generated. Underestimating the demanding nature of a Delphi and the fact that the respondents should be recognized as consultants and properly compensated for their time if the Delphi is not an integral part of their job function (p. 6). To summarize there are some disadvantages, limitations, arui problems in using the Delphi technique. Many of these 33 are due to the way Delphi has been used rather than to inherent weaknesses in the Delphi technique. However, further research needs to be done to determine how to strengthen the process. Advepteges and Applications of Using the Delphi Technigpe Helmer'Y1966) details several advantages to using the Delphi technique. They are: 1. Expert opinion is a necessity when a choice needs to be made among alternative solutions when there is no accepted body of knowledge about what is the one best course of action. 2. The technique reduces the tendency for some experts to be influenced by the bandwagon effect of majority opinion or a particularly persuasive person. 3. Feedback by a peer group can stimulate the experts to take into consideration elements that they may not have thought about before or thought were unimportant. Linstone and Turoff (1975) lists these reasons to use 'the Delphi technique. The problem does not lend itself to precise analytical techniques but can benefit from subjective judgments on a collective basis, The individuals needed to contribute to the examination of a broad or complex problem have no history of adequate communication and may represent diverse backgrounds with respect to experience or expertise, More individuals are needed than can effectively 34 interact in a face-to-face exchange, Time and cost make frequent group meeting infeasible, The efficiency of face-to-face meetings can be increased by a supplemental group communication process, Disagreements among individuals are so severe or politically unpalatable that the communication process must be refereed and/or anonymity assured, and The heterogeneity of the participants must be preserved to assure validity of the results, i.e., avoidance of domination by quantity or by strength of personality (bandwagon effect) (p. 4). Weatherman and Swenson (1974) detail the advantages for using the Delphi technique. They are: (1) anonymity is valuable, (2) it provides a means of obtaining information about complex issues, (3) it helps experts to conceptualize complex issues, (4) it is a simplifying device, and (5) it is efficient because it focuses attention on the topic and allows control by the survey manager. Weaver (1972) concludes..."that Delphi, in combination with other tools, is a very potent device for teaching people to think about the future in much more complex ways than they ordinarily would" (p. ii). He also gives some suggestions that he believes will correct some of the disadvantages of using the Delphi technique. They include: (1) Participants :making sure they are familiar with the topic under consideration, (2) The use of category width rather than specific dates or pro and con, (3) Direct confrontation of participants, (4) Participants should be asked to explain their answers and choices, (5) Participants should be given at: least 2 sets of factors that might influence their topic, =-= p. I .Ci I I II A I A! v, EVA :- :VR' nu u.d. I 1‘53...“ a no “.ng Ablcy 'huv‘ '5 I I“. .~~‘~. 35 (6) Participants should weigh the desirability of an outcome, (7) Feedback should be open discussion in small groups, and (8) Convergence should be taken as an indicator of force of an argument, not how accurate an outcome is (p. 48-49). Although some of these suggestions would be controversial to other experts of the Delphi technique, they do provide a forum for discussion. In summary, there are many advantages to using the Delphi technique. These advantages have been recognized by the experts in the field as well as researchers in other fields. A computer search of Eric for the years 1966 through 1991 run by this researcher showed that there are 631 listings that have Delphi as one of the descriptors. It is clear that the Delphi technique is a popular research technique because of some of the advantages of using it. ELEMENTS OF COGNITIVE/LEARNING STYLE AND THE INVENTORIES THAT MEASURE COGNITIVE/LEARNING STYLE In this section the definitions of cognitive/learning .are examined. Additionally, the literature about the applications of cognitive/learning style and the value of teaching to learners' cognitive/learning style are explored. .Finallyy assessment inventories of cognitive/learning style are investigated . 36 Definitions of CogpitivelLeerning Style Several definitions have been used for cognitive/learning style. Some experts in the field have even given separate definitions for cognitive style and learning style. Hill (1981) defines cognitive style as, "the general mode of activity, method, or approach employed by an individual in the process of conceptualization, i.e. in the process of forming shared or relatively well-agreed upon interpretations of a set of sensations" (p. 63). He defines learning style as "the mode of activity, method, or approach employed by an individual in both the process of perceptualization, i.e., in the process of form an individual interpretation of a set of sensations, and in the process of conceptualization" (p. 63-4). Rita Dunn (1981) also gives separate definitions for learning and cognitive style. Learning style is the way in which individuals respond to the environmental, emotional, sociological, and physical stimuli that surround them: whereas cognitive style-whether it refers to field dependence or independence, global or analytic approaches the 'brain' concept of learning, or specific study skills-describes the ways in which the brain processes information (p. 34). Zanden and Pace (1980) define style which they label cognitive style in this way. In its broadest sense, cognitive style may be thought of as an individual's typical mode of processing information. It refers to consistencies in an individual's way of functioning in his or her day-to-day activities, especially when the activities have to do 37 with organizing and categorizing perceptions (p. 114). Curry (1987) feels that the terms learning style and cognitive style should not be used because they are overused and regularly confused. He proposes, instead, a series of new terms to replace learning style and cognitive style. The terms are: (1) Cognitive personality style, (2) Information processing style, (3)1nstructional preference, and (4) Learning strategy. Blakemore, McCray, and Coker (1984) acknowledge the variety of definitions given by researcher, theorists, and instrument developers. Some of the definitions are restricted only to the factor(s) included in a specific instrument, whereas others attempt to encompass all the definitions used by the people working in the area. Some of the definitions are explicitly stated whereas others must be inferred from the statement of the purpose of a particular instrument (p. 1). Cornett (1983) feels that educators are often confused by the labels and categories used to describe and define style. She uses the term learning style as a general term to define learning and cognitive style. Essentially, learning style can be defined as a consistent pattern of behavior but with a certain range of individual variability. When persons learn they use learning styles that are uniquely their own, but make moment-by-moment style adjustments, depending on the nature of the task and the teaching style being used. Styles then are overall patterns that give general direction to learning behavior (p. 9). Finally, the definition that is used in this study because it is a broad definition including both cognitive style and learning style, is the definition developed by Keefe and Languis adopted by the NASSP Task Force and 38 reiterated by Keefe and Monk (1987) in their Learning Style Profile Examiner's Manual. The composite of characteristic cognitive, affective, and physiological factors that serve as relatively stable indicators of how a learner perceives, interacts with, and responds to the learning environment. It is demonstrated in that pattern of behavior and performance by which an individual approaches educational experiences. Its basis lies in the structure of neural organization and personality which both molds and is molded by human development and the learning experiences of home, school, and society (p. 1). In summary, there are many definitions of style. Some definitions only define cognitive style, others define learning style, others define both. A broad definition of style is the Keefe and Languis definition adopted by the NASSP Task Force and, it is the definition used in this research. CognitivezLearning Style The development of theory and instruments to measure cognitive/learning style has been in response to educators asking what makes instruction successful. There is ample research which seems to prove the cognitive/learning style of students does affect instruction. As examples: reading achievement and cognitive/learning style has been researched thoroughly. It has been found that children learn to read better when they are taught through their style (Carbo, 1983), style is a predictor of reading comprehension (Smith, 1981), cognitive/learning style affects reading skills (Readence, 1977), and cognitive/learning style is a better - \ Q 39 predictor of reading achievement than I.Q. (Kaley, 1977). There has been some research that indicates cognitive/learning style and affective learning outcomes are related. When teaching style and cognitive/learning style were matched affective or behavioral outcomes were influenced (Cotterell, 1982), style correlated with course satisfaction (Drummond & McIntire, 1977), and student's style and self- concept are correlated (Griggs & Price, 1981). Cognitive/learning style is also a predictor of academic achievement (Cohen, 1968; Letteri, 1980; Yeats & Strag, 1971), and an understanding of a particular style should help teachers to evaluate individual performances on classroom standardized tests (Coop & Sigel, 1971). There is research to show that cognitive/learning style varies among different groups. For example, the style of Nigerian and American children differ (Hale, 1981), style differs between gifted and nongifted students (Stewart, 1981), the style between learning disabled and nondisabled differ (Sigg & Gorgirilo, 1980; Guyer & Friedman, 1975) and economically disadvantaged students have common elements on an cognitive/learning style profile (Hallahan, 1970). Research has also shown thinking patterns and style have a relationship such as concept identification (Davis & Klausmeier, 1970) and formal patterns of reasoning (Lawson & Wollman, 1977). Students' choices in academic programs often notches their style (Witkin et al., 1977). Research shows that teachers who know and adapt to a student's style do a 40 more effective job of teaching in terms of learning outcomes (Mullally, 1977; Dunn & Dunn, 1981). Witkin (1977) in a summary article about cognitive/learning style makes these points that are backed “P 1. 10. 11. 12. 13. 14. 15. 16. 17. by the research. A person is consistent in style across different tasks. Learners are very stable in a preferred mode of style over a very long time. There is a difference in style between men and women. Differences in style appear in problem solving. Style affects perceptual and intellectual activities. How people respond in social situations is affected by their style. How well people can express feelings and attitudes is determined by style. Whether people like to be with people or not is affected by style. Style affects a person's body language. How well people are liked is affected by their style. What jobs people select is affected by their style. The amount and kind of reinforcement needed to learn is affected by style. How well criticism is tolerated is affected by style. The degree of structure that a teacher needs to offer in order for students to learn is affected by style. Teacher and student match or mismatch in style cause a difference in student satisfaction in a course. Career choices are affected by style. Style and performance in specialized areas are related. Okto— a. 6 41 th Literature Conce in rob e s d ss 5 Ad resse CognitivezLearning Styles A number of problems and issues have been addressed in the literature concerning cognitive/learning style. In the area of the theoretical base and the construct of cognitive/learning style some questions have been posed. Denson (1977) believed that researchers needed to build some theory to undergird the area of cognitive/learning style. McNary, Michael, Richards and Lovell (1975) in a study of 100 fifth and sixth grade students concluded that there was no universal construct of cognitive style. Curry (1987) stated that there was no unitary concept of cognitive/learning style. Resulting from the lack of a unitary theoretical base is the issue of the multitude of definitions of cognitive/learning style and the definitions of the terminology used to describe cognitive/learning style (Curry, 1987; Denson, 1977). The field of cognitive/learning style is often described in the literature as fragmented (Curry, 1987). How stable is an individual's cognitive/learning style is another issue addressed in the literature. Coop and Sigel (1971) and Davis (1971) found that cognitive/learning style among young children was fairly stable, but by the time students reached college age cognitive/learning style was not very stable. However, Witkin (1977), Keefe (1977), and Dunn, DeBello, Brennan, Krimsky, and Murrain (1981) disagree and offered evidence that cognitive/learning style is relatively 42 stable for each individual learner. Matching a learner's cognitive/learning style with his or her learning environment should bring about better learning according to several developers of inventories (Dunn et al., 1981: Keefe, 1977; Gregorc, 1977). However, Coop and Brown (1970) in a study of 80 college students concluded, ...there is no significant interaction between cognitive style and teaching method in regard to either factual content achievement or conceptual-generalization content achievement. This finding may have resulted, in part, from the inability of the experimenter to design teaching methods which were specifically analytic or specifically nonanalytic ...(p. 404) Davidman (1981) objected to trying to match learners' styles with instruction. He contended that mismatching styles and instruction makes students learn to use several types of cognitive/learning styles and makes them better learners. CogpitivelLearping Style Assessment Inventories Research in education and the behavioral sciences is extremely complex and fairly recent. Education did not begin to emerge as a science until the start of the twentieth century. Scientific progress is based to a large degree upon the precision of our instruments and upon our ability to measure the phenomena concerned with the science in question. A student who examines the history of any sciences will find that development of better tools is almost invariably followed by important gains in scientific knowledge, disproof of some theories, and confirmation of others. The relatively late emergence of education, psychology, sociology, and other behavioral sciences is due largely to the complexity of the phenomena they attempt to measure and the consequent 43 slow progress in developing measuring tools (Borg & Gall, 1979, p. 1-2). Research into cognitive/learning style and the assessment inventories that measure style is further complicated by the fact that both psychologists and educators, at different times, have studied cognitive/learning style and have come up with two separate or parallel lines of research (Keefe, 1982). Also there seems to be no integrated theory of learning (Thompson [NASSP] 1982) or a theoretical framework for cognitive/learning styles. According to Keefe, (1988) "diagnosis is the most neglected function of schooling. Diagnosis includes the assessment of student developmental characteristics, acquired knowledge and skills, and cognitive/learning style. Learning style is probably the least understood element of diagnosis" (p. 2). Besides, learning style research has been hindered by the lack of available instruments or inventories to measure the full range of educational styles (Ferrell [Keefe Ed.], 1988). Another complicating factor that adds to the lack of clarity to the field is the feeling that there should be one solution to magically solve all learning problems. For example, from this viewpoint came the idea that individualized instruction would solve all learning problems. As with all "one pronged" approaches to learning, some students benefited and some Students failed (NASSP, 1979) . The panacea approach, whether it be for educational styles or individualized learning, can 44 only cause suspicion of the whole topic (Gregorc, 1982). Guilford (1980) sums up what needs to be done in the field of educational style by commenting that research needs to be done to determine connections, definitions, to develop models and to clarify the field. An ERIC (Educational Resources In Education) search done in February, 1988 revealed 2,714 documents in ERIC under cognitive style which is the descriptor that is used in ERIC for learning/cognitive style. ERIC only lists documents written from 1966. Presently it indexes government-funded research grants, state publications, curriculum guides, papers presented at professional meetings, educational pamphlets and, since 1969, journals in education. It does not generally index dissertations and books. An ERIC computer search that matched comparison, contrast and analysis with learning and cognitive styles with inventory(s) test measurement and instrument(s) showed there were only 6 documents that were indexed under these terms. Of these 6, only 2 compared, contrasted, and evaluated different types of cognitive/learning styles inventories.. The first of these 2 articles was written by Helen S. Lepke in the Eezeigp_Lepgpege_Apppele (1978). It is titled "Assessing Individual Learning Styles: An Analysis of Five Instruments". It was mainly a descriptive study of five learning style instruments. They were: Harry Reinert's ELSIE; Joseph Hill's Cognitive Style Interest Inventory: Anthony Papalia's Learning Modalities and Individual 45 Difference Inventories: David Hunt's Paragraph Completion Method; and the Dunn, Dunn, and Price Learning Style Inventory. Ms. Lepke briefly described each inventory but offered very little comparison and/or contrast among the 5 instruments. Because this was mainly a descriptive study there was very little analysis of the different inventories. This study contained no standardized criteria to evaluate the different inventories. The second article was written by Barbara G. Ferrell (1983). Its title was, "A Factor Analytic Comparison of Four Learning-Styles Instruments," and was published in the Journai of Educapional Psychology. Ms. Ferrell looked at 4 inventories. These inventories were: Grasha and Riechman's Student Learning Style Scales; Johnson's Decision Making Inventory: Kolb's Learning Styles Inventory; and Dunn, Dunn and Price's Learning Style Inventory. The author's primary objective in this article was to determine if these 4 instruments had construct validity and a theoretical base. Using a theoretical conceptualization developed by J.W. Keefe, Ms. Ferrell did a factor analysis of all 4 learning styles inventories and found they measured different constructs. Keefe (1983) conceptualized learning style (cognitive/learning style) as composed of 3 types of behaviors: cognitive, affective and physiological/physical. A cognitive behavior was viewed as one resulting from a preference for a given type of information processing or cognitive style. An affective behavior was the result of a given attitude or opinion. Physical/physiological learning style behaviors are of two types: KL: Rhifizy' 'U—‘b I. “by r, in. ‘ A “‘35 . “an. - FIII'.; ~ \ h.l..~ ‘ . *H-n ,. "Ubb. o I...- 3.,“ I ‘ u 'A.‘- 5-."bu one“ ““~‘Ab I ‘h~.:.‘ to” ‘4. 5 I'atc ?‘~.,S :ma 46 environmental factors that impinge on learning and biological factors in the makeup of individual that have an impact on the learning situation (p. 33). This article gave a brief description of each inventory and compared, contrasted and analyzed the 4 inventories only as they related to Keefe's conceptualization. A model was built that measured the cognitive, affective and physical/physiological characteristics from the Keefe conceptualization of learning styles. Because of the narrow scope of this research and the complex mathematical procedures the value to practitioners is somewhat limited. As a result of this computer search this researcher was able to find that there were 64 published assessment inventories of cognitive/learning style. Of the 64 inventories, 26 had been written about in several professional journals and magazines. Another ERIC search in March, 1989 matched combinations of 3 terms: test construction, cognitive style and models. Only 3 documents were found and none of them directly related to the research proposed here. In addition, the term evaluation methods was also matched with cognitive style. For this match 3 documents were found, but none were directly related to this research. In turn, the descriptor cognitive style was matched with combinations of measures, individual educational and diagnosis testing. There were 21 documents fOund using this match but none of them were directly related to this research. Lastly, the terms test construction and eNaluation methods were matched with test-validity and test- 47 reliability. There were 26 documents found, and two of the documents were relevant to this research. The first document that related directly to this research was a research report titled A Guide To Learning Styie Assessment by Thomas Blakemore, Paul McCray and Charles Coker (1984). This report described 12 learning style assessment instruments. They were: 1. Learning Style Inventory and the Productivity Environmental Preference Survey, by Dunn et al. 2. Learning Styles Inventory, by Renzulli & Smith 3. Your Style of Learning and Thinking, by Torrance et al. 4. Learning Styles Inventory, by Canfield & Lafferty 5. C.I.T.E. Learning Style Inventory, by Canfield & Lafferty 6. Learning Interaction Inventory, by Jacobs & Fuhrmann 7. Grasha-Reichmann Learning Style Inventory 8. The Learning Style Inventory, by Hanson & Silver 9. Learning Style Inventory, by Kolb 10. Cognitive Style Mapping, by Hill 11. Assessing Conceptual Level (with paragraph completion method), by Hunt, and 12. 4-MAT System by Bernice McCarthy In this report, the 12 inventories were compared and contrasted by using a model that looked at five factors. The factors were cognitive, social, motivational, physical and instructional. The 12 assessment instruments were chosen because they could be used by rehabilitation professionals. I I). ....V \ 48 The 5 factors in the model were selected because the authors felt they described the content in the 12 assessment instruments. Although a model was constructed from this research it was developed for a limited audience. The model developed for this dissertation will be for a broader more general audience of users. The second related document was a paper titled Three Meaegpee pf Qogpitive Styie: ghazectepistics. Facto; e a m ' 'o s o esearc ers. by Teri A. Denson (1977). In this paper 3 assessment inventories of cognitive style were compared and contrasted. The inventories were: 1. Group Embedded Figures Test, by Witkin et al. 2. The Nowicki Strickland Locus of Control Scale, by Nowicki and Strickland, and 3. Sutton-Smith and Rosenberg Impulsivity Scale, by Sutton- Smith & Rosenberg. The inventories were compared and contrasted on distribution characteristics, psychometric properties and an underlying factor structure. Although differences were described, no visual model was developed to compare and contrast the inventories. Additionally, the author did not suggest that the process she had developed could be transferred to other inventories. A manual search of Currept Ipdex To Journele Of Edgcetiop, an index that overlaps the materials in ERIC but includes earlier journals, did not reveal any additional 49 information. A computer search of Psychological Abstracts done in February of 1988 revealed 3,034 documents with the descriptor cognitive-style. However, a match with the terms inventory, tests, measurement, instrument with comparison, contrast, analysis only yielded 10 documents. Out of the 10 documents, there were none that compared or contrasted or analyzed 2 or more cognitive styles. A March 1989 search that matched the terms test-construction with test-validity, and test reliability with evaluation found 10 documents. None of the 10 documents directly related to this research. Esyepelpgieei Apstzacts indexes journals and serial publications about psychology and related disciplines world wide. The computer indexes of Psychological Abstracts cover 1967 to present. pisseptation Abstracts Intepnational indexes doctoral dissertation abstracts submitted to University Microforms International. There are 430 cooperating institutions in the U.S. and Canada. The computer search run by this researcher covered from 1861 to March 1988. Abstracts are subject indexed by the key words in the title. When the computer searched for the key words of cognitive style(s), it found 1,274 documents. However, when cognitive style(s) was matched with inventory, inventories, test, measurement, instrument with comparison, contrast and/or analysis only 6 documents were found. Of the 6 documents, only 1 was relevant to this research. It was "A Factor Analytic 50 Comparison of Four Learning-styles Instruments" which has already been discussed. An updated computer search and expert review in 1990 revealed 2 more documents that were relevant to this study. The first is written by Lynn Curry (1987), titled, Integrating Concepts of Cognitive or Learning Style: A Review pith Attentipp pp Peychometzic Stepgagds. In this document Curry investigates 22 assessment inventories of cognitive/learning style in relationship to an "onion" organizational model. He also looks at the reliability and validity of all of the inventories. The way he evaluates the strength of the validity and reliability of each instrument is by reviewing well designed studies and counting the number of acceptable results across several measures of validity and reliability. The inventories that he reviews and evaluates are: 1. Biggs—Study Process Questionnaire, 2. Canfield-Learning Styles Inventory, 3. Dunn, Dunn & Price-Learning Style Inventory, 4. Entwistle and Ramsden-Approaches to Studying, 5. Friedman & Stritter-Instructional Preference Questionnaire, 6. Goldberg-Oregon Instructional Preference Inventory, 7. Grasha-R1echmann-Students Learning Style Scale, 8. Hill-Cognitive Style Interest Inventory, 9. Hunt et al. Paragraph Completion Method, 10. Kagan-Matching Familiar Figures Test, ..- .4- .0 .dl . \I‘. 'u 1 II I”): ’4- 51 11. Kempa & Dube-Cognitive Preference Test, 12. Kalb-Learning Style Inventory, 13. Myer-Briggs Type Indicator, 14. Papalia-Learning Modalities Inventory, 15. Reinert Edmonds Learning Style Identification Exercise, 16. Renzulli & Smith-Learning Styles Inventory, 17. Rezler & Rezmovic-Learning Preference Inventory, 18. Schmeck et al.-Inventory of Learning Processes, 19. Schroder-Paragraph Completion Test, 20. Tamir et al.-Cognitive Preference Inventory, 21. Witkin-Embedded Figures Test, and 22. Witkin-Group Embedded Figures Test. This is the largest study done that has been found by this researcher. It also gives the greatest details about individual inventories in relationship to content, format, psychometric properties, and even places to order the inventories. The problems with this research are that several of the tests reviewed are not commonly used tests, and the information presented, although valuable, is not easily converted for use by the average practitioner. The second document is by Thomas C. De Bello (1988), and titled, Qomparison pf Eleven Major Learning Styles Models: Variables; Appropriate Popuietions; Validity of Instrumentation: and the Researcn Behind Them. He looks at 11 inventories. They are: 1. Dunn, Dunn, & Price-Learning Style Inventory, 2. Keefe-NASSP Learning Style Profile, 10. 11. 52 Hill-Cognitive Style Profile, Letteri-Cognitive Style Delineators, Ramirez-Child Rating Form, Reinert-Edmonds Learning Style Identification Exercise, Schmeck-Inventory of Learning Processes, ‘ Hunt-Paragraph Completion Method, Kolb-Learning Style Inventory, Gregorc-Gregorc Style Delineator, McCarthy-4 MAT System. This study, although it details the 11 inventories, is somewhat limited in scope because it only reviews 11 inventories. gunman y A review of the literature seems to show: Cognitive/learning style is an important measurable element in learning. Research on cognitive/learning style is fragmented and scattered so that it is difficult to evaluate the inventories that measure style. There has been limited research on evaluation of the different cognitive/learning style inventories. Models that have been developed that might help practitioners to decide which inventory to use are very limited in scope and/or audience that they address. V DOV: -. ‘Inn .44.. s I. Dd Rah- UV... RAI- run: 5 AA. 5'” IN. ‘Vt .D ANN .. CHAPTER 3 DESIGN OF THE STUDY Outline of Research Design The purpose of this study was to develop a model for evaluating and choosing among widely used assessment inventories of cognitive/learning style. The Delphi research method was used to query experts in the field of cognitive/learning style assessment inventories, to obtain consensus from the experts, and to construct a model from the consensus. The model was built as an aid for practitioners in the selection of an appropriate assessment inventory to meet a specific educational need. The Delphi Method was used as the research methodology. Three instruments in separate rounds were sent to each of the 41 participants in the study. Participants were chosen by reading the literature on cognitive/learning style assessment inventories and making a list of people who had written on or presented materials about cognitive/learning style assessment inventories. Addresses and phone numbers of the initial list of 142 experts were obtained by using the literature and directories listing colleges and universities and professors. Also phone directories were used. Of the 142 experts, this researcher was able to contact 60 experts by telephone from the initial list. From the list of 60 experts, 41 agreed to Participate in the study. The questions for Round One (see 53 54 Appendix A) were sent to the 41 participants on February 26, 1990. The questions for Round 2 (see Appendix B) were sent to the 41 participants on April 9, and the questions for Round 3 (see Appendix C) were sent to the 41 participants on September 2. Postcards were sent to participants who had not responded by the deadline date to remind them to return the surveys. The first round of the Delphi presented a series of broad questions and each subsequent round built on the responses from the previous rounds. The object was to achieve consensus among the experts. Although the experts were selected randomly, the population was not. Participants were chosen because they were the best informed concerning the subject of this study. Random sampling would have been inappropriate for the purposes of this study. "This (was) a purposive sample, rather than a representative one that would provide a statistical basis for generalization to a larger population..." (Willis & Bartell, 1988, p. 20-1). Results from the three Delphi Rounds assisted the researcher to respond to the following research questions. Research Questions esearc u s i What process can practitioners follow reliably to select a cognitive/learning style assessment inventory that effectively match their needs? This question was posed because there is a need to 55 choose between numerous inventories in order to help estimate students' cognitive learning style so that practitioners can deliver instruction in the most effective and efficient manner 0 Subquestions 1. Which cognitive/learning style inventories are perceived as being widely used? This question was posed in order to build a list of widely used inventories. Questions were also posed to give guidelines about what age groups the inventories had been used with, whether the experts were using the inventory currently, and which inventories the experts were familiar with but had not used. Information from this question was used to build Matrix B (see Chapter 5) which is a selection tool for practitioners. 2. What elements of cognitive/learning style inventories are perceived to differentiate effectively and efficiently among widely used inventories? All inventories have cognitive (content) and technical (operational) elements. Both the technical and content elements can be important to the practitioner when selecting an inventory. 3. Are some elements in widely used cognitive/learning style inventories perceived as more important than other elements? This question queried the experts to determine what 56 elements were the most important and what elements the practitioner should look for in an inventory. Matrix C (see Chapter 5) was formulated from this list of elements and characteristics in order to aid the educational practitioner in choosing a specific inventory which might more effectively and efficiently meet her/his instructional goals. 4. What is the perceived range of elements of cognitive/learning style inventories in terms of their respective importance? The results from this question were used to start to build a ranked list of elements and characteristics that were included in Matrix C. 5. What elements are perceived as important for a model that assists practitioners in selecting an appropriate cognitive/learning style inventory? This was the base list in Matrix D (see Chapter 5), and again, was part of the selection tool that practitioners would use in selecting an inventory. Descriptive statistics were used to report the results of all research questions. Literature Support Three areas of literature were reviewed for this study. The first area had to do with the methodology of this study. It was focused on research about and description of the Delphi methodology. The second area of review was the topic of cognitive/learning style. Research on cognitive/learning 57 style assessment inventories was the final area of review for this study. Computer searches of the electronic data bases of ERIE, Dissertation npstract International, and geyenpiegieei Apstzects were used to find major portions of the literature related to these 3 areas. These data bases, according to Martha Meaders, an expert educational researcher of data bases at Michigan State University Library, should include most of the relevant information on cognitive/learning style, cognitive/learning style assessment inventories, and Delphi. methodology. The various writings were reviewed to determine which ones were to be included in this study. A manual search of ERIC was done to pick pre 1966 articles since that is when the computer listings begin in ERIC. In addition, a number of books were read on each of the 3 areas. Also, major journals, relevant to the research, but not stored in any of the available electronic data bases, were studied. Material sent in by the experts as relevant to the research was reviewed. Some of the actual cognitive/learning style assessment inventories and manuals were used to help generate some of the first round questions for the Delphi. POPULATION AND SAMPLE The initial sample consisted of 41 professionals who had 58 written or given presentations about cognitive/learning style assessment inventories. Of the 41 participants 40 had Ph.D.s or Ed.D.s, and all participants had some affiliation with a college or university. Most of the participants were either professors, lecturers or administrators at a university or college. There were 12 disciplines represented by the 41 participants. The disciplines were: Education (16 participants), Educational Psychology (4 participants), Science (1 participant), Psychology (5 participants), Educational Services (2 participants), Chemical Engineering (3 participants), Allied Health and Medicine (2 participants), Health and Physical Education (3 participants), Mathematics (1 participant), Zoology (1 participant), Educational Technology (1 participant), and Music (2 participants). The 41 participants came from 17 states and Washington, D.C. Geographically, 15 participants came from the Midwestern States, 6 participants came from the Southwestern States, 10 participants came from the Southern States, 6 participants came from the Eastern States, 1 participant came from the Western States, 1 participant came from the Southcentral States, and 2 participants came from Washington, D.C. In this study there were 29 males and 12 females. Table 1 summarizes the demographic data as well as the response rate by Round. Table 1 umm of er s' emo r h'cs and Res onse Rates 59 EXPERT SEX LOCATION Ph.D.DISCIPLINE RDl RDZ RD3 OR Ed.D. A M Southwest Yes Education X X* B M East Yes Educational Psychology C M Midwest Yes Education X X D M Midwest Yes Education X X E F South Yes Education X X F M Southwest Yes Science X X G M Midwest Yes Educational X Psychology H M Midwest Yes Psychology X X I M South Yes Educational X X Services J M South Yes Chemical X X Engineering K F Southwest Yes Allied Health X X & Medicine L M Midwest Yes Psychology X M M South Yes Education X X central N M South No Health & X X Physical Education * "X" indicates expert participation in the Round 60 (Table i continned) EXPERT SEX LOCATION Ph.D.DISCIPLINE RDl RD2 R03 OR Ed.D. O M South Yes Education X X X* P M East Yes Education X X X Q F South Yes Health & X X Physical Educetion R F East Yes Education X X X S M Midwest Yes Education X X X T M Southwest Yes Zoology X U M East Yes Psychology V F Midwest Yes Education X X X W M South Yes Education X X X X F WashingtonYes Psychology X X X D.C. Y M Midwest Yes Health & X X X Physical Education 2 F WashingtonYes Educational X X x D.C. Technology AA M Midwest Yes Education X X BB M Midwest Yes Psychology CC M Midwest Yes Music X X X * "X" indicates expert participation in the Round 61 (Table 1 continued) EXPERT SEX LOCATION Ph.D.DISCIPLINE R01 R02 RD3 OR Ed.D. DD F Midwest Yes Music X X X* EE M West Yes Chemical X X Engineering FF M Midwest Yes Education X X X 66 M East Yes Educational Psychology HH M Southwest Yes Chemical X Engineering II M Midwest Yes Mathematics X X X JJ F South Yes Education X X X KK M East Yes Education X X X LL M Midwest Yes Education X X X MM F Southwest Yes Allied Health X X & Medicine NN F South Yes Educational X X X Services 00 F South Yes Educational Psychology * "X" indicates expert participation in the Round 62 Sam lin Procedure Participants were chosen by reading the literature on cognitive/learning style assessment inventories and making a list of people who had written on or presented materials about cognitive/learning assessment inventories. Addresses and phone numbers of the initial list of 142 experts were obtained by using the literature and directories listing colleges and universities and professors. Also phone directories were used. Of the 142 experts, this researcher was able to contact 60 experts by telephone from the initial list. When a person was contacted, this researcher gave this phone contact speech. Hello, my name is Diane Genshaw. I am a graduate student at Michigan State University working on my dissertation. I am certainly glad to get in touch with you. My research is on learning style inventories, and it has come to my attention that you are knowledgeable in this area. (PERSONAL INFORMATION-including titles of their publications, speeches, or inventories) I am trying to get expert opinions and some consensus on what components or elements are needed in learning style inventories, and I would like to ask you to participate in a Delphi Study on cognitive/learning style inventories for this purpose. It is my goal to be a facilitator and bring together the best information about learning style inventories and share that information with the people in the field. Additionally, I hope to stimulate interest in the area by creating a practical tool for educators to use to try to determine what learning style inventory they should use in a given situation. Finally, I hope to gain some insight about‘ what inventories are being used for what purposes. If you choose to participate, I will send you a copy of the results of this study. These results should give you an idea of what your colleagues are thinking and doing about cognitive/learning style assessment inventories. Of course your participation is voluntary and you may 63 choose to answer or not answer any question or to withdraw from the study at any time. The first questions that will be posed to you are to list the technical and content components or elements that you believe are important in a learning style inventory and to check off any inventories that you have used or had familiarity with. This question will be sent to you. After you and other experts have responded to this question and the data has been processed, another set of questions based on the results from the first question will be sent to you. The last question will ask you to evaluate a learning style inventory which you are familiar with using a checklist developed from the previous rounds of the Delphi. It is anticipated that there will be 3 or 4 rounds with each round taking 30 minutes or possibly a little more of your time. There will be approximately 3 to 4 weeks between each round. At the latest, all rounds should be completed by early May. Of course, all responses will be treated with strict confidence and all names will remain anonymous. It is anticipated that this research will add to the existing body of knowledge in the field, and will result in a practical tool that educators can use in the field. Would you be willing to participate? If The Answer Was Yes Thank you for your help. Your first mailing will be in about two weeks. I will be looking forward to your response. If you have any questions, you may call me collect at 517-355-8229 or write me at 9248 Cherry Lane, E. Lansing, MI 48823. I will also include this phone number and address on all my mailings. Do you know names of other people who are knowledgeable in this area that you think might be willing to participate? Thank you again. If The Answer Was No Thank you for your time. If you think it over and change your mind, please feel free to call me collect at 517-355-8229. Do you know any names of other people who are knowledgeable in this area that you think might be willing to participate? Thank you again. From the list of 60 experts , 41 agreed to participate in the study. To encourage continued participation over the 3 rounds, postcards were sent to remind participants that 64 they had not returned their survey. However, the actual numbers of participants for each round were: 36 participants for Round 1, 33 participants for Round 2, and 28 participants for Round 3. All Round 2 participants had been included Round 1. Similarly, all Round 3 participants had been included in both previous rounds. The sample was not selected randomly as this study was looking for expert opinions and most of the people who qualified as experts either worked or were affiliated with a university or college. The sample was selected using accepted criteria for choosing a Delphi panel. ...participants must have a deep interest in the problem and important knowledge or experience to share....Once the general characteristics of desired respondents are agreed upon, a nomination process should be used to select specific individual respondents. In other words, the work group should first identify target groups likely to possess relevant information or experiences concerning the objective toward which decision makers are aiming the Delphi. The staff should solicit nominations of well-known and respected individuals from members within the target groups if the Delphi is aimed at experts...(Delbecq et al., 1975, p.88). By using journal articles, ERIC listings, pissentetion Apstracts International, Psychological Abstracts, and recommendations from other participants, the nominations came from varied resource bases. Using these resource bases assured that the participants would be knowledgeable in the area, and have some interest in the area. The sample size of each Delphi Round varied from 28 to 36. Delbecq (1975) states the sample size of a Delphi can vary from 10 to 15 in a homogenous group to several hundred. However, "Our experience indicated that few new ideas are 65 generated within a homogeneous group once the size exceeds thirty well-chosen participants (p. 89). Since this group is homogenous in terms of their area of expertise and well- chosen, the numbers match the criteria for Delphi studies. INSTRUMENTATION Reiiapiiity and Validity of Instruments According to Linstone and Turoff (1975) the validity of the Delphi is in the procedure. The procedure is about as pure and perfect a Lockean procedure as one could ever hope to find because, first, the 'raw data inputs' are the opinions or judgments of the experts: second, the validity of the resulting judgment of the entire group is typically measured in terms of the explicit 'degree of consensus' among the experts (p. 22). Also content-related validity was used in this study. Content-related validity is the degree to which a sample of items represents the content that the questionnaire is designed to measure. A literature review and expert opinions (American Psychological Association, 1985) are 2 of the methods used as the basis for content-related validity. For Round One both an expert opinion survey and a literature search was conducted to sample the content of cognitive/learning style assessment inventories. Rounds 2 and 3 were built from the expert opinions from the previous rounds. 66 Design Round i Delphi Items for the questionnaire of Round 1 of the Delphi were developed by using a literature review. According to Turoff (1975) most Delphis try to maintain a 3 to 4 round limit in order to keep the participation of the experts. This 3 to 4 round limit is accomplished, in part, by preformulating the obvious issues and preceding the initial list with a range of options but allowing the experts to add to this initial list. Generally, the range of options or the preformulation is accomplished by reviewing the literature (Moore 1987; Young 1977: Linstone & Turoff 1975; Weatherman & Swenson 1974: Sweigert & Schabacker 1974). There were 4 parts to Round 1 (see Appendix A). The first part asked the experts what cognitive/learning style assessment inventories they had used, what age group they used them with, and when they last used them. It also asked what cognitive/learning style assessment inventories they were familiar with but had not used. Eighteen inventories were listed, and experts were encouraged to add any inventories not on the list. Part 2 of Round 1 (see Appendix A) asked the experts to determine which cognitive (content) elements should be measured by a cognitive/learning style assessment inventory. Different elements of physical, affective, sociological, sensory, and inference components were listed. Participants 67 were asked to add any additional elements that they thought were important. Any elements that participants thought were important, they marked with an "X". Part 3 of Round 1 (see Appendix A) asked the experts to determine which technical (operational) elements should be measured by a cognitive/learning style assessment inventory. Eight major elements were listed. They were: (1) time needed to test, (2) test booklet, (3) total cost (scoring, booklet, score sheets), (4) manuals, (5) what manuals should report, (6) inventory administration procedures, (7) scoring, and (8) student response to inventory. Under each element, subelements were listed for the experts' consideration. Participants were asked to add any additional technical elements that they thought were important. Any elements the participants thought were important, they were asked to mark with an "X”. Part 4 of Round 1 (see Appendix A) asked the experts to define 3 terms and complete 2 open-ended questions. They were asked to define learning style, cognitive style, and inventories. The 2 open-ended questions were: (1) The reason(s) I use or have used the inventories is (are), and (2) My disappointment(s) when using the inventories is (are). Participants were asked not to leave blank spaces. If they felt unprepared to answer any question or term they were asked to draw a line to indicate that they had seen the question and decided not to answer it. If they felt any of the terms were the same, they were asked to define the first 68 term and write same as for the second or third term, (see Appendix A). The Round 1 questions were sent to participants on February 26, 1990. Each respondent received: a cover letter, the Delphi questions and a self addressed stamped envelope. The cover letter asked participants to copy their responses if they wanted to keep track of their answers, and to return the survey by March 7, 1990. On March 13, 1990 postcards were sent out to 18 participants who had not responded initially to remind them to return the survey. After the postcards, 13 more participants returned the survey making a total of 36 participants. 2 hi The items in Round 2 were developed using the data from the Round 1 responses. In the Delphi technique, The procedure calls for iteration in eliciting perceptions from participants, so that they make a series of judgments, each successive one being made in the light of a summary of judgments of all participants on the previous round. This process is designed to produce increasing accuracy of judgment and increasing agreement among participants from round to round. (Sweigert and Schabacker, 1974, p. 2) There were seven major parts to Round 2 (see Appendix B). All items selected and added by the experts in Round 1; parts 1, 2, and 3 were listed out and the experts were asked to rank the items. For each of the major cognitive elements of cognitive/learning style (physical, affective, sociological, sensory, and inference), and for each of the 69 major technical elements of cognitive/learning style (time needed to test, test booklet, total cost, manuals, manuals should report, inventory administration procedures, Scoring, and Student response to inventory) the experts were given 10 points to distribute among subelements of each major element. This was done for parts 1 and 2. Participants were also given 10 points for 2 blocks in part 3 to rank the major elements of style in the cognitive and technical areas. Participants were given these directions for parts 1, 2, and 3: In each block you have 12 POINTS. Distribute your points so that the greatest amount of points goes to the element(s) that you think are the most important, and the least amount of points goes to the element(s) that you think are the least important. Some elements within a block can have the same amount of points, "O" points, or one element in a block can have all 10 points. Use only whole numbers. Parts 4, 5, 6 and 7 asked the experts to rank specific items: Part 4-characteristics of learning style, Part 5-characteristics of cognitive style, 4 Part 5-characteristics of cognitive/learning style inventories, Part 6-reasons to use cognitive/learning style inventories, and £e;t_1—disappointments with cognitive/learning style inventories. In parts 4, 5, 6, and 7, participants were given the same directions that they were given in parts 1, 2 and 3 exeept they were given ;5 points to distribute. Round 2 was sent to 70 the participants on April 9, 1990. Each mailing included a cover letter, the Delphi questions and a self addressed stamped envelope. The cover letter asked participants to copy their responses if they wanted to keep track of their answers, and to return the survey by April 16, 1990. The cover letter also gave a summary of the Round 1 responses. On May 2, 1990 postcards were sent out to 11 participants to remind them to return the survey. After the postcards, 4 more participants returned the survey. One person withdrew from the study. A total of 33 responses were received for the second round. Rennd 3 pelpni The questions in Round 3 were developed from the responses in Round 2 (see Appendix C). Only 1 new item was added. It was added as a response to a written comment from one of the participants. Participants were given ranked lists of elements, subelements, disappointments with, and reasons for using cognitive/learning style assessment inventories (the ranking was developed from the previous rounds). Additionally, asterisks divided 2 rankings that were separated by 10 or more points. Then participants were asked to comment on each ranking if they desired to comment. If they did not comment on any ranking, that meant they agreed with the ranking. There were 18 blocks of ranked items. Some blocks were very long containing as many as 48 items. 71 Round 3 materials were sent to participants on September 2, 1990. Each mailing included a cover letter, the Delphi questions, and a self addressed, stamped envelope. As in previous rounds, the cover letter asked participants to copy their responses if they wanted to keep track of their answers, and to return the survey by October 1, 1990. The cover letter gave a summary of Round 2. The total number of returns were 28. On October 7, 1990, postcards were sent out to 11 participants to remind them to return the survey. After the postcards, no new surveys were returned. The total percentages of returns were 87.8%, 80.5%, and 68.3% for the 3 rounds. The time lag between Round 2 and Round 3 took into consideration that professors are often gone over the summer months. Also the extended time given to the experts to respond to Round 3 reflected comments by the experts that they did not have enough time to respond on the 2 previous rounds. DATA COLLECTION PROCEDURES Resources Regnired Since the Delphi is a survey done usually in at least 3 rounds it required chiefly only the resources of this researcher who was the compiler and manager of the information. Members of the researcher's dissertation committee provided input to the revision of the questions for each round. A typist was used to type and format the 72 surveys. Sent to the participants were surveys, reminder postcards, and self addressed stamped envelopes. Additional envelopes were used to mail the Delphi materials to the participants. Trainin Re ired This researcher had to review the literature on cognitive/learning style assessment inventories and on the Delphi Technique to: 1. Learn how to direct a Delphi Study, 2. Determine what cognitive/learning style elements needed to be in the initial survey, 3. Determine what cognitive/learning style assessment inventories were popular and available to include in the first survey, 4. Determine what were some of the disappointments with cognitive/learning style assessment inventories to include in the first survey, and 5. Determine what were some of the reasons practitioners used cognitive/learning style assessment inventories to include in the first survey. The most important resource needed in any Delphi Study, and this research was no exception, was time. This process took more time than most survey methods because it consisted of 3 separate rounds of data collection. 73 PLAN FOR ANALYZING DATA Stetispicai Treatment of Data Validity Because Round 1 questions were developed by this researcher, the question of validity must be addressed. Traditionally, evidence of validity evidence has been grouped into 3 major categories (American Psychological Association, 1985). These categories are content-related, criterion- related, and construct-related. Criterion-related and construct-related validity were tested by this researcher because of insufficient data from previous studies and lack of available instruments to test the variables in this study. Content-related validity is the degree to which a sample of items represents the content that the questionnaire is designed to measure. Unlike other types of validity, content-related validity is tested subjectively (Borg & Gall, 1979). A literature review and expert opinions (American Psychological Association, 1985) are 2 of the methods used as the basis for content-related validity. For Round 1 an expert opinion survey and a literature search were conducted to sample the content of cognitive/learning style assessment inventories. Rounds 2 and 3 were built from the expert opinions from the previous rounds. W]. In Round 1 the questions were developed by using a 74 literature search and expert opinions. Computer searches of E19. WW. and We; Abstnects were used to help develop the questions. A manual search of ERIC was done to include any literature that was written before 1966 when the computer listings began in ERIC. A number of books written about cognitive/learning style assessment inventories were reviewed by this researcher. Manuals and test booklets of cognitive/learning style assessment inventories were reviewed in order to develop some of the cognitive and technical questions for the first round. Experts that were contacted by telephone added material to be used in the questions for Round 1. Experts on this researcher's dissertation committee also suggested questions for Round 1. The expert opinions and the literature search met the guidelines for content-related validity according to Borg and Gall (1979) and the American Psychological Association (1985). Round 1 had some background questions about what cognitive/learning assessment inventories the experts had used, when the experts had used the inventories, and with what age groups the experts had used the inventories. These responses were tabulated to determine widely used inventories and information about their use. These responses were not used in Rounds 2 and 3. Round 2 Data Analysis All responses from the experts in Round 1 were recorded. Duplicate responses were discarded. Responses that had 2 or more ideas were broken apart so that there was only 1 idea 75 per statement. Then the responses were arranged into categories already established in Round 1 (see Appendix B). No rank, order or weight was given to any of the responses. All responses were sent back to the original list of 41 experts. The instructions to the experts read: Below are the responses you chose in Round 1. In each block you have ;Q or 2; (depending on the block) POINTS.Distribute your points so that the greatest amount of points goes to the element(s), characteristic(s), reason(s) to use, or disappointment(s) with (depending on the block) that you think is/are the most important, and the least amount goes to the element(s) that you think is/are the least important. Some elements in the block can have the same amount of points, "O"(zero) points or one element in a block can have all 10 points. Use only whole numbers. (see Appendix B) When the 33 responses were returned, all the points in each block were totaled. Responses were put in a ranked order. If more than 10 points separated any 2 items, the items were separated by a row of asterisks. All the responses receiving "O"(zero) points were recorded as receiving "O"(zero) points. Round 3 Data Analysis The ranked list was sent out to the 41 original participants. The directions that were sent out to the participants were: ' Here is the last Delphi Survey on cognitive/learning style inventories in which you have been participating. Please read the results and make appropriate comments, if desired, in the spaces provided. Note: A ranking of "1" indicates the element or characteristic that received the most points as determined by the respondents. In addition, a line of asterisks dividing two rankings means that there are more than 10 points separating those two rankings. Descriptive statistics were used in this research to 76 report the results of the Round Three questionnaire. Results are reported in percentages, mean, and median. A narrative was used to give minority, opposing, or differing positions that the experts expressed in the comment section. Decision Rules for Interpreting Data All responses by the experts were recorded and reported. Responses that received "0" (zero) responses in Round 1 were deleted. Responses that received "0" (zero) responses in Rounds 2 and 3 were reported. They were reported to give feedback to the participants and were not used in the statistical process to remain consistent with Round 1. At decision making junctures in the flow chart, criteria were given for each decision point (see flow chart in Chapter 5). Presentational Format Used to Present Findings Tables, a flow chart and matrices in conjunction with a narrative were used to illustrate findings. SUMMARY In this chapter this researcher outlined the research design. The purpose of developing a model for evaluating and choosing among widely used assessment inventories of cognitive/learning style in order to aid practitioners in selecting an appropriate assessment inventory was stated. How experts were chosen and contacted for this Delphi, which 77 was the research methodology in this study, were outlined. The mailing dates of each round of the Delphi study were given. The population of this Delphi study was defined. The primary and subquestions of this research were listed and reasons were given for the individual questions and subquestions. Areas of the literature that were reviewed and searched for this study were given. The 3 areas of literature reviewed for this study were: the Delphi Method, cognitive/learning style, and cognitive/learning style assessment inventories. The data bases and other literature that was searched was discussed. The educational background, degree status, content area specialty, geographical location, and gender of the 41 participants of the study were described. The procedure for obtaining participants by using the literature was outlined. The phone contact speech was described. The number of Participants in each round of the Delphi study was given. Justification of the nonrandom sampling procedure was Presented. Data collection procedures were reported. Resources and training required for collecting the data were outlined. Instrumentation design and development were outlined. How the questions for each of the 3 rounds of this Delphi were developed and specific areas the questions covered were discufist-.d. Mailing dates for the each round and the reminder po""‘tc<'=11:‘