C ‘3.”- FI _S_. 2 per ‘Ricn': .t. at.— my L E”§.;":i"~G zessx;\__.v__.= E- ':- ‘. bu: rs THE PROCESS AND OUTCOMES OF DIAGNOSTIC PROBLEM SOLVING AMONG EIGHT READING CLINICIANS By Ethelyn Maxwell Hoffmeyer A DISSERTATION Submitted to Michigan State University in partial fulfillment of the requirements for the degree of DOCTOR OF PHILOSOPHY Department of Elementary and Special Education 1980 © Copyright by ETHELYN MAXWELL HOFFMEYER l980 ABSTRACT THE PROCESS AND OUTCOMES OF DIAGNOSTIC PROBLEM SOLVING AMONG EIGHT READING CLINICIANS By Ethelyn Maxwell Hoffmeyer If correct diagnosis of reading problems is a prerequisite of effective remediation, then one approach toward improving remedial practices might be to improve the diagnostic skills and training of reading clinicians. One effort to study clinical diagnosis in medicine and subse- quently in reading has been research relating to the Inquiry Theory .of Clinical Problem Solving. The Inquiry Theory was developed by a team of researchers at Michigan State University to provide a formal theoretic structure that would integrate and account for the numerous concepts and empirical findings on clinical problem solving. The major objective of this study was to answer the follow- ing questions relative to the Inquiry Theory: l. Do experienced reading clinicians agree on the data they collect for a specific reading case in order to make a diag- nosis? 2. Do experienced reading clinicians agree on the diagnos- tic statements they make for a specific reading case in order to make a diagnosis? Ethelyn Maxwell Hoffmeyer 3. Do experienced reading clinicians use hypotheses to direct their inquiry in diagnosis? Eight experienced reading clinicians from the mid-Michigan area participated in this study. Each clinician in three clinic sessions, no less than one nor more than four weeks apart, interacted with three simulated reading problem cases. The third case, unbeknown to the clinicians, was a replicate of the first case. The four simu- lated cases and their equivalent forms used in the study were counter- balanced to minimize systematic effects. Clinicians were randomly assigned to case order. Procedures were as follows. Clinicians were asked to: l. interact with materials of a simulated reading problem case, 2. write a diagnosis and remediation, 3. transfer the written diagnosis to the Reading Diagnostic Check List, 4. check responses to questions concerning why_they asked for certain case data and what information the data provided, 5. indicate the content of a "good" diagnosis, 6. explain how they usually conduct a diagnosis, and 7. define for a specific case the skills of (a) instant 'word recognition, (b) word analysis, (c) reading fluency, and (d) reading comprehension. To measure subject reliability on use of the Reading Diagnos- tic Check List, the clinicians were mailed an uncircled, carbon copy of each of their written diagnoses one week after the third clinical Ethelyn Maxwell Hoffmeyer session. An accompanying letter instructed subjects to follow the same procedures they had used in the clinical session for transfer- ring their written diagnostic statements to the check list. Analysis of the data consisted of (l) formal product measures (including proportional agreement, commonality scores, inter/intra- and intraclinician agreement Phi correlation, and the Porter statis- tic), (2) formal process measures (using correlation, partitioned Phi coefficients, and cue-to-statement relationship statistics), and (3) informal product/process measures (using Sherman's Model of Read- ing and Learning to Read). Regardless of the small sample of reading clinicians partici- pating in this study, there is evidence to support a number of con- clusions. These conclusions are: l. Experienced reading clinicians using simulated reading cases appear not to share a common data base (memory) regarding what information (cues) should be included in a diagnosis or what diag- nostic statements are important in writing a diagnosis. 2. Experienced reading clinicians using simulated reading cases appear not to share a common diagnostic routine (strategy) in terms of how to go about a diagnosis. 3. Experienced reading clinicians using simulated reading cases appear not to use consistently a theoretic process model of reading diagnosis as might be reflected in hypothesis-directed inquiry. This dissertation is dedicated with admiration and apprecia- tion to Dr. Byron VanRoekel. It was he who first sparked my interest in the reading profession. His love and concern for children, his understanding of the many facets of the reading process, and his interest in sharing his skills to teach reading teachers and clini- cians have been an inspiration to me throughout my graduate school and teaching years. To Dr. Van Roekel, a heartfelt thank-you for the many years of dedicated work with the children and reading specialists who were and are a part of the Michigan State University Reading Clinic. ii ACKNOWLEDGMENTS I wish to express my sincere appreciation to those individuals who have been an inspiration and a help to me throughout this research endeavor: Dr. John Vinsonhaler, who directed my research, for his con- fidence in my ability and for sharing his many research skills and his enthusiasm in the Clinical Studies Project. Dr. George Sherman, my doctoral committee chairman, for his faithful support. His ability to communicate his knowledge and understanding of the reading process helped me develop many ideas that went into this dissertation. Dr. Lawrence Lezotte, doctoral committee member, for his interest in my work and his insightful suggestions. His sunny dis- position and encouragement helped me forge ahead and complete this study. Dr. Glen Cooper, doctoral committee member, for his interest and his helpful comments. Dr. Donald Freeman, for his scholarly and creative editorial comments. His suggestions were invaluable in the completion of this dissertation. Dr. Andrew Porter, who developed the research design for this study and who gave unselfishly his valuable time and expertise. iii Mrs. Ruth Polin, of the Clinical Studies staff, for her patient assistance with the computer terminal and the data analyses. Mr. Christian Wagner and Ms. Francine Kitchen, of the Clini- cal Studies staff, who designed the computer programs used for data analysis and assisted me in those analyses. Dr. Lois Bader, for sharing her expertise in reading and helping me formulate many ideas for research. Dr. C. Jay Stratoudakis, for her help in piloting the pro- cedures used in this study. Mrs. Jerri Zemper and Ms. Debie Salters, for their technical assistance in preparing the dissertation proposal. Dr. Keith Goldhammer, Dean of the College of Education and my favorite teacher, for providing me with the incentive to strive for excellence. Mrs. Susan Cooley, for her professional expertise in editing and typing this manuscript. Her organization and her ability to work calmly under pressure are amazing. My husband, Tom, our children, Kevin and Kelly, and my parents, Mr. and Mrs. Ralph Maxwell, for their sacrifices, their tolerance, and their encouragement. iv TABLE OF CONTENTS Chapter I. THE PROBLEM ...................... Introduction .................... Purpose ....................... Theory ....................... Introduction ................... The Clinical Encounter .............. The Basic Management Information System (BMIS) . Theoretic Implications .............. Summary ....................... Overview ...................... II. REVIEW OF THE LITERATURE ............... Introduction .................... Research Procedures in Problem Solving ....... Total-Task Studies ................ In-Basket Studies ................. Tab-Item Methods ................. Problems Inherent in Reading Diagnosis ....... The Essence of Reading Diagnosis ......... The Debate Over Causation ............. Lack of Standardized Terms ............ The Inquiry Theory: Related Research ........ Introduction ................... The Medical Inquiry Project ............ Clinical Information Processes in Reading (CLIPIR) . Observational Study, l977 ............. Outcomes, l977 .................. CLIPIR Application Exploratory Study in Educational Research (CAESER), 1977 ....... Chapter Page Implications, 1976-1977 .............. 46 Recent Studies .................. 47 Summary ....................... 47 III. DESIGN AND PROCEDURES ................. 50 Introduction .................... 50 Sample ....................... 50 Research Design ................... Sl Instrumentation and Data Collection ......... 54 Reliability .................... 55 Instruments .................... 59 Statement Concerning Hypotheses ........... 60 Procedures ..................... 61 Summary ....................... 64 IV. ANALYSIS MEASURES AND RESULTS ............. 66 Product Measures .................. 67 The PrOportional-Agreement Statistic ....... 68 The Commonality-Agreement Score .......... 74 Inter/Intraclinician Agreement .......... 76 Product Measures: Summary ............. 87 Process Measures .................. 89 Basic Process Statistics ............. 9l Process-Agreement Statistics ........... 94 Cue-to-Statement Relationship Statistics ..... 99 Process Measures: Summary ............. ll2 The Informal Product/Process Measures ........ llS Informal Product/Process Measures: Summary . . . . 123 Summary ....................... l25 V. SUMMARY, DISCUSSION, CONCLUSIONS, AND RECOMMENDATIONS . l27 Summary ....................... l27 Introduction ................... l27 The Problem .................... l28 Review of the Literature ............. l29 Design and Procedures ............... l30 Analysis Measures and Results ........... 133 Discussion ..................... 137 Study of Product ................. l37 Study of Process ................. l43 Biasing Effects .................. 144 Conclusions ..................... l48 Training ..................... l49 Check List .................... lSl Recommendations ................... l52 vi Chapter APPENDICES BIBLIOGRAPHY vii Table 10. ll. 12. 13. LIST OF TABLES Correlation (rxy) and Porter Index for Clinic and Home Conversion of Written Diagnosis to RDCL on Four Cases ...................... Means and Standard Deviations for Subjects Based on Correlation (rxy) for Use of RDCL ........... Means and Standard Deviations for Cases Based on Correlation (rxy) for Use of RDCL ........... Proportion of Clinician Agreement on Most Frequently Collected Cues, Case l ........... Percentages for Proportion of Agreement on Cues Most Frequently Requested for Four Cases .......... Diagnostic Statements Most Frequently Selected From a Standard Check List .................. Percentages in Proportion of Agreement for Most Frequently Mentioned Diagnostic Statements on Four Cases ...................... Commonality Agreement on Cues Collected and Diagnostic Statements Made in Six Diagnoses on Each of Four Cases ...................... Means and Standard Deviations for Inter/Intraclinician Agreement (Phi and Porter) on Cues for Four Cases . . . Means and Standard Deviations for Inter/Intraclinician Agreement (Phi and Porter) on Diagnostic Statements for Four Cases .................... Means and Standard Deviations for Intraclinician Agreement for Cues Collected on Four Cases ...... Intraclinician Agreement for Two Conversions (Clinic and Home) of Written Diagnoses to the RDCL ...... Means and Standard Deviations for Intraclinician Agreement for Diagnostic Statements on Four Cases . . . viii Page 58 58 69 71 72 73 75 79 79 80 82 83 Table Page 14. Agreement Statistics on Cues Collected for Four Simulated Reading Cases ................ 86 15. Agreement Statistics for Diagnostic Statements Made on Four Simulated Reading Cases ......... 87 16. Basic Process Statistics Data for Four Simulated Cases in Reading ................... 92 17. Means and Standard Deviations for Value of r_on Egg Times, Including Missing Data, for Four Cases ..... 96 18. Means and Standard Deviations for Value of r_on Diagnostic Statement Times, Including Missing Data, fOr Four Cases .................... 97 19. Inter/Intraclinician Phi on Four Time Partitions for Diagnostic Statements and Four Cases ......... 99 20. Average of Percentage of Number of Times Each Relationship Appeared for Cues-~Case 1 ........ 103_ 21. Average of Percentage of Number of Times Relationships Appeared for Cues Using Hypothesis-Directed and Cue-Directed Inquiry Responses Across All Subjects and All Cases ..................... 106 22. Means and Standard Deviations Across All Subjects and Cases for Average of Percentage of Number of Times Relationships for Hypothesis-Directed and Cue- Directed Inquiry Responses Appeared for Cues ..... 107 23. Average of Percentage of Number of Times Each Relationship Appeared for Diagnostic Statements-- Case 1 ........................ 109 24. Average of Percentage of Number of Times Relationships Appeared for Diagnostic Statements Using Hypothesis- Directed and Cue-Directed Inquiry Responses Across All Subjects and All Cases .............. 110 25. Means and Standard Deviations Across All Subjects and Cases for Average of Percentage of Number of Times Relationships for Hypothesis-Directed and Cue- Directed Inquiry Responses Appeared for Diagnostic Statements ...................... 111 26. Sample 1 of Responses to Informal Questions Relating to Memory (1) and Strategy (2)--Subject 104 ...... 118 ix Table 27. 28. 29. 30. A1. 01. 02. 03. D4. 05. D6. D7. D8. D9. D10. D11. Sample 2 of Responses to Informal Questions Relating to Memory (1) and Strategy (2)--Subject 106 ...... Sample of Responses to Four Questions Regarding Specific Aspects of Reading~-Subject 104, Run 3, Case 4 ........................ Results of Responses by Eight Clinicians to Specific Questions Regarding Four Areas of Reading for Four Cases ...................... Agreement Statistics on Cues Collected and Diagnostic Statements Made by Eight Clinicians in Each of Two Studies Using Four Simulated Reading Cases ...... Background Information on Reading Clinician Subjects Subject Responses to Informal Questions Regarding Memory (1) and Strategy (2)--Subject 101, Case 3 Subject Responses to Informal Questions Regarding Memory (1) and Strategy (2)--Subject 102, Case 2 Subject Responses to Informal Questions Regarding Memory (1) and Strategy (2)--Subject 103, Case 1 Subject Responses to Informal Questions Regarding Memory (1) and Strategy (2)--Subject 105, Case 3 Subject Responses to Informal Questions Regarding Memory (1) and Strategy (2)--Subject 107, Case 1 Subject Responses to Informal Questions Regarding Memory (1) and Strategy (2)--Subject 108, Case 2 Subject Responses to Informal Questions Regarding Specific Aspects of Reading--Subject 103, Case 1 Subject Responses to Informal Questions Regarding Specific Aspects of Reading--Subject 107, Case 1 Subject Responses to Informal Questions Regarding Specific Aspects of Reading--Subject 102, Case 2 Subject Responses to Informal Questions Regarding Specific Aspects of Reading--Subject 108, Case 2 Subject Responses to Informal Questions Regarding Specific Aspects of Reading--Subject 101, Case 3 X Page 119 120 122 139 160 202 203 204 205 206 207 208 209 211 212 214 Table Page 012. Subject Responses to Informal Questions Regarding Specific Aspects of Reading--Subject 105, Case 3 . . . 216 013. Subject Responses to Informal Questions Regarding Specific Aspects of Reading--Subject 106, Case 4 . . . 217 61. Procedures for Two Studies of Diagnostic Decision Making Using Simulated Cases and Eight Clinicians . . 228 H1. Proportion of Agreement on Cues, Case 2 ........ 230 H2. Proportion of Agreement on Cues, Case 3 ........ 231 H3. Proportion of Agreement on Cues, Case 4 ........ 232 H4. Percentages in Total Proportion of Agreement (P.A.) on Cues for Four Cases ................ 233 H5. Percentages in Total Proportion of Agreement (P.A.) on Diagnostic Statements for Four Cases ....... 234 H6. Commonality of Agreement on Cues Collected in Six Diagnoses on Each of Four Simulated Reading Cases . . 235 H7. Commonality of Agreement on Diagnostic Statements Made in Six Diagnoses on Each of Four Simulated Reading Cases .................... 236 H8. Inter/Intraclinician Agreement (Phi and Porter) on Cues Collected on Four Simulated Reading Cases . . . . 237 H9. Inter/Intraclinician Agreement (Phi and Porter) on Diagnostic Statements Made on Four Simulated Reading Cases .................... 238 H10. Correlation Coefficient for ggg_Times, Including Missing Data, for Case 1 ............... 239 Hll. Correlation Coefficient for Cug_Times, Including Missing Data, for Case 2 ............... 239 H12. Correlation Coefficient for §gg_Times, Including Missing Data, for Case 3 ............... 240 H13. Correlation Coefficient for ng_Times, Including Missing Data, for Case 4 ............... 240 H14. Correlation Coefficient for Diagnostic Statement Times, Including Missing Data, for Case 1 ...... 241 xi Table Page H15. Correlation Coefficient for Diagnostic Statement Times, Including Missing Data, for Case 2 ...... 241 H16. Correlation Coefficient for Diagnostic Statement Times, Including Missing Data, for Case 3 ...... 242 H17. Correlation Coefficient for Diagnostic Statement Times, Including Missing Data, for Case 4 ...... 242 H18. Average of Percentage of Number of Times Each Relationship Appeared for Cues--Case 2 ........ 243 H19. Average of Percentage of Number of Times Each Relationship Appeared for Cues--Case 3 ........ 244 H20. Average of Percentage of Number of Times Each Relationship Appeared for Cues--Case 4 ........ 245 H21. Average of Percentage of Number of Times Each Relationship Appeared for Diagnostic Statements-- Case 2 ........................ 246 H22. Average of Percentage of Number of Times Each Relationship Appeared for Diagnostic Statements-- Case 3 ........................ 247 H23. Average of Percentage of Number of Times Each Relationship Appeared for Diagnostic Statements-- Case 4 ........................ 248 H24. Individual Case Data (Mean and Standard Deviation) on Porter Statistic, Including Number of Cues and Diagnostic Statements in Domains, for Two Studies . . 249 xii LIST OF FIGURES Figure Page 1. The Clinical Encounter ................. 7 2. The Clinical Interaction ................ 7 3. The Clinical Case ................... 9 4. The Clinician ..................... 11 5. Random Assignment of Subjects to Clinical Sessions . . . 53 6. Research Design .................... 54 Bl. Diagram of Clinical Setting .............. 163 11. Contingency Table for Calculation of Phi--Examp1e 1 . . 254 12. Contingency Table for Calculation of Phi--Example 2 . . 254 13. An Example of a Completed Contingency Table ...... 255 I4. Contingency Table for Calculation of Intra Phi-- Example 1 ...................... 257 15. Contingency Table for Calculation of Intra Phi-- Example 2 ...................... 258 16. An Example of a Completed Contingency Table ...... 259 xiii Appendix A. on 2120'” LIST OF APPENDICES CLINICIAN BACKGROUND FORM AND INFORMATION ...... CLINICAL SETTING AND INSTRUCTIONS .......... CLINICAL OBSERVATION INSTRUMENTATION ......... CLINICIAN RESPONSES ON FORMAL AND INFORMAL MEASURES EXAMINER'S NOTES FORM AND LETTER TO SUBJECTS ..... CUE DOMAIN SAMPLE FOR A CASE ............. PROCEDURES FOR TWO STUDIES: OS '77 and OS '78.3 . . . TABLES ........................ OBSERVATIONAL STUDY DATA-ANALYSIS SYSTEM (OSDAS) STATISTICS ..................... xiv CHAPTER I THE PROBLEM Introduction DevelOpments in the field of reading in recent years have called for more effective and efficient diagnostic techniques and remediation practices (Chall, 1978). Reading researchers have responded to this challenge with a continuing interest in studies of how reading diagnosticians should behave or perform. However, research slights the problem of how reading diagnosticians think abggt_their students' reading problems (Shulman & Elstein, 1975). Assuming that correct diagnosis of reading problems is impor- vtant for effective remediation, diagnostic acumen is highly desirable for reading diagnosticians. It might then follow that one approach toward improving the success of therapy or remediation in clinical diagnosis of reading problems would be to improve the diagnostic skills and training of reading clinicians (Hoffmeyer 8 Bader, 1978; Hoffmeyer, 1979). At this time, there is little documented evidence of how pre- viously trained and currently trained reading diagnosticians think about their cases. Although there is evidence that some clinical diagnosis is effective in terms of remediation that results in improved reading behavior (Spache, 1976), the reasoning skills that characterize "expert" reading clinicians have not been identified. 1 Effective diagnostic procedures might be determined through the study of the diagnostic decision-making and problem-solving behavior of expert reading clinicians. Additionally, once these clinical reasoning skills are identified, training programs in read- ing designed to teach more effective clinical skills might be devel- oped. Reading diagnosticians might be taught how to seek answers to pertinent questions and to determine what information is of prime importance and what information need not be dealt with for a given reading problem. Recent studies have shown this to be effective in teaching clinical skills to physicians (DeDombal et al., 1974; Elstein, 1975; Elstein, Shulman et al., 1976). Until recently, studies of clinical problem solving and decision making have involved research in medicine (DeDombal et al., 1972, 1974; Schwartz, 1973). However, in a thorough review of a number of theoretical models and research methods on thinking, human judgment, and decision making, Shulman and Elstein (1975) determined that those studies of information processing, decision making, policy capturing, and lens model "have rarely been applied to the investiga- tion of educational problems" (p. 32). They then pr0posed a variety of ways in which the models and methods reviewed might be valuable in education. One major area of their discussion dealt with possi- bilities of research in diagnosis and remediation of reading diffi- culties. Shulman and Elstein stated that "as in so many situations involving clinical judgment, the principles governing decision making are typically unclear" (p. 34). They concluded that it is crucial to examine hgw_the problem solver "sizes up the situation, how the problem is formulated, what is judged to be relevant and what irrele- vant, which sources of information are considered useful and which of no importance" (p. 37). To move in the direction proposed by Shulman and Elstein, there is a need to observe and study the decision-making and problem- solving behavior of ”expert" reading clinicians so as to distinguish those reasoning skills that characterize them as "experts." Then, as in medicine (DeDombal et al., 1972), attempts might be made to use these "experts" as models, by means of simulation, in teaching effective clinical diagnosis and remediation in reading. These previous studies in medicine, dealing with the investi- gation of clinical problem solving (Barrows et al., 1976; Elstein, Shulman, Sprafka et al., 1978; Vinsonhaler, Wagner,l&Elstein, 1977) provided the foundation for research on clinical diagnosis in reading. From these earlier studies, three basic principles have directed the investigation of reading clinicians: First, the recognition of the practical value of a well under- stood theoretic base for empirical research on clinical problem solving; second the recognition of the efficacy of the methods developed in medicine to examine problem solving under well- controlled conditions using simulated cases and stimulated recall interviews; third, the recognition of the need for a sys- tematic program of research studies which share a common method- ology (Vinsonhaler, 1979a. p. 4). The present study was undertaken as part of a larger research effort by the Clinical Studies group of the Institute for Research on Teaching at Michigan State University. The goal of the Clinical Studies Project was "to better understand, both theoretically and empirically, the clinical skills involved in diagnosing and remediating reading problems" (Gil, Hoffmeyer et al., 1979, p. 12). Further, it was h0ped that this improved understanding would lead to improved instruction and evaluation of reading clinicians and ultimately to more effective and efficient diagnosis of reading problems. Purpose The general purpose of this study was to use the Inquiry Theory of Clinical Problem Solving as a theoretic base in determining hgw_eight experienced, highly trained reading clinicians diagnosed specific reading problem cases (process) and what_information they used in making diagnostic decisions about specific cases (product or outcome). Specifically, the purpose was to test three basic com- ponents of the Inquiry Theory. These components, subsequently explained as corollaries, are (l) the agreement of reading clinicians in collecting data (cues) on a specific case in order to diagnose a reading problem, (2) the agreement of these same clinicians in making diagnostic statements, and (3) reading clinicians' use of hypotheses to direct their diagnostic clinical inquiry. The general main line questions, based on the three components of the Inquiry Theory, that were addressed in this study through jg:- mal_measurement are: 1. 00 experienced reading clinicians agree on the data they collect for a specific reading case in making a clinical diagnosis? 2. 00 experienced reading clinicians agree on the diagnostic statements they make for a specific reading case in making a clinical diagnosis? 3. 00 experienced reading clinicians indicate the use of hypotheses to direct their clinical inquiry in diag- nosis? These three questions are restated more specifically for research purposes in Chapter IV. . Additional secondary-level questions that were addressed in the study through informal assessment are: 1. 00 experienced reading clinicians agree on what informa- tion should be included in a "good" diagnosis? 2. 00 experienced reading clinicians agree on how to go about a diagnosis? 3. To experienced reading clinicians employ a schema in making diagnostic determinations about specific skill areas of reading for a particular case? These questions also are more clearly defined in Chapter IV. The next section contains a description of the Inquiry Theory of Clinical Problem Solving and its related corollaries. Theory Introduction For the last few years, a team of researchers at Michigan State University has been actively involved in studying the behavior of clinical problem solvers and in developing a theory to explain such behavior. The develOpment of a formal theoretical structure that can integrate the numerous concepts and empirical findings on clinical problem solving has become known as the Inquiry Theory of Clinical Problem Solving (Elstein, Shulman, Sprafka et al., 1978). In understanding the important concepts of the Inquiry Theory, it is necessary first to establish the basic definitions and parameters of the theory. Three major assumptions are presently associated with the Inquiry Theory: (1) the clinical-encounter assumption, (2) the simulated-case assumption, and (3) the simulated- clinician assumption. The Clinical Encounter The behavioral domain of the Inquiry Theory is known as the clinical encounter and may be defined as the events that occur as a clinician (e.g., a reading clinician, a teacher, or a physician) attempts to solve a problem in a case (a student, client, patient, or patient record) by making a diagnosis (Dx: What is the problem?) and prescribing a treatment (Rx: What can be done to solve the problem?). The first assumption of the Inquiry Theory is that the clini- cal encounter involves (l) a clinician, (2) a case, and (3) the inter- action that occurs as the clinician analyzes the information provided by the case, makes a diagnosis, and prescribes remediation or therapy. Three basic components, therefore, comprise the clinical encounter: (1) the clinical interaction, (2) the clinical case, and (3) the clinician (Figure l). The clinical interaction.--The clinical interaction, or the reciprocal behavior that occurs between the clinician and the case, is a part of the clinical encounter. Key behaviors that have been observed in the clinical interaction are presented in Figure 2. The direction of interaction is indicated by arrows. The Inquiry Theory attempts to predict only those aspects of the clinical interaction that may be repeatedly observed, i.e., \ CLINICAL INTERACTION Figure l.--The clinical encounter. .15."): MAJOR EVENTS &THE|R SEQUENCE d—IPRINCIPAL COMPLAINTIT ~4cue REQUESTS . n—lcue VALUES l-—-— omewosus }__. oecssnow TREATMENT _____, DECISION ‘1 FOLLOW UP ‘ r DECISIONS Figure 2.--The clinical interaction. the features observed when several clinicians interact with the same case or when a single clinician interacts with one case or with sev- eral cases (Gil, Hoffmeyer et al., 1979; Vinsonhaler et al., 1977a). The major elements in the clinical interaction and their sequence include: 1. a principal complaint (statement of a symptom initiated by the case), 2. cue requests (gathering of case information by the clinician), 3. cue values (the clinician's assessment of the signifi- cance of the information collected in terms of its relationship to the problem), 4. diagnosis decision (or determination of the problem) by the clinician, 5. treatment decision (selection of treatment seen as being most appropriate), and 6. follow-up decision (determination of treatment efficacy). The clinical case.-—The second component of the Inquiry Theory, the clinical case (Figure 3), involves the second assumption of the theory: "Cases can be effectively simulated (able to elicit some of the same problem solving behaviors [i.e., the clinical interaction] as a real case) by providing the clinician with sets of requested information" (Vinsonhaler, 1978, p. 4). An alternative to presenting the behavioral domain of a live client to a clinician is to use simulated cases (SIMCASEs). A simu- lated case is designed to elicit many of the same problem-solving behaviors from the clinician as would a live case. Relevant informa- tion (e.g., physical records, background information, test behaviors) can be collected and stored in a file box (manually based SIMCASE) or in a computer file (computer-based SIMCASE) (Lee & Weinshank, 1976). Case simulation is used in the study of the clinical encounter so as to achieve a level of objectivity that is scientifically accept- able and replicable. THE CASE} 0 Sr: of Problems 8-" C Set of Cuc Name. {C} 0 Set of Cm: Value. {V3 0 Set of Responses to all Potential Treatnents 4..— INTERACTION CLINICIAN Figure 3.--The clinical case. 10 A simulated case (SIMCASE) is a set of data representing a client. The cognitive elements of a case include (1) a set of problems (e.g., sight words inadequate), (2) a set of cue names or items of information in a case that might be used to help a clini- cian make a diagnosis (e.g., Dolch Word List), (2) a set of cue values specifying the client's state with respect to the cue (e.g., 10 per- cent correct on Dolch Word List), and (4) a set of responses to all the potential treatments (e.g., Work with student on developing a basic sight vocabulary through drills, games, etc., using Dolch Word List). Although the validity of this assumption in reading has yet to be obtained because of lack of funding and the legal limitations on use of subjects, it has been shown to be valid in medicine (Taylor, Skakun, & Wilson, 1977), where simulated cases have been widely used. The clinician.--The third Inquiry Theory component, the clini- cian, concerns the third assumption of the theory and involves those factors that govern the behavior of the clinician in the clinical encounter. This assumption states that "the major events in the clinical interaction are determined probabilistically by the CASE and the CLINICIAN'S MEMORY AND STRATEGY" (Vinsonhaler et al., 1977, p. 11). Research findings in medicine support the simulated clinician assumption (see Elstein, Shulman, Sprafka et al., 1978). 11 Clinical memory consists of problem, cue values, prescrip- tion and treatment descriptions, and the relations between them (Figure 4). A CLINICAL CLINICAL MEMORY STRATEGY 0 Set of Problems {P3 Sequence of Inlurmatlon Processing 0 Set of Cue Values {(21 -\ctions, Including: ,1? 0 Relations (.R(C.P)j O Hypoth. Generation 0 Cue Collection 0 Set of Treataentsflt} O Hypoth. Evaluation 0 On Judgment 0 Set of Relations 0 Rx Evaluation . “(,'h)3 NI CLINICAL A_. CLINICIAN : CASE K\\u—“_‘£ :\::::::::::;V Figure 4.--The clinician. A clinical memory example from medicine might be as follows: Problem Representation for Angina Pectoris Elevated blood pressure Chest pain = 1.0 Past history ischemia = 1.00 ECG shows RS-T deviations or T-wave inversions = 1.0 12 Cue Representation for Angina Elevated blood pressure <40 138/80 <20 120/70 Chest pain Does the patient report: (1) chest pain after exercising, e.g., walking up stairs? (2) chest pain after a heavy meal? (3) pain localized behind the sternum or radiating to the left shoulder and arm? Treatment for Angina Pectoris Restricted exercise No smoking Alcohol in moderation Fat-restricted diet Nitroglycerin Prescription for Angina Pectoris Nitroglycerin (Glyceryl Trinitrate) Sublingual tablets (.3 to .6 mg) q. 2 h. to q. 3 h. as required In addition to problem, cue values, and prescription and treat- ment descriptions, clinical memory consists of a number of important relations: 1. a set of relations between cues and problems used to infer the presence of problems in a given case, based on the cues already collected--R(C,P); 2. a set of relations between problems and cues used to determine which cues should be collected next, in order to confirm or disconfirm the hypotheses currently under consideration--R(P,C); 3. a set of relations between problems and treatments used to evaluate and select treatment plans for a given case and diagnosis-- R(P,T); and 13 4. a set of relations between treatments and prescriptions used to define the specifics of case management for a particular case and treatment plan--R(T,Rx) (Vinsonhaler et al., l977a). Clinical strategy consists of a sequence of tasks that trans- late memory into action. These tasks mainly involve information gathering and information processing as the clinician makes decisions about diagnosis and treatment. Those tasks that were empirically derived from studies of clinical problem solving with simulated cases (Elstein et al., 1978) include the following: 1. Cue acquisition--the process by which the clinician decides which information (cue) should be collected in a "medical history, physical examination, and laboratory work-up" and the rela- tive value of those cues selected. Cues may be chosen on the basis of (a) confirming or disconfirming one or more competing hypotheses concerning the patient's problem or (b) according to some information- gathering routine work-up. 2. Hypothesis generation--the process of retrieving from memory a number of problem formulations (hypotheses) based on (1) some limited number of cues and (2) the relations between the cues and problems R(C,P) that are part of the clinician's memory. Early generation of hypotheses may be used to direct the work-up. 3. Cue interpretation--the process by which case information (CUES) is "evaluated in terms of [its] 'fit' to specific hypotheses." 4. Hypothesis evaluation and diagnosis judgment--the process by which an estimate of the likelihood of each hypothesis being con- sidered is determined. This is done by (l) eliminating unlikely 14 hypotheses and (2) accepting as the diagnosis those hypotheses with sufficiently high likelihood. "Likelihood is calculated on the basis of the relations between problems and cues R(P,C)” (Vinsonhaler et al. 1977a, p. 12). 5. Treatment evaluation--the process by which an estimate is made of the expected gain from each available treatment for the diag- nosed problem. Expected gain is calculated on the basis of the rela- tions between the problem and available treatment plans R(P,T). These relations may include effications, cautions, contraindications, cost, and preference appropriate to the patient or case. A treatment plan is selected on the basis of highest Expected gain. 6. Prescription selection--the process by which relations between treatments and prescriptions R(T,Rx) are used to write out the specifics of the case management. With the knowledge of effective clinical problem solving, computer programs can be developed to simulate a clinician engaged in clinical diagnosis. The memory and strategy of the computer arrive at a diagnosis of a problem. Thus the computer applies a diagnostic process used by human clinicians. One computer simulation system is discussed in the following section. The Basic Management Information System (BMIS) The Basic Management Information System (BMIS) computer pro- gram makes it possible to simulate a clinician. First, a simulated memory must be described as in the previous medical example of angina. Then a description of a strategy or sequence of information-processing 15 actions (see Figure 4) is presented. Finally, a case is presented and the computer behaves according to the described memory and strategy. Simulated clinicians (SIMCLINs) are used to test and improve the Inquiry Theory by establishing valid deductions and quan- titative predictions and comparing them with the clinical problem- solving behavior of real clinicians (Vinsonhaler et al., l977b). As will be shown later, the present study was intended to examine certain predictions derived from the existing Inquiry Theory. Theoretic Implications This research adhered to an IRT Clinical Studies Project objective of developing and empirically testing clinical problem- solving theory and its application to reading. The present conceptual replication study was therefore concerned with testing theoretic implications of three basic components of the Inquiry Theory of Clini- cal Problem Solving. The theoretic implications that seem relevant to this study are (l) the effect of clinical memory and strategy on cue—collection agreement, (2) the effect of clinical memory and strategy on the diagnostic (Dx) agreement of clinicians, and (3) the effect of clini- cal memory and strategy on hypothesis generation. These three components may be restated as corollaries or pr0positions derived from the informal* Inquiry Theory and implications noted for each of the three. *The Inquiry Theory is often divided into three parts: (1) the formal theory, a set of Fortran computer programs--BMIS; (2) the interpreted Inquiry Theory, simulation study results based 16 l. Cue (Cx) Agreement Corollary Informal statement: The greater the number of common cues represented in memory, the greater the number of common cue elements in the clinical interaction. Formal statement: If N (Cirle) 3 N (CkflCl), then N (Cxifl x-) 2 N (Ckk()Cx]), all else equal, where Ci denotes the J cue component of clinical memory and Cxi denotes the set of cues present in the ith clinical encounter. Implications of cue (Cx) agreement corollary: Following arguments similar to those presented for the Dx agreement corollary, two predictions may be offered for observational study results. Those predictions follow. a. Same clinician, same case versus different clinicians, same case: Assuming that the cognitively formed cue memory is stable (the common or shared elements do not vary or change significantly over time), the agreement in cue selection by the same individual should be equal to or greater than the agreement between individuals. Since N (CTOC‘i) z N (CiOCj), then N (CxiOCxi')z N (CxiOij'), assuming all other factors are constant. b. Commonality of cue selection: Given a set of cues, if common elements exist in the clinical memories, then there should be "commonality" or agreement among clinicians on cues selected for any given case. on BMIS program data; and (3) the informal theory, a natural English- language summary and interpretation of simulation studies. 17 Measurement problems: Many of the problems associated with the use of numerocity of sets in diagnosis (Dx) predictions are also common to cue (Cx) predictions. Therefore, both proportional fre- quency and correlational measures of agreement are used for measuring cue "commonality." 2. Diagnostic (Dx) Agreement Corollary Informal statement: Given any two diagnoses (statements con— cerning the problems or conditions of the client) using the same techniques and based upon the same case, the greater the number of common or shared elements in clinical memory (problems, cues, and the relations among them), the greater the number of common or shared elements in the diagnoses, assuming all other factors are constant. Formal statement: If N (Pifle) 2 N (PkflPl), then N (Dxilexj) 2 N (kalexl), where Pi denotes the set of state or problem descriptions, i.e., strengths and weaknesses of the case, and ij denotes the set of diagnostic statements about the case. Implications of diagnostic (Dx) agreement corollary: Based on the Observational Study, 1977 (see Chapter III) data analysis to date, there seem to be two implications for the present research study. Those implications follow. a. Same clinician, same case versus different clinicians, same case: Informal statement: Assuming that the cognitively formed diagnosis (Dx) is accurately reflected in a written diagnosis and that the clinical memory remains reasonably stable (the common or shared elements do not vary or change significantly) over short time 18 periods, then the number of common elements between diagnoses prepared by the same clinician on two forms of the same case should be greater than or equal to diagnoses prepared by different clinicians on the same case or on alternate forms of the same case. Formal statement: If N (PiflPi) 2 N (Pifle) for i f j, then N (Dxilexi')3 N (0x10 ij). b. Commonality of diagnostic statements: Informal statement: Given a set of diagnoses, if common elements exist in the clinical memories, then two or more clinicians should have common or shared problem or state descriptions in their diagnoses; i.e., there should be a commonality of diagnostic state- ments. Formal statement: If N (PiO PjA Pk . . . 0 Pn) # 0, then N (Dxi O ij O ka . should yield nonminimum values; i.e., . .0 Dxn) f 0. Hence an index of commonality Number of Dx's including statement > 1 Number of diagnoses I (Dx statement k) = 2l Measurement problems: The numerosity of the sets of common diagnostic elements or statements N (Dxilexj) seems to be an inade- quate basis for measurement because the number of common elements in the diagnoses depends on the number of diagnostic categories (background information, Dolch Word List, etc.) used in the set .); i.e., the larger the set, the more likely one is to 3 get agreement. Rather, various "normalized" statistics have to be (Dxi and Dx used to evaluate the various hypotheses stated previously. These 19 include a diagnostic (Dx) commonality score and various types of correlation and frequency matches. 3. Hypothesis (Hx) Generation Corollary Informal statement: Clinical strategy based on deductive reasoning, using hypotheses to direct inquiry, should show a tendency toward early hypothesis and observation generation. Hence, the clini- cal interaction of a deductive reasoner may be characterized as fol- lows: The first statement of hypothesized or observed states of the clinician (later included in the diagnoses or otherwise dropped) generally occurs in the first half of the interaction. Formal statement: th . = N (new statements I quarter) (1 PSI)’ where PSI N (total new statements) The hypothesis statement generation score is equal to the value in the first, second, third, and fourth quarters of the sum of the product of I (the quarter = l,2,3,4) and P (the proportion of SI original, not previously stated, descriptions of the client's clinical states occurring in the Ith quarter). In general, deductive reason- ers are characterized by low ng scores, e.g., less than 2.0, whereas inductive reasoners are characterized by high ng scores, e.g., greater than 2.0. Implications of hypothesis (Hx) generation corollary: The major implication of the Hx corollary is that clinicians having hypothesis statement generation (ng) scores above 2.0 (hypotheses generated in the third and fourth quarters of the session) are 20 probably not using deductive reasoning. Rather, such individuals must be using some approach that directs their inquiry on the basis of something other than hypotheses. Given recent research findings with medical and reading clinicians, another pattern of clinical strategy has been postulated--that of inductive reasoning. Inductive reasoners may be defined as clinicians who collect cues on a presently unknown basis, perhaps largely at random from some fixed preferential set, and who then interpret those cues by generating hypotheses. It then appears that once these clinicians are satisfied that a sufficient number of cues has been collected, they then attempt to state a diagnosis (Vinsonhaler, 1979a). Research involving the hypothetico-deductive approach or early generation of hypotheses in clincial diagnostic inquiry has been investigated in medical studies involving the Inquiry Theory ,(see Elstein, Shulman, Sprafka et al., 1978). For the sake of clarification and to avoid confusion over the terms deductive and inductive as used in the psychological lit- erature, discussions of hypothesis generation in this study refer to deductive reasoning as hypothesis—directed inquiry and to inductive reasoning as cue-directed inquiry. (See Process Measures, Chapter IV.) Summary The purpose of Chapter I has been to direct attention toward the need for the investigation of clinical problem solving in read- ing. The need is threefold: (1) there is practical value in using a theoretical base for empirical research, (2) there is efficacy in 21 using scientifically based methods (medical studies) to examine clinical problem solving in reading using simulated cases, and (3) there is augmentative value for research on clinical problem solving when research studies share a common methodology (Vinson- haler, 1979a). The purpose of this study was to investigate three components of the Inquiry Theory of Clinical Problem Solving, which were described in the second part of Chapter I. The Inquiry Theory pos- tulates that the clinical encounter involves a case and a clinician and is characterized by the interaction that occurs between the case and the clinician's memory and strategy. The three major questions addressed in this study and that relate to the Inquiry Theory are: (1) 00 experienced reading clinicians agree on the data they collect for a specific reading case in making a clinical diagnosis? (2) Do experienced reading clinicians agree on the diagnostic statements they make for a specific reading case in making a clinical diagnosis? and (3) 00 experienced reading clinicians indicate the use of hypotheses to direct their clinical inquiry in diagnosis? Overview This dissertation is a summary of the application of the Inquiry Theory of Clinical Problem Solving as it relates to the clini- cal decisions of reading clinicians. In Chapter II, the literature pertinent to clinical diagnosis is reviewed. It includes a discussion of research procedures in clinical problem solving, problems inherent in reading diagnosis, and closely related research involving the 22 Inquiry Theory. In Chapter III, the design for the study is explained. This explanation includes a description of the sample, the Operational procedures, operational and reliability measures, research questions, and the design. The analysis methods and results are presented in Chapter IV. The chapter format deviates from that which is often used in the pre- sentation of analyses, the reason being the somewhat complex nature of the methods of analysis, many of which were calculated on data generated by computer programs developed specifically for this and related studies. In Chapter IV, one analysis measure is explained, and the data for that measure are presented before the next measure and its corresponding data are presented. An attempt is made to relate each measure to the Inquiry Theory. Turning now to a review of literature related to clinical decision making in reading diagnosis, the focus is on what ideas pervade reading professionals' discussion of diagnosis. Additionally, it is shown that the pervasive trend is changing, as evidenced by new theoretically based research in the area of clinical diagnosis. CHAPTER II REVIEW OF THE LITERATURE Introduction In the reading literature there is no definition or general consensus about what constitutes a "good" diagnosis. Neither is it obvious what procedures a clinician should follow in reaching an effective and efficient diagnosis, one that will direct equally effective and efficient remediation. Additionally, there is disagree- ment in the reading field over terms used to describe reading prob- lems. For example, the word "dyslexia" has become so nebulous that it is virtually useless; the same is true of such terms as "perceptual deficit" and "minimal brain damage." The literature is replete with examples. Compounding the problem is the debate among reading Special- ists over causation. These and other dilemmas in the reading field have' undoubtedly contributed to the dearth of systematic, scientific, and theory-based research in an area most vital to successful reading diagnosis and remediation--that of clinical decision making. In the psychological literature, related research can be variously represented on cognition, thinking, human judgment, problem solving, and decision making, with terms referring to different research paradigms and models. Although such terms are conceptually 23 24 unclear, their common relationship in realistic task environments seems to be accepted (Shulman & Elstein, 1975). The manner in which processed information is acquired seems to be as varied as the terms used to describe it. A reviéw of some of the approaches used to study the intellectual process of subjects as they render judgments, solve problems, and make decisions pro- vides the impetus for ways in which these research approaches can be extended to the study of relevant issues and concerns in reading as well as in other areas of education. In Chapter II, a review of literature relative to clinical decision making in reading and to the questions being addressed in this study is presented. The review is divided into three major parts: (1) Research Procedures in Clinical Problem Solving, (2) Problems Inherent in Reading Diagnosis, and (3) The Inquiry Theory Research. Research Procedures in Problem Solving For purposes of this study, the discussion of research pro— cedures in problem solving is confined to three methods: total-task or process-tracing studies, in-basket techniques, and tab-item methods. Total-Task Studies One means of studying problem solving involves describing the intellectual processes of subjects as they make decisions and render judgments. This "process-tracing" approach typically 25 characterizes human thinking and problem solving through verbal reporting or restatement in a computer-simulated program. Process-tracing studies may involve partial or subtask exami- nation or may investigate a total task. "Total task studies investi- gate the sequential character of information seeking that leads to judgments or decisions. They use forms of simulation to represent the task environment. . ." (Shulman & Elstein, 1975, p. 5). One example of a total-task, high-fidelity study involved the problem solving of physicians (Elstein, Kagan, Shulman, Jason, & Loupe, 1972; Sprafka & Elstein, 1974). Elstein and his colleagues attempted to describe the cognitive processes of physicians beginning with an initial patient encounter and continuing through the final diagnosis. Thinking-aloud protocols of physicians and simulated patients were analyzed using data, hypotheses, and the relation between them. The general model of medical inquiry that emerged included four major activities: (1) acquisition of data or cues, (2) generation of hypotheses, (3) interpretation of data or cues, and (4) evaluation of hypotheses. The most universal characteristic of the sample of physicians and problems in the study was the early generation of hypotheses. Diagnostic accuracy was associated both with slightly higher thoroughness of acquisition of data and with greater accuracy of interpretation of data. There was no correlation between accuracy of interpretation and thoroughness of cue acquisition. Research on clinical problem solving using verbal reporting as a measure of process tracing has been studied in a variety of ways: Clarkson (1962)—-process tracing to model the decision process of a 26 bank trust investment officer. De Groot (l966)--investigation of the thought processes of chess players. Kleinmuntz (l968)--diagnostic problem solving among clinical psychologists interpreting MMPI pro- files and clinical neurologists using simulated data in their specialty. Discussing his own work, De Groot (1966) included a set of principles that, according to Shulman and Elstein (1975), ”serve as a credo for the process-tracing approach, ethnographic stylez" First, the research is directed toward systematic descrip- tion of cognitive phenomena rather than to strict hypotheses testing. Second, we keep machine simulation in mind, but we hardly do it as yet. Third, the experimental settings are often more like real-life than the strictly controlled arti- ficial conditions of the laboratory. Fourth, extensive use is made of introspective techniques of various kinds. Fifth, as a result, protocol coding and interpretation are of crucial importance (and consume a large part of our time). Sixth, pro- spective outcomes are expected to be primarily valuable to the extent we succeed in providing adequate, systematic process descriptions, possibly to be used as a basis for simulation (pp. 19-20). In-Basket Studies Another approach to the study of decision making is the in-basket technique. The decision maker in the in-basket studies receives the inputs for decisions he must make. The in-basket, after which the technique was named, might contain letters, reports, or messages that would need an action or a decision. Shulman (1965) and Shulman, Loupe, and Piper (1968) created a modification of the in-basket approach of representing decision making. . . to simulate aspects of a classroom's problems. This made it possible to study teacher inquiry behavior under circumstances 27 in which subjects would function in a highly unstructured problem-rich task environment. Rather than focusing on teacher behavior as such, the variables of interest were problem sen- sitivity, use of diverse information sources, use of time (tasks had no time limit), quality of decisions, task organiza- tion, sequence of activities, and the like (Shulman & Elstein, 1975, p. 8). Using in-basket-type techniques, Hemphill, Griffiths, and Frederiksen (1962) studied the behavior of educational administra- tors. Tab-Item Methods Tab-item methods of studying problem solving increase the objectivity and reliability of interpretation and reduce problems of coding, analysis, and interpretation by predesignating the available items of information or choices of action available to the subject. There is no introspection or thinking aloud. In a study of troubleshooting performance, Glaser, Damrin, and Gardner (1954) conducted the earliest published tab-item study. In that study, the performance failure of a piece of electronic equipment was described, and a list of all possible tests a trouble- shooter might make in order to locate the problem was given. A paper tab covered the test-results information. By removing the tab cover- ing, the subject would leave a record of the steps he had taken in determining the source of difficulty. Rimoldi (1955, 1961) and Rimoldi, Devine, and Haley (1961) were responsible for the development of a large number of sequential problem-solving tests using tab—item methods. They experimented with a variety of scoring procedures that compared the subjects' 28 information-gathering sequence either logically defined or defined by criterion-group performance. Problems Inherent in Reading Diagnosis Before understanding what might be involved in the process of reading diagnosis, it seems judicious to note some problems that may be considered inherent in the diagnostic process. Although not all- inclusive, three interrelated problem areas relevant to this study are (1) the essence of reading diagnosis, (2) the debate over causa- tion, and (3) the lack of standardized terms. The Essence of Reading Diagnosis The word "diagnosis“ is derived from two Greek roots, "dia," meaning thorough or thoroughly, and "gnosis," meaning knowledge. The literal meaning of "diagnosis" is a thorough knowledge, whereas its medical meaning is the determination, by examination, of the nature and circumstances of a diseased condition. The definition of the word "diagnosis" that seems most applicable to the study of reading prob- lems is "a determining or analysis of the cause or nature of a problem or situation" (Random House Dictionary of the English Language, 1966). If one is to take this definition literally, one would be more con- cerned, it seems, with the etiology of reading problems. It is not surprising to find this emphasis on causes, since the word "diagnosis" has long been associated with the medical profession, and medical ter- minology tends to label learning problems in terms of causation or etiology. 29 The word “remedy" or "remediation" has to do with a "healing treatment" or restoring to a "natural or proper condition," i.e., a therapeutic concern (Random House Dictionary of the English Language, 1966). Although diagnosis and remediation might be defined sep- arately, they are probably most frequently considered part of the same process, occurring simultaneously during treatment (Spache, 1976). However, some in the reading field would tend to dichotomize the diagnostic process. For example, Bond and Tinker (1957) described two types of diagnosis--etiological and therapeutic. Etiological diagnosis is concerned with causation and hhy the child is in diffi- culty. According to Bond and Tinker, to prevent reading problems, it would be useful to know that a child had been absent for a month in first grade, but such knowledge would not be useful "for the imme- diate job of correcting a reading disability that began several years earlier" (p. 127). Harris and Sipay (1975) also discussed the limited usefulness of etiological diagnosis; they concluded: "Time spent on attempting to determine etiology can often be more profitably spent on helping children to overcome their present problems" (p. 242). They were careful to point out, however, that etiology is important from the standpoint of prevention and correction, but they also felt that opinions about causation are "unproved hypotheses" and that more research is needed in this area. Continuing the idea of a dichotomized diagnostic process, in a therapeutic diagnosis the concern is with present conditions and 30 situations for the child in order to give direction to reading instruction. Knowing current strengths and weaknesses is more impor- tant in therapeutic diagnosis than is awareness of a temporary hear- ing impairment that occurred several years ago (Bond & Tinker, 1957). Part of the process of reading diagnosis is collecting infor- mation on why a student is having difficulty, and this necessarily involves giving tests. However, diagnosis is not testing. Rather, it is an intelligent interpretation of information based on theoreti- cal knowledge and practical experience. It involves knowing (1) what questions to ask, i.e., what information to obtain that will aid in comprehending the reading problem; (2) how to interpret cor- rectly the meanings of information; and (3) how to understand the interrelationships of information and meanings. Another area of diagnosis that seems to need clarification is that of the actual diagnostic procedures themselves. In other words, should one follow certain steps in making a reading diagnosis? Although the answer is again anything but definite, a look at the literature shows that several experts in the reading field have sup- ported various sequential procedures that reading diagnosticians should follow. Others have pointed out factors to be considered in a diagnosis. After the collection of diagnostic evidence, Harris and Sipay (1975) suggested the following procedures: 1. consider complete picture and arrive at conclusions about what child's major difficulties in reading are, 2. determine most reasonable explanations of how these diffi- culties have come about, 31 3. decide what persisting handicaps may impede progress, determine what remedial procedures should be employed to overcome handicaps, 5. use periodic checks of remedial work with formal and stan- dardized tests to determine effectiveness of procedures, alter remediation if necessary, and 7. retest to consider pupil progress (p. 346). Robinson (1956) named the following principles of diagnosis in what appears to be a sequential ordering: secure as much information as possible obtain highly accurate level of reading ability administer standardized reading survey test analyze data to determine if there is a reading problem make detailed analysis of the problem identify factors inhibiting reading progress collate all data and interpret results ooucsmwa—n make appropriate recommendations for remedial therapy (pp. 152-53). Robinson's approach to data collection seems to be that of amassing quantities of materials without specification of importance. 0n the other hand, Bond and Tinker (1957) emphasized a more efficient approach to data collection for reading diagnosis. They discussed three levels of diagnosis: (1) general diagnosis for all children or those for special study, (2) analytical diagnosis to explore specific strengths and weaknesses and only in cases where warranted, and (3) case-study diagnosis, individual diagnosis in which reading skills and abilities need careful study. At the case-study level, Bond and Tinker (1957) gave what appears to be a sequential procedure for data collection: 32 secure information from standardized reading diagnostic tests or procedures study child for appraisals of his mental capability, vision, hearing, and physical characteristics consider child's reactions to his reading disability evaluation of environmental factors--home, school, commu- nity, etc. (p. 131). Recognizing the differential nature of diagnosis and that "opinions as to causation remain in the realm of unproved hypotheses," Harris and Sipay (1975) advised reading Specialists and teachers to follow this sequence in dealing with children who appear to have a reading problem: 1. determine the individual's general level of reading achieve- ment and compare it with his potential; if a reading problem exists, determine the learner's specific reading skill strengths and weaknesses, determine which factors are most probably hampering the child's ability to learn at that time, remove or lessen those factors that can be controlled or cor- rected, either before or during remedial treatment, select the most efficient and effective way to teach the needed skills, conduct a program of skill mastery, and refer to an appropriate clinic or agency any child who does not respond to treatment after a reasonable period of time p. 242 . Although he did not discuss the specifics of data collection, Carter (1970) concerned himself with four levels of diagnosis, which he presented on a schematic scale. The four levels are (l) Identi- fication of Problem, (2) Classification, (3) Identification of Reading Needs, and (4) Determination of Causal Factors. At the upper or fourth level, Carter then emphasized use of the following procedures. though not necessarily in sequence: 33 1. Identify the problem and possible causal factors. 2. Assume and reject hunch after hunch until one can be accepted tentatively. 3. Discover possible determinant and explain consequential relationship. 4. Predict that with treatment the disability will be overcome. The clinician must verify this prediction (p. 20). Although stressing the continuous nature of diagnosis, Wilson (1977) pointed out that the reading specialist is in a flexible diagnostic role, having three levels of clinical diagnosis from which to choose: (1) initial screening or brief, concise evaluation of student's reading skills (approximately one hour); (2) selective testing (not overtesting), and (3) case study or in-depth testing. Bond and Tinker (1957) mentioned eight principles or aspects of diagnosis: 1. Direct diagnosis toward methods of improvement. Have the diagnosis go beyond appraisal of reading skills and abilities. Make the diagnosis efficient--don't overdiagnose. Collect only important information by most efficient means. Use standardized test procedures when possible. Use informal testing when diagnosis needs to be expanded. Formulate diagnosis on basis of "patterns of scores." Make diagnosis a continuous process (p. 126). mummbw According to Harris (1961), the term "diagnosis" involves two major components: discovery and exploration. The discovery part of diagnosis involves a careful study to determine the nature of the reading condition, whereas the exploration part concerns the causes of the reading difficulty. 34 In determining the nature of the reading condition, Harris considered both informal teacher-made tests and standardized tests. The areas of reading that he emphasized were (1) reading level; (2) comprehension; (3) rate, fluency, and accuracy in oral reading; (4) word-recognition skills; and (5) learning potentialities in word recognition. Harris further believed that making a reading diagnosis involves understanding the dimensions of the student's reading per- formance and of those factors, both present and past, that con- tributed to the difficulties manifested. He stated that although tests may provide insights and facts needed, “the heart of diagnosis is not testing" but rather involves 1. intelligent interpretation of facts in the light of theoreti- cal knowledge and practical experience, knowing what questions to ask, knowing how to select procedures, including tests, which can supply needed facts, knowing how to interpret the meaning of findings, and 5. comprehending the interrelationships of these facts and meanings (pp. 220-21). Spache (1976) characterized diagnosis as: l. a continuous process of testing, observing and hypothesizing in a flexible trial-conclusion strategy, pragmatic and directly related to remedial practice, eclectic and thorough, a constant exploration of the student's strengths and recog- nizes variability from one subskill to the next, broad enough to explore all possible causes of problems, and only temporary, supportive help for the student (p. 9). 35 According to Smith and Dechant (1961), knowing the student's strengths and weaknesses is the essence of reading diagnosis. It is important, they said, to study the student's instructional needs based on the expectancies of his chronological age, mental age, and grade placement in light of his general abilities and reading potential. They stated that identifying causal factors in regard to reading development is also a part of diagnosis. The way one conducts the search (i.e., gathers information) for factors contributing to a reading problem is most likely influ- enced by one's beliefs about causation (VanRoekel, in progress). The Debate Over Causation For centuries, philosophers have been pondering the meaning of causation. When two types of traits, events, or actions can be observed to occur simultaneously more frequently than could be expected on the basis of chance, and if one is consistently preceded by the other, it might be easy to assume that the first is cause and the second effect. This correlation, or the fact that two or more measur- able characteristics tend to be found together, does not prove causa- tion. In fact, simple causal relationships can seldom be established in the study of people. This is true of reading disability, said Harris and Sipay (1975). They stated: In a particular child with a reading disability several characteristics may be found, each of which has been shown by research to be somewhat correlated with reading disability. To determine which of these characteristics may have interfered with the child's learning to read, and their relative importance in this case, is a difficult detective job at best and often cannot be solved. 36 For this reason, it is safer to discuss the correlates of reading disability than the causes. Correlation can be readily demonstrated or disproved; causation is much more difficult to establish (p. 239). Another issue that Harris and Sipay cautioned about in drawing conclusions with regard to cause and effect was the idea that strength in one area of reading can compensate for weakness in another area. Research by White and White (1972) and by Bell and Aftanas (1972) showed that good and disabled readers differed primarily in the num- ber of abilities in which they showed special immaturities rather than strengths and weaknesses. Weak abilities, said Harris and Sipay (1975), may combine to have a causal effect that could not be pro- duced by one weakness alone. The following resolution prepared by the Disabled Reader Committee was approved by the 1972 Delegates Assembly of the Inter- national Reading Association: There is no single cause for reading disabilities. Reading problems can be caused by a multiplicity of factors all of which are probably interrelated. Just as there is no single etiology, there is no one choice of intervention. For these reasons we deplore the action of those individuals and institutions who suggest that their methods are infallible, appropriate and optimal for every child, and universally efficacious (Harris & Sipay, 1975. p. 241). The search for contributing or causal factors that one con- ducts in specific reading disability cases may be strongly influenced by one's background and beliefs about causation of reading problems. Neurologists, for example, have concerned themselves with what they call "congenital word blindness or dyslexia." They see this as a constitutional condition, often of a hereditary nature, and often accompanied by other communication difficulties such as problems in 37 listening, speaking, spelling, handwriting, and written composition (Penn, 1966; Gomez, 1972; Rosenthal, 1973). According to Harris and Sipay (1975), regarding causation: From a practical standpoint, the aim of a thorough diagnosis is not to fix the blame for the child's difficulties, but to discover each of the many conditions that may require correc- tion. A person who develops an enthusiasm for any one theory of causation can frequently find evidence of the handicap he looks for, but is likely to overlook many other significant complications while doing so. An unbiased search is needed for a really comprehensive and satisfactory diagnosis. This usually requires the combined efforts of professionals from sev- eral different professions (p. 310). Recognizing the interrelationships of causal factors in the exploration of reading problems was Harris's (1961) approach. He believed that the purpose of a thorough diagnosis is not to "fix the blame" for the student's reading deficiency, but to determine each of the multiple conditions that might be corrected (i.e., more a remediation or therapeutic approach). The debate over causation, whether one takes a singular or pluralistic view, or dichotomizes in the etiological or therapeutic sense, is directly related to the issue of terms used to describe a reading problem. Thus, depending on who is describing the educa- tional problem, the student might be called reading disabled, retarded reader, perceptually handicapped, or learning disabled. Lack of Standardized Terms Despite the fact that reading problems have been studied for many years, terminology in the reading profession is not yet stan- dardized. Those definitions that do exist are, according to Spache 38 (1966), little more than "armchair descriptions and lack the prag- matic definitions needed" (p. 22). One example of the lack of agreement among reading special- ists in defining terms is the wide discrepancy in estimates of significant retardation in elementary schools. The estimates vary from 10 or 12 percent up to 30 percent or more. De Boer and Dallman (1960) believed that the differences may be largely due to a lack of agreement about "what constitutes retardation" (p. 267). Smith and Dechant (1961) defined reading retardation as reading below one's "present general level of development" (p. 420), and including physical, emotional, social, and mental development limits. Durrell (1940) believed that reading retardation of six months in first grade is more serious than a deficiency of a year or more in the sixth grade (p. 279). Harris (1953), on the other hand, did not consider that a first-grade student has a reading problem unless his reading age is at least six months lower than his mental age. In grades four and above, Harris defined the problem as a discrepancy of a year or more (p. 299). Bond and Tinker (1957) grouped disabled readers into four descriptive categories, according to problem severity: 1. Simple retardation includes those students whose reading ability is somewhat immature but balanced. 2. Specific retardation involves children who are low in one or more types of reading but are competent in the basic reading skills. 39 3. Limiting disability includes those children deficient in basic reading skills, which precludes further growth in reading. 4. Complex disability involves children whose reading growth is inhibited by basic skill deficiencies and who exhibit other accompanying problems such as physical or personality handicaps (pp. 81-82). Spache (1976) defined the disabled reader as one who 1. is retarded in a number of major reading skills (such as rate, vocabulary, comprehension or word analysis); 2. is retarded by one year or more, if in the primary grades, or by two years or more if older (one year or more at primary level, two years from grades four to eight, and three years or more at secondary should be dividing line for pupils who can be dealt with in the classroom and those who need special clinic help); 3. is an individual who has had normal opportunities for schooling; and 4. has continued to show this degree of retardation below his sociocultural peers despite corrective efforts (pp. 4-8). In addition, Spache pointed out certain mitigating factors that must be considered in identifying disabled readers. These include the student's sociocultural status, the nature of his reading difficulties, the degree of retardation below a level common to his peers, the duration of his problem, and the need for special profes- sional assistance beyond what has been or can be done in the class- room. Spache said that other factors, although not part of the operational definition of a reading disability, should also be con- sidered before selecting students for intensive diagnosis and remedial treatment. These include estimates of treatment duration in light of the severity of the problem, the student's age and grade place- ment, his 1.0., and other background information. 40 It appears, then, that basic disagreement affecting reading diagnosis might stem from the lack of standardized terminology or taxonomy. This has complicated communication both within and between professions (Whitecraft, 1971). It is not uncommon to find the same term used with different meanings or the same condition given various labels. As Brown and Botel (1972) noted, further complications are caused by the confusion of causes with symptoms, diagnostic criteria, and correlated characteristics. The next area of research being presented represents an attempt by some educators at Michigan State University to lessen the confusion in the field of reading diagnosis. This attempt was made by linking one discipline, that of reading, with another, more scien- tific and precise discipline, namely medicine. The common bond between them is clinical decision making, but the foundation is pro- vided by the Inquiry Theory of Clinical Decision Making, known as the Inquiry Theory. The Inquiry Theory: Related Research Introduction To comprehend fully the magnitude of the Inquiry Theory, it seems important to review the major historical or sequential events that led to the development of the theory and to present further those subsequent research efforts that are most closely aligned with the present study. 41 The Medical Inquiry Project The Medical Inquiry Project is an extensive observational study of clinical data gathering and information processing among expert physicians, which led to the development of the Inquiry Theory of Clinical Problem Solving. One major purpose of the project was to study problem solving and reasoning in a complex task environment where previous experience was clearly relevant and the data inherently probabilistic. Medical practice seemed to provide the best oppor- tunity for studying reasoning under those conditions. The resulting model of medical reasoning was thus derived mainly from the intensive study of a few medical problems worked up by approximately two dozen physicians (Elstein et al., 1972). The pursuit of these studies of medical reasoning led to a computer simulation of medical thinking. From the observational study of physician performance, three major variables emerged: cues, hypotheses, and the relationship between them. Cues are items of data or information in a case that may be used by the physician to help him make a diagnosis. Hypothe- ses are possible diagnoses that the physician uses to direct inquiry and rubrics in short-term memory under which the cues or data may be stored. The relationship between cues and hypotheses involves the interpretation of cues and testing of hypotheses. Clinical tasks areactions directed toward alleviating problems. There are two types of clinical models, the diagnostic and the thera- peutic. In the therapeutic mode, the patient's underlying condition or state is identified to the point at which an action can be taken. The clinical task is to determine what action should be taken. 42 Concern focuses on probabilities and values of possible outcomes, not on determining the underlying state of the problem. Treatment in this mode may confirm a diagnosis. In the diagnostic mode, emphasis is on determining the nature of the problem and identifying its causes. Treatment or action develops naturally from proper char- acterization of the problem (Elstein, 1977). Diagnostic problem solving has been described as an iterative process consisting of four tasks: collection of information, genera- tion of hypotheses, interpretation of evidence, and evaluation of hypotheses for a diagnostic decision (Elstein, Shulman, Sprafka et al., 1976). Also related to the diagnostic and therapeutic modes are two other clinical activities: screening and follow-up. Screening is a process of focused data gathering directed at a specific problem. The amount of material collected is relatively small, and the client contact is brief. Follow-up involves observing the occurrence of anticipated outcomes, determining if new outcomes have resulted in changes in the nature of the system, and updating decisions of action based on the update (Elstein et al., 1977). Elstein (1977) noted that clinical reasoning employs a "hypothetico-deductive" method of determining data collection. In this method preliminary hypotheses are generated, and the clinician seeks data to test them. By limiting the problem space, through the generation of a small number of working hypotheses (short-term memory store), the clinician simplifies the problem of the larger, long-term memory store. It was found that physicians consider from four or five 43 up to six or seven hypotheses at one time. This number is "well below the number of hypotheses in the long-term memory store of any reasonably experienced physician and is evidence for the proposition that the size of the working memory is considerably smaller than the size of the long-term memory store" (Elstein, 1977, p. 38). It has been emphasized that not all clinical tasks involve physicians and the medical profession. Likewise, it has been noted that “the clinical model is especially well suited to the study of reading clinicians because it is problem-initiated and problem- directed." However, it has been suggested that some untrained read- ing specialists may not really be able to use the clinical model in its entirety because "they lack the skills necessary to carry out the last two stages of the model" (interpretation of evidence, and hypothe- ses evaluation) (Elstein et al., 1977, p. 6). Clinical Information Processes in Reading (CLIPIR) The Clinical Information Processes in Reading (CLIPIR) study was conceived to explore the nature of clinical problem solving in reading diagnosis and remediation among reading clinicians and teach- ers and to test empirically the Inquiry Theory of Clinical Problem Solving as it applies to the field of reading. Because of the exploratory nature of the research, it was necessary to design studies to establish piloting procedures, determine relevant variables, and construct a data base for the study of clinical behavior. 44 SIMCASE development.--To provide an empirical data base of the clinical problem-solving behavior of teachers and reading clini- cians, simulated cases of reading disabilities were developed. A simulated case or set of data representing a child with a reading problem is referred to as a SIMCASE (Lee & Weinshank, 1976). 10. The SIMCASE development team followed these procedures: Generation of list of problems which might be included in a case. Variables considered were (a) frequency of problems, (b) range of etiologies, and (c) alternative views of the reading process. Selection of the problems for possible SIMCASE development from the list. Review of case pool in M.S.U. Reading Clinic and M.S.U. reading diagnosis courses for purpose of identification of suitable simulation cases. Development of Data Categories, all the data which might be requested on a case by clinicians, such as home and school background information, formal and informal test results, audio tapes of reading, etc. Evaluation of overall case contents by senior clinicians. Preparation of SIMCASEs, including audio tapes, writing samples, test results, etc. Re-evaluation of the complete cases by the independent evaluation of two senior clinicians for internal consistency, fidelity and comprehensiveness of case contents. Production of two versions of each SIMCASE: a manually based or "boxed" version and a computer-based version, con- sisting of all information that could be keyset into the IRT computer plus tapes and other materials to which the computer could refer clinicians. Piloting of procedures and Data Categories (Cue List) using senior clinicians. Evaluation of manually based and computer-based SIMCASEs by SIMCASE Selection and Evaluation Team and SIMCASE Develop- ment Team. 45 Observational Study, 1977 . In 1977, a study was conducted on the interaction of eight "very senior" clinicians with eight SIMCASEs, four of which were alter- nate forms of the other SIMCASEs. The paid clinicians were selected on the basis of (a) recommendation of the local school administrators, (b) recommendation of the M.S.U. faculty, and (c) performance in SIMCASE verification. The task for the clinicians studied was threefold: (1) given unlimited time, to select materials, using a cue list contained in the SIMCASE (subjects were encouraged to verbalize their thinking); (2) to write a diagnosis and suggested remediation based on material they had examined in the SIMCASE; and (3) in a debriefing session, to verbalize their rationale for cue selection and interpretation (stimulated recall) (Vinsonhaler, 1979b). Outcomes, l977 Results of the Observational Study, 1977 data indicated that by using an analysis of the diagnostic consistency of clinicians as they described the strengths and weaknesses of a SIMCASE, it was found that the senior reading clinicians studied lacked the consis- tency that might be considered necessary in medicine. Diagnostic consistency was measured by comparing clinicians' diagnostic reports for each SIMCASE with reports on alternate forms of the same SIMCASE and by comparing an individual clinician's diagnostic report with those of other clinicians diagnosing the same SIMCASE. The low agreement found in these comparisons suggested that clinical diagnosis in reading might be unreliable. 46 CLIPIR Application Exploratory Study in Educational Research (CAESER), 1977 During the summer of 1977, an exploratory study designed to apply the clinical problem solving theory in training teachers was conducted. Two primary questions were addressed: (1) "Could teach- ers be trained to diagnose in the same manner as an effective experi- enced reading clinician?" and (2) "Would increases in diagnostic performance be accompanied by increases in diagnostic memory?" (Sherman, 1979). Results of a five-week study involving 36 students in a reading-diagnosis course tentatively indicated that (1) students did learn to diagnose in the same manner as the senior reading clinicians and that (2) increases in diagnostic performance were accompanied by increases in diagnostic memory (Gil, Hoffmeyer et al., 1979). (Implications, 1976-1977 The Observational Study, 1977 results seemed to indicate that some expert reading clinicians were less precise in both diagnosing and reporting diagnostic results than were clinicians in other fields such as medicine. However, the results of that study also suggested that the diagnoses and diagnostic reports of some expert reading clinicians do remain fairly consistent and yield high diagnostic com- monality scores. As in medicine, these clinicians employ a hypotheses- directed approach to the clinical-inquiry process of problem solving. The application study (CAESER) seemed to indicate that with proper training involving SIMCASEs with feedback, clinicians, reading 47 specialists, and classroom teachers can be taught to prepare diag- nostic reports that are consistent and that yield high commonality SCOPES. Recent Studies Recently completed and/or ongoing studies based on the theo- retic structure of the Inquiry Theory have investigated the clinical problem-solving skills of experienced reading and learning-disability specialists and classroom teachers as they diagnose and then propose remediation for a variety of reading problems. The ultimate goal of these collaborative research efforts is to improve the instruction, evaluation, and performance of reading clinicians. Additionally, researchers are now attempting to explicate and refine the Inquiry Theory of Clinical Problem Solving, thereby increasing its predictive powers (Gil, Hoffmeyer et al., 1979; Stephens, 1978). Summahy In the reading as well as in the psychological literature, conceptually unclear terms abound, thus confusing and complicating the study of how information is processed. Research procedures in problem solving, however, seem to have a common relationship in realistic task environments (Shulman & Elstein, 1975). Such procedures include (1) total-task or process- tracing studies, which are concerned with the sequential character of gathering information in order to make a decision or judgment; (2) in-basket studies, in which the decision maker receives informa- tion "input" for making a decision; and (3) tab-item methods, which 48 provide objective, reliable data through predesignation of choices of action, such as troubleshooting electronic equipment performance failure. The major concern of investigators using the information approach (total-task investigations, in-basket, and tab-item studies) is to observe the process of thinking and judgment in as actual a task environment as possible so as to conceptualize human behavior as task determined or related. In a thorough review of a number of theoretical models and research methods on thinking, human judgment, and decision making, Shulman and Elstein (1975) determined that those studies of informa- tion processing, decision making, policy capturing, and lens model "have rarely been applied to the investigation of educational prob- lems" (p. 32). They then proposed a variety of ways in which the models and methods they reviewed might be valuable in education. One major area of their discussion dealt with possibilities of research in diagnosing and remediating reading difficulties. One suggestion Shulman and Elstein made was to collect thinking-aloud protocols of experienced reading diagnosticians as they dealt with a series of cases. This research strategy, as well as others they suggested, is important because, "as in so many situations involving clinical judgment, the principles governing decision making are typically unclear." They concluded that it is crucial to examine hph_the prob- lem solver "sizes up the situation, how the problem is formulated, what is judged to be relevant and what irrelevant, which sources of information are considered useful and which of no importance" (pp. 34- 36). 49 While looking at the process of reading diagnosis, it was pointed out that there are problems inherent in diagnostic problem solving that should be considered. These interrelated problems include (1) the essence of diagnosis, the etiological and therepeutic aspects and the procedural steps; (2) the causal debate, singular or pluralistic and etiological or therapeutic; and (3) the lack of standardized terms. The basic disagreements over such issues have complicated communication both within and between professions (White- craft, 1971). One effort to uncomplicate matters of clinical diagnosis in medicine and subsequently in reading has been the research relating to the Inquiry Theory of Clinical Problem Solving. A number of studies, each building upon the other, have been undertaken to examine clinical diagnosis in reading. These studies, designed to test the ,Inquiry Theory, hold promise for improving the instruction, perform- ance, and evaluation of reading clinicians. CHAPTER III DESIGN AND PROCEDURES Introduction The primary objectives of this study were (1) to obtain data on the kinds of cues and the time and order in which cues were col- lected by each clinician in each clinical session, (2) to obtain objective data on the diagnostic statements made in a written diagnosis on each case by having clinicians transfer their written diagnostic statements to a standardized diagnostic check list, and (3) to obtain objective data on the use of hypothesis generation by having clini- cians complete the Hypothesis/Observation Check List (H/OCL) and then transfer their responses to the standardized Reading Diagnostic Check List (RDCL). Secondary objectives were to obtain informal data from the clinicians in terms of (l) hhat_information should be included in a good diagnosis and (2) hph_the clinicians themselves conduct a diagnosis. seams The eight subjects for this study were chosen by university faculty recommendation from a list of reading clinicians who had taught the summer institute courses in reading diagnosis and remedia- tion offered by Michigan State University. This procedure was used in an effort to select some of the most experienced clinicians in 50 51 the mid-Michigan area. From those individuals recommended, eight volunteer clinicians (subjects) were selected. There were two male and six female clinicians. Six of the eight subjects either had, or were in the process of obtaining, a Ph.D. All were experienced classroom (5 to 7 years) and reading teacher/diagnosticians (3 to 15 years), and all had taught university courses in reading diag- nosis and remediation. All of the clinicians were paid at a profes- sional rate for their participation in the study. Biographical data on the clinician subjects are presented in Appendix A. Research Design The eight cases (four cases and their replicates) used in this study were based on children who had at one time been clients of the Michigan State University Reading Clinic. An attempt was made to select those cases that were described by a number of reading clini- cians as being representative of the reading problems most frequently encountered in the public schools. The representative reading prob- lems upon which the cases (SIMCASEs) were based included sight-word deficiencies, inadequate structural and phonetic analysis skills, inadequate fluency of oral reading, and poor comprehension. Data on four students were used in developing the eight SIMCASEs; each SIMCASE had two equivalent forms. Equivalent forms were prepared by making minor alterations in the original data base, such as changing the artist's sketch, the name, birthdate, father's occupation, or age and sex of siblings. 52 The available information for any particular SIMCASE included such data as family background, classroom information, achievement and intellectual-capacity tests, and individual and group reading measures. The information for the SIMCASEs was presented in five forms: as test scores, examiner's comments, test booklets, audio recordings, and test directions. Each of the cases contained initial contact information, which included an artist's drawing of the child based on his taped voice, an audio recording of an interview with the child, and brief background information. The clinicians for the study participated in three, approxi- mately three-hour sessions spaced no less than one week nor more than four weeks apart. All stimulus materials, including SIMCASEs and equivalent forms, were subjected to counterbalancing to minimize systematic effects. Subjects for the study were randomly assigned to the case order in the manner shown in Figure 5. A summary of the research design for this study is presented in Figure 6. As shown in Figure 6, an attempt was made to balance the design in terms of easy and difficult cases. The decision about which cases were the easiest and which were more difficult was made by experienced reading clinicians who had worked with the students upon whom the cases were based. These clinicians seemed to agree that easy cases would be those that appeared most frequently. Difficult cases would be (1) those not seen often, (2) those in which the explanation of the problem was less obviously related to its cause(s), and/or 53 .mcowmmmm Fmowcwpo op muomnnzm Co unmasmwmmm Eoucmm--.m mczmwu cowom6w_amm new mmmu o_=u_ccwo omoz A cowomuw_amm tea ammo o_=uwccwo see: A cowpmuwpqmm use mmmu ommwmmm ecoumm A :o_pmuwpamm new mmmu ommwmmm A .e .N .m .— .e .N .m ._ u cop mop pop mo— cop mo— moF u mop DNQJUDLIJQ) lMQ’r—NMQ'F-N OJ'ULJJDGJ‘CLLI ls—MNQr-MNQ' OLIJQJ'UQLIJQ) r—NMQ'LONDNw ammo m cowmmmm mmm N cowmmuw F commmwm % pumwaan cowuwccoo U 0) U1 f0 0 54 (3) those in which there were more obscure answers to why the child couldn't read. Middle Case D/d E/e D/d d D d' D e D' First/Last Case E/e E d E' e E e' Case Difficulty Scale E e d D E' e' d' D' EASY F— l 4 El DIFFICULT Case: 2 4 l 3 Figure 6.--Research design. Instrumentation and Data Collection Since the analysis of the data involved the use of instruments created especially for this study, the problems of validity and relia- bility of the instruments must be considered. The problem of validity, i.e., whether the instrument measures what it purports to measure, was tested by means of a pilot study of the instruments and the instruc- tions for their use. Instruments and instructions were found to be valid. Content validity was considered adequate justification for using the instruments developed for this study. 55 The question of reliability, i.e., how consistently the instrument measures whatever it measures, entails a check on the extent to which the instrument yields similar results upon repeated trials. Reliability of the instruments used in this study was a recognized concern of the researcher. However, since no standardized tests or instruments that purport to measure the decision-making and problem-solving behavior of reading clinicians were known to exist, instruments were developed for the study. Those instruments are the Reading Diagnostic Check List (RDCL) and the Hypothesis/Observation Check List (H/OCL) (See Appendix C). Reliabi1ity To measure the subject reliability on use of the RDCL, the clinicians were mailed an uncircled, carbon copy of each of their written diagnoses one week after the third clinical session. An 'accompanying letter (see Appendix E) instructed subjects to follow the same procedures used in the clinical sessions for circling and numbering their written diagnoses and then transferring them to the RDCL. Table 1 shows the correlation (rxy) for the clinic conversion and the home conversion of the written diagnoses, which had been done in the observational session, to the RDCL. Clinic conversion refers to the transfer by the subject in the clinical session of his written diagnosis to the RDCL. Home conversion refers to the transfer by the subject at home of his written diagnosis (done in the clinic and mailed to him) to the RDCL. 56 Table l.--Correlation (rx ) and Porter Index for clinic and home conversion of written diagnosis to RDCL on four cases.a Subject Case Run No. No, No. Pearson (rxy) Porter Index 101 3 1 .57 ' .41 101 3' 3 .47 .32 101 4 2 .36 .24 102 2 1 .69 .54 102 2' 3 .68 .53 102 1 2 .65 .50 103 1 1 .50 .33 103 1' 3 .56 .41 103 3 2 .25 .15 104 4 1 .52 .36 104 4' 3 .56 .41 104 2 2 .55 .39 105 3 1 .15 .10 105 3' 3 .38 .28 105 4 2 .41 .41 106 4 1 .55 .36 106 4' 3 .53 .41 106 2 2 .40 .39 107 1 1 .44 .29 107 1' 3 .52 .38 107 3 2 .34 .24 108 2 1 .38 .25 108 2' 3 .62 .47 108 1 2 .34 .23 Note: Prime (') = alternate form for case. ar to z transformation was not used because distribution was normal. 57 The Pearson product-moment coefficient was calculated on the relationship between the two measures of check list use. The Porter Index, a statistic developed by Andrew Porter, at Michigan State University, to analyze data in the National Day Care Home Study (see Wilcox, 1977), was also computed on clinic and home data in the present study. Results of the Porter Index analysis appear in the last column of Table l. A more detailed discussion of the Porter statistic is presented in Chapter IV. As can be seen in the table, the correlation ranged from .15 for Subject 105, Run 1, on Case 3, to .69 for Subject 102, Run 1, on Case 2. The reported values indicate that the clinicians in this study did not show a high degree of reliability in their use of the check list at Time 1 (clinic) and Time 2 (home). Subject differences using mean and standard deviation for rxy are shown in Table 2. The values indicate that subjects 102 and 105 were higher and lower, respectively, than the other clinicians in the group in terms of the reliability of their check-list use. When case differences were considered, the variability was greatest for Case 3. This was influenced by the extreme value of .15 for Subject 105, Run 1, who diagnosed Case 3, the most difficult case. Case differences using mean and standard deviation for rxy are reported by case in Table 3. The values indicate that Case 3 had the least check-list reliability. The variability in check-list reliability could have resulted from subject differences or case difficulty. Other possible reasons for the limited reliability of the check list across all cases and 58 Table 2.--Means and standard deviations for subjects based on correlation (rxy) for use of RDCL. SUNS)?“ .11 1: it: 101 .47 .11 102 .67 .02 103 .44 .16 104 .54 .02 105 .31 .14 106 .50 .08 107 .43 .09 108 .45 .15 Table 3.--Means and standard deviations for cases based on correlation (rxy) for use of RDCL. we .11 r. .59 l .50 .ll 2 .55 .14 3 .36' .15 4 .49 .08 Mean total = .48 SQ = .08 59 subjects might have been the check list's complexity and/or its length (507 items), the lack of experimental control during the home ' conversion of written diagnoses to the check list, other factors, or combinations thereof. Instruments The RDCL was designed to objectify the data collection and analysis. Each clinician converted his written diagnosis to the RDCL, indicating strengths, weaknesses, or observations for diagnostic statements on the list corresponding to his written statements. This procedure was used for each session. The RDCL was deve10ped from a Taxonomy of Reading Factors (TRF), which has been under development in the Clinical Studies Research Project of the Institute for Research on Teaching (IRT). In its present form, the RDCL includes 9 major categories and 169 subcategory items. In addition, there is a tenth category called "Other," and within each of the nine categories is an “Other" subcategory. The statements for "Other" were not included in the data analysis; they are reported separately in Appendix D. The size of the RDCL (169 items with 3 possible responses per item, for a total of 507 possible responses) was another concern in the study. Earlier attempts had been made (Vinsonhaler, 19790) to reduce the size of a similar check list by combining categories, since most agreement measures are sensitive to vocabulary size. However, combining the categories yielded "large numbers of inconsistencies (e.g., the same diagnosis often includes inconsistent statements such 60 as: 'no problem with phonics' and 'needs work on long vowels')" (p. 14). The H/OCL (Hoffmeyer, 1979) was designed to objectify and simplify the data collection and analysis. Using the H/OCL, the clinician subject was asked to respond to two questions regarding his request for information (cues) on a case (SIMCASE) by selecting his answers from a list of responses to the questions "Why did you ask for this piece of information?" and "What did it tell you?" The clinician was also asked to write an explanation for each response he checked. The researcher later applied these data to the RDCL, to compare subjects' responses more objectively. The subject was not asked to make the conversion because of the additional session time that task would have required. In converting the H/OCL (debriefing data) to the RDCL, every effort was made to represent the clinician's inten- tions by referring to his written diagnosis for clarification. Statement Concerning Hypotheses Because of the exploratory nature of this study, research hypotheses were not stated. Rather, this research was concerned with formulating a systematic description of the reasoning of reading clinicians as they diagnosed simulated reading-problem cases. The questions addressed were investigated through both formal and informal measures designed to provide information on hph_experi- enced reading clinicians diagnosed specific reading problem cases, hha§_information they used in making a diagnosis, and hhy_they asked for certain information. The formal questions concerned the agreement 61 of eight clinicians on the data (cues) they collected and the diag- nostic statements they made when diagnosing reading problem cases and whether the clinicians used hypotheses to direct their clinical inquiry in arriving at a diagnosis. The informal questions pertained to the clinicians' own perceptions of hhah information should be included in a good diagnosis and hph_the clinicians went about making a diagnosis. Procedures The procedures for the three sessions in this study were as follows: 1. A background questionnaire was completed by clinicians and followed by instructions with practice on a sample SIMCASE. 2. Initial contact information was provided on the SIMCASE. 3. Data were collected by the clinicians without an inven- 'tory of data available; directions were given and a recording was made by the examiner. 4. Each clinician wrote a diagnosis and a remediation report. 5. The clinicians then transferred their written diagnoses to the RDCL. 6. Clinicians recalled their reasons for data collection by responding to a written questionnaire (stimulated recall). 7. Oral questions regarding the last case were asked in the third session only. 62 Taking one subject at a time for each session, each individual in the study was asked to complete a background questionnaire. (See Appendix A.) Then the examiner gave the clinician detailed instruc- tions and had him practice using a sample SIMCASE and sample data identical in form to the ones he would use in the sessions. Next, the clinician was given some initial contact data on the case: (1) an artist's sketch of the student based on taped voice, (2) a taped interview with the student, and (3) written background information on the student. The subject was asked to request information on the SIMCASE and to use the data he requested in making his diagnosis. All requests were recorded on audio tape, and the examiner noted on the H/OCL the time of the request and the name of the requested information. The SIMCASE was a simulated case or set of information repre- senting a child with a reading problem. SIMCASE materials were con- tained in a small file box, and the examiner provided the information by handing the clinician the requested information. The clinician did notknow what materials were in the SIMCASE; i.e., he was not given a cue list. Subjects were given 45 minutes to request and use SIMCASE materials. Subjects could take notes if they wished, but they were not required to do so. Following the SIMCASE/clinician interaction, the clinician was asked to write, in sentence form, a diagnosis and remediation for the SIMCASE in the way he would usually write a diagnosis and remedia- tion. Special carbon paper was used, to make a duplicate copy of the subject's handwritten diagnosis and remediation. The clinician was 63 given 25 minutes to write a diagnosis and 25 minutes to write a remediation. Next, the clinician was shown a written diagnosis of a sample case for which diagnostic statements had been circled and numbered. The subject then practiced the task with another sample diagnosis. The clinician was subsequently given a sample RDCL and was instructed in its use. This was followed by practice with the sample check list in transferring the circled and numbered sample diagnosis to the RDCL sample. The clinician was then asked to circle and number, on one copy of his written diagnosis, all diagnostic reading statements. He then transferred these statements to the RDCL. The number from a written diagnostic statement was placed in the appropriate place on the check list, as either a strength, a weakness, or an observation, beside the statement on the check list that corresponded most closely to the clinician's own statement. An observation was defined as a nonvalue or neutral statement; e.g., "The student has brown hair and brown eyes." The circling and numbering of statements and the trans- fer to the check list were hpt_timed tasks. During a short break for the subject, the examiner arranged the materials the subject had requested, in the order in which they had been requested. After the break, the subject was asked to com- plete the H/OCL, responding to questions of hhy he had asked for a particular piece of information (cue) and hhat_the information had told him (hypotheses, observations, or hunches). The examiner recorded the order in which cues had been requested; the actual 64 materials requested provided the stimulus for recall. This was hp; a timed task. The above procedures were followed for each of the three clinical encounters, except that in the third or last session the clinicians were asked to respond to a short oral questionnaire regarding the third case. (See Appendix C.) The questions dealt with the clinician's Opinion about the student's ability in specific areas of reading. Also, the questionnaire was designed to determine if the clinicians used a "model“ in the diagnostic process (Sherman et al., 1978). Summary The major objective of this study was to determine the nature and extent of agreement of eight experienced reading clinicians as they collected cues, made diagnostic statements, and formulated hypotheses relative to making a diagnosis on simulated reading- problem cases. The eight clinician subjects were asked to interact with simulated materials representing a reading problem case and then to write a diagnosis and a remediation. The written diagnosis was transferred to the RDCL for objectification of the data. Responses to questions of hhy_certain case data were requested on each case and HEEL information was provided by those data were also applied to the RDCL. Additional procedures for the third or last session included questions designed to (l) elicit information regarding the clinician's 65 conception of the content or data base of a I'good" diagnosis, (2) determine the clinician's diagnostic routine, and (3) determine the source or schema that allowed the data base (memory) to be trans- lated into action (strategy). Other questions dealt with four areas of reading and how they could be defined for the third or last case each subject diagnosed. The four areas were (1) instant word recog- nition, (2) word analysis, (3) reading fluency, and (4) reading com- prehension. A11 stimulus materials used in the study, including cases and equivalent forms, were subjected to counterbalancing to minimize systematic effects. Subjects were randomly assigned to the case order. Subject reliability on the use of the RDCL, a previously unused instrument, was measured using a test-retest procedure. The Pearson product-moment correlation coefficient (r) was calculated as the measure of relationship. The value of r_ranged from .15 to .69. The Porter statistic, ATBTC" an index of the proportion of agree- ment that excludes clinician agreement not to select a cue or make a diagnostic statement, was also calculated on the same data. The Porter values ranged from .10 to .54. CHAPTER IV ANALYSIS MEASURES AND RESULTS The measures of clinical problem-solving behavior used in this research were deve10ped for this study as well as for the larger research project of which it is a part. Earlier work on problem solv- ing in medical diagnosis provided the framework for determining the appropriateness of the measures for investigating clinical problem- solving behavior in reading diagnosis (Norman, 1977; Elstein et al., 1976). Because of the limited sample size used in this study and the fact that the methodology and statistical measures are still Open to speculation and investigation, all findings and conclusions offered here must be considered tentative. Two major types of formal measures were used in the study reported here: (1) the product measures, which were intended to measure the outcomes of the clinical interaction between a reading clinician and a case (SIMCASE); and (2) the process measures, which were intended to measure the way in which the reading problem was diagnosed. A third type of measure used in the study was informal and dealt with clinic interviews, which pertained to clinicians' diagnos- tic Opinions regarding certain aspects of reading diagnosis. No 66 67 formal or statistical measures were employed to analyze the interview data. Rather, simple descriptive techniques were used. Product Measures Certain key behaviors govern the clinical interaction between a clinician and a case. The product measures were used to account for the results of the interaction, i.e., the data collected and the written diagnostic statements made by a given clinician to diagnose a specific case. The analysis of the resulting data is divided into four parts: 1. Proportional agreement, which is a measure of group agreement on cues collected and diagnostic statements made on the same case or an alternate form of the case; 2. Commonality, which is a measure of agreement between an individual clinician and a defined group of clinicians in terms of 'cues collected and diagnostic statements made on the same case or its alternate form; 3. Inter/intraclinician agreement, which is a measure of the agreement between one clinician and another clinician (or one clinician with himself) on the cues collected or the diagnostic statements made on the same case or on an alternate form of the case; and 4. Intraclinician agreement, which is a measure of the agreement of one clinician's cue collection for a case with his own cue collection for an alternate form of the case. The same analysis applies to diagnostic statements made. 68 The Proportional-Agreement Statistic The proportional-agreement statictic provides data on the similarities between the cues collected (or diagnostic statements made) by a group of clinicians for a given case. To determine simi- larities among cues and among statements, a standard for comparisons was first established. The standard developed for cues was the cue domain of data available on each particular SIMCASE. (See Appendix F.) The standard developed for statements was the statement domain or RDCL for all four cases. The statement domain was composed of categories and subcategories of diagnostic reading factors designed to encompass the spectrum of reading problems. The computation formula appears in Appendix I. Results of proportional agreement for cues collected.-- Table 4 contains the results for the proportional agreément statistic for the cues most frequently requested (by at least three of the six clinicians) for Case 1. Data for the other three cases appear in Tables H1, H2, and H3 of Appendix H. A sample cue domain for one case appears in Appendix F. Proportional agreement for a given cue was the proportion of those clinicians who diagnosed a particular case and who requested that same cue. For example, for Case 1 (Table 4), cue number 17, "Durrell Silent Reading (Test Booklet)," was requested by roughly 83 percent or five of the six clinicians diagnosing that case. It should be recalled that the clinicians did hpt_know what cues or data sources were available for a case. 69 Rm u mono mpamem>m pouch mcwucoumm owuz< u m< mucmssou m.cmcwsmxm u om mcowaumgwo ummh n o.— uwpxoom ”~me u E. mmLoom ummh n m._. uxmx Amhv augmeLoccma 26.;ummz m_ umaz om. em Amev _anca> Lm_;6863 A umaz om. Pm Ampv cowmcmgacasou mcweaam Avenues m mwoc=_wumz-mmomw _ wzw om. m4 Ampv xcms_ca--mecoz co steam: _6=ww> Ppmccso EN mzo om. om Ambv mwm»_m=< ago: new =o_o_=moumm ego: Fpmccao .N «so om. mm Amev meccaomws _chcsa ms man me. _~ Amev m=_oemm ocmpwm __mcczo a man mm. A, Am ;6_oo m Joe No. . N Aumv :oFBaELOCEH acetamm_u om gem mm. a e... .3 smug»... sun”. I ._ ammo .mmau cmuum_pou xpucmacwc» “mos co acwemmcmm :mqucwpu Co :owucoaocmuu.¢ mpamp 70 As indicated in Table 4, the most frequently requested cues for Case 1 were Examiner's Comments on Classroom Background Informa- tion (.83) and the Durrell Oral Reading (.83) and Durrell Silent Reading (.83) Test Booklets. Fourteen cues out of a possible 87 total cues available for Case 1 were requested by three or more clinicians. The most frequently requested form of information was Test Booklet (TB), which provided the actual test items. The clini- cians diagnosing Case 1 were apparently interested in seeing the kinds of errors the student made. The percentage of agreement on the most frequently collected cues (by three or more clinicians) is indicated, by case, in Table 5. The reported data tend to convey more agreement than actually occurred because only the most frequently requested (.50 to 1.0) cues were used to compute the proportion of agreement. The percentages in tptal_proportion of agreement (P.A.) for cues on four cases appear in Table H4 of Appendix H. As shown in that table, between 42 and 52 percent (an average of 48 percent) of the cues available were not collected on each of the four cases. An average of only 23 percent of the total cues available for Cases 1 through 4, respectively, was agreed upon by three or more (.50-1.0) of the six clinicians diagnos- ing each case. Results of proportional agreement for diagnostic statements.-- Table 6 shows results of proportional agreement for the diagnostic statements most frequently used in all four cases when the written diagnoses were converted to the standardized check list. (See Appendix C for the diagnostic statement domain check list for the 71 Table 5.--Percentages for proportion of agreement on cues most frequently requested for four cases. Case ggmgfigs 3 Sessions 4 Sessions 5 Sessions 6 Sessions Number Collected (P.A.=.50) (P.A.=.67) (P.A.=.83) . (P.A.=l.0) Case 1 14 43% 36% 21% 0% Case 2 24 29% 38% 33% 0% Case 3 27 44% 22% 26% 7% Case 4 21 48% 14% 14% 19% four cases.) The check list contained 169 diagnostic statements with three evaluative choices--strength, weakness, or observation--for each statement. Therefore, the check list domain had a total of 507 items. Proportional agreement for a given diagnostic statement was the pro- portion of those clinicians who agreed by mentioning that statement. For example, "Potential for grade-level work--reading (strength)" was mentioned by 50 percent (three out of six) of the clinicians for Case 4, was not mentioned for Case 2, and was mentioned only once each for Cases 1 and 3. The first column in Table 6 lists the diagnostic statements from the check list that were used in 50 percent (three out of six) of the diagnoses for a single case. The last four columns indicate, by case, the proportion of clinicians making each statement. For example, the first diagnostic statement, "Word recognition--general (weakness)," was used by 50 percent of the clinicians for Case 1, 33 percent for Case 2, 33 percent for Case 3, and 50 percent for Case 4. 72 Table 6.--Diagnostic statements most frequently selected from a standard check list.a Diagnostic Statement Proportion of Diagnoses Including From Standard Statement: Arrayed by Case VOCRPUIRVY CDECkIISF Case 1 Case 2 Case 3 Case 4 Word recognition--general W .50 .33 .33 .50 Rate of reading--si1ent W .50 .33 .33 .17 Intell/ed. potent.--general S .33 ... .50 .83 Progress in school-~reading W .50 .33 .50 .50 Comprehension--general W ... .50 .67 .17 Hearing--acuity W .50 .33 ... Attitude toward reading-- independent W .50 ... .17 .33 Basic sight words--score W .33 ... ... .67 Word analysis--general W ... .33 .17 .67 Phonetic analysis--general W .17 .17 .33 .83 Use of suffixes W ... .50 ... .33 Rate of reading--oral W .50 67 ... ... Word recognition--basic sight word W 33 .83 Vision--general statement 5 ... ... ... .50 Verbal intellectual poten. S .17 ... .17 .50 Potential for grade-level work--reading S .17 ... .17 .50 Emotional adjust.--general W ... ... .50 ... Visual discrim.--whole word W .50 ... ... .17 Word recognition--general O .17 .50 .17 ... Word recognition--basic sight word S .50 .17 ... Use initial consonant sounds W ... .67 ... .17 Use of blends--specific W .17 .50 ... .17 Comprehension--general S .83 .17 ... .17 Comprehension--general O ... .50 ... ... Oral reading-~general W .. .50 aStatements mentioned in 50 percent of the diagnoses for a single simulated case, or in the diagnoses for 50 percent of the cases, or both. Total diagnoses per case = 6. Key: S = strength (h =6) Proportion Diagnoses O = observation (h=2) 83 _ 5 W weakness (h_ 17) .67 = 4 .50 = 3 .33 = 2 .17 = 1 73 The data presented in Table 6 seem to indicate that the most frequently selected diagnostic statements over all cases were, equally, "Word recognition--general (weakness)" and "Progress in school--reading (weakness)." However, the statements with the highest agreement on any single case were "Intellectual/educational potential-- general (strength)" for Case 4, "Phonetic analysis--general (weak- ness)" for Case 4, "Word recognition--basic sight words (weakness)" for Case 4, and "Comprehension--general (strength)" for Case 1, each having 83 percent agreement or being selected in five out of six sessions for the particular case. The percentages in proportion of agreement for the most fre- quently mentioned diagnostic statements made in three (.50), four (.67), five (.83), and six (1.00) sessions are indicated, by case, in Table 7. Table 7.--Percentages in proportion of agreement for most frequently mentioned diagnostic statementsa on four cases. Case 3 Sessions 4 Sessions 5 Sessions 6 Sessions Number (P.A.=.50) (P.A.=.67) (P.A.=.83) (P.A.=l.0) Case 1 24% 0% 4% 0% Case 2 28% 8% 0% % Case 3 16% 4% 0% 0% Case 4 20% % 12% 0% Most frequent statements for four cases = 25. aN0 statements were mentioned in all six sessions for any case. 74 The results of proportional agreement for statements indi- cated that clinicians in this study had only slight agreement as a group on the diagnostic statements made for a given case. The low agreement for diagnostic statements might have been a result of the low reliability of the check list (.48), the length of the check list, and/or the number of check-list categories used to determine agreement. The percentages in hphal_proportion of agreement (P.A.) on diagnostic statements for four cases appear in Table H5 of Appendix H. As shown in the table, between 77 and 82 percent (an average of 80 percent) of the diagnostic statements appearing in the check list were not checked for the four cases. An average of only 1 percent of the same diagnostic statements in the check list were checked by three or more (.50-1.0) of the six clinicians diagnosing each case. The Commonality-Agreement Score The commonality score is intended to reflect the agreement of one clinician's collection of cues with the cues collected by a given group of clinicians on the same case (Vinsonhaler, 1979b). This score indicates the comparison between an individual clinician's cue collec- tion and a group of cues collected by all other clinicians for the same case. It accounts for which cues are collected by each clinician, and how many are collected. The same score can be calculated for diag- nostic statements. Results of commonality agreement.--The values for commonality agreement on cues collected and diagnostic statements made in six 75 diagnoses on each of four simulated reading cases are presented in Table 8. The data indicate that there was approximately 75 percent commonality or individual-to-group agreement on cues collected across all subjects and all cases. A comparison of the means for the indi- vidual cases indicated little variability between cases. The com- monality scores seemed not to be affected by case difficulty as both an easy case (Case 4) and a difficult case (Case 3) had the same average commonality score. The data in Table 8 also indicate that there was approximately 55 percent commonality agreement on diagnostic statements made across all clinicians and all cases. Table 8.--Commonality agreement on cues collected and diagnostic . . . a statements made 1n s1x d1agnoses on each of four cases. Case Cues Diagnostic Statements Number M_ Range §Q_ h_ Range .SQ l (héfi) .71 .48/.82 .13 .55 .37/.76 .16 2 (_E6) .74 .30/.93 .24 .54 .25/.66 .15 3 (_E6) .77 .39/.95 .20 .47 .28/.64 .15 4 (_;6) .77 .62/.86 .11 .62 .42/.78 .12 Mean total = .75 Mean total = .55 §Q_= .03 §Q_= .06 aCommonality is bounded by 0 and 1. Results of calculations shown in the commonality table sug- gested that when individual clinicians were compared with the group of clinicians diagnosing the same case, there was greater agreement, on the average, among clinicians in this study on cues collected than 76 on diagnostic statements made. Comparing cues and statements by case, it can likewise be shown that for any given case there was more commonality agreement on cues collected than on diagnostic statements made. Two tables of commonality (cues and diagnostic statements) showing frequency for intervals appear in Appendix H (Tables H6 and H7). Table 8 is a summary of the two commonality tables in the appendix. Inter/Intraclinician Agreement There are several ways to describe inter/intraclinician agreement for the cues collected and the diagnostic statements made in the clinical encounter. In this study, two different indices (the Phi coefficient and the Porter Index) were used, each providing some- what different information. In each instance, the value reported is by case and for the clinicians diagnosing a particular case or its ‘alternate form. The Phi, denoted by 0, is the traditional coefficient of correlation for nominal dichotomous data. One Phi coefficient was computed for each pair of clinicians. (See Appendix I.) When interpreting the results, it should be noted that because of an oversight in doing the statistical analysis for interclinician agreement, the pairs of scores for intraclinician agreement were also included. The interclinician analysis for each case should only have made calculations for subjects compared with other subjects diagnos- ing the same case or its alternate form, hp£_for the subject compared with himself. The inclusion of the jhhra_data (two comparisons per case) along with inter data in the analysis of agreement might possibly 77 have raised interclinician agreement slightly since intraclinician agreement was usually somewhat higher than interclinician agreement. The data were not reanalyzed because the Observational Study of 1977, used for comparison of the data in this study (see Chapter V), had the same analysis. The data for inter/intraclinician agreement in both studies will be reanalyzed later to exclude the intra portion. Comparisons of an individual to other individuals will be referred to as inter/intraclinician agreement in the dissertation text. The Porter Index, the second of the two indices used in this study to describe inter/intraclinician (and intraclinician) agreement, was ATBTC" which describes the proportion of agreement when the base was the total number of cues collected (or diagnostic statements made) for which one or the other or both clinicians collected the cue or made the diagnostic statement (see Wilcox, 1977, pp. 54-60). The upper bound of the index would be the value of the index K;%$%;fi-, which describes the pr0portion of agreement when the base was the ‘hppal number of cues in the particular cue domain (or diagnostic statements in the statement domain). The K;%;E-index excludes clinician agreement hpt_to request a cue or make a diagnostic state- ment, i.e., the "d'I cell (--) in the 2 x 2 contingency table. In general, the values of the Porter Index would be expected to be some- what lower than the values of Phi when the "d" cell (--) is large. Results of inter/intraclinician agreement.--The data presented in Table 9 show results of calculations for inter/intraclinician agree- ment on cues collected for four simulated reading cases. Both of the indices reported are for the same data. Table 9 is a summary of 78 Table H8, which appears in Appendix H. Table 9 indicates that there was consistently low agreement on cues collected for all four cases. Results of calculations using the two statistics do not appear to differ greatly, as indicated by the means and standard deviations for individual cases and totals. In considering case difficulty, it should be noted that the easiest case (Case 2) and the most diffi- cult case (Case 3) had mean Phis of .35 and .38, respectively. Table 10 is a summary of Table H9 in Appendix H. The summary table shows the means and standard deviations for inter/intraclinician agreement on diagnostic statements for four cases. The statistics used were the Phi coefficient and the Porter Index. As indicated in the table, there was consistently low inter/intraclinician agreement on diagnostic statements for the four cases, with slight differences between the values for the two measures reported. The data indicate very low to almost no agreement (Case 3) for subjects on diagnostic statements made for the four cases. Case 3 was the most difficult of the four cases and may account for the lower diagnostic agreement for that case. A comparison of data for cues and diagnostic statements across all clinicians and all cases indicates lower agreement for diagnostic statements made than for cues collected. Table 11 shows the intraclinician agreement statistics (Phi and Porter), by subject and case, for cues collected on four cases. The third column on the left side of the table shows the value for the Phi coefficient for each subject, when compared with himself, in terms of cues collected, on alternate forms of the same SIMCASE. The first 79 Table 9.--Means and standard deviations for inter/intraclinician agreement (Phi and Porter) on cues for four cases. Case R Phi Coefficient R Porter Index Number ange M 50 ange M SD 1 (flf15) -.O4/.55 .30 .18 .09/.46 .28 .12 2 (hElS) -.18/.87 .35 .28 .05/.83 .36 .20 3 (hfls) .04/.63 .38 .15 .09/.54 .33 .12 4 (hElS) .21/.60 .42 .11 .23/.52 .37 .08 Mean total = .36 Mean total = .34 §Q_= .05 §Q_= .04 Table 10.--Means and standard deviations for inter/intraclinician agreement (Phi and Porter) on diagnostic statements for four cases. Case R Phi Coefficient Porter Index Number ange M. ED. Range U. _§Q 1 (flfl5) .00/.28 .10 .08 .02/.17 .07 .05 2 (flF15) -.03/.43 .12 .11 .00/.29 .09 .07 3 (fl?15) -.05/.26 .07 .08 .00/.16 .06 .04 4 (hElS) .04/.31 .16 .07 .05/.20 .11 .05 Mean total = .11 Mean total = .08 §Q_= .04 §Q_= .02 80 column on the right side of the table provides the values for the Porter statistic on the same data used to compute the Phi coeffi- cients presented in the third column of the table. Table ll.--Means and standard deviations for intraclinician agreement for cues collected on four cases. Case Subject Phi Coefficient Porter Index Number Number M. §Q_ fl_ .§Q 103 .05 .13 1 107 .42 .24 .26 .36 .25 .16 102 .50 .42 2 108 .87 .69 .26 .83 .63 .29 101 .43 .32 3 105 .38 .41 .04 .38 .35 .04 104 .60 .52 4 106 .39 .50 .15 .36 .44 .11 Mean total = .46 Mean total = .42 §Q_= .19 SQ.= .16 1 More Difficult Most Difficult N II II #00 II II Easiest Second Easiest As the data in Table 11 suggest, there was considerable varia- bility among clinicians in this study in self— or intraclinician agreement on cues collected. The Phi ranged from .05 for Subject 103 on Case 1 (difficult) to .87 for Subject 108 on Case 2 (easiest). These differences might be case specific (i.e., relating to case dif- ficulty) or a result of subject differences, since the other reported 81 Phi for Case 1 was .42, which was more ‘hi line with the values for the other cases. Additionally, the data for intraclinician agreement on cues indicate that, on the average, the reading clinicians in this study showed only limited agreement with themselves on the cues they col- lected on alternate forms of the same case. The notable exception was Subject 108 for Case 2, who had a Phi value of .87 in agreement with himself on cues collected for the same case, alternate forms. This would indicate that Clinician 108 collected 87 percent of the same cues at Time 1 and Time 2 for the same case (alternate forms). However, since Case 2 had been determined to be the easiest of the four cases, one can speculate about the effect case difficulty might have had on Clinician 108's self-agreement. Although it has not been determined what the value of Phi §h9p1g_be clinically, statistically the low average intraclinician agreement on cues collected (Phi = .46) seems to indicate that, in general, the clinicians in this study followed no particular diagnos- tic routine or strategy in collecting information or data on a case in order to arrive at a diagnosis. However, individual clinicians, 108 for example, did appear to have a strategy or routine, as reflected in the agreement Phi for cues (Phi = .87). Case difficulty, however, might be an intervening variable that affected the value of Phi in this study. Except for Case 1, the values of the Porter Index appear to be somewhat lower than the values reported for Phi. 82 Table 12 shows the intraclinician agreement (Phi and Porter) on diagnostic statements for both the clinic and the home conversion of diagnoses to the RDCL for four cases. The reliability of the two conversions (clinic and home) of the written diagnosis to the RDCL was reported in Chapter III, the mean reliability being .48 for the two conversions (.49 if z scores were used). Table 12.--Intraclinician agreement for two conversions (clinic and home) of written diagnoses to the RDCL. Case Subject Phi Coefficient Porter Index Number ”“mbe'” Clinic Home It §p_ Clinic Home h _s_p 1 103 .28 .40 .34 .08 .17 .26 .22 .06 107 .26 .31 .29 .04 .17 .21 .19 .03, 2 102 .43 .41 .42 .Ol .29 .27 .28 .01 108 .21 .22 .22 .01 .14 .14 .14 .00 3 101 .26 .30 .28 .03 .16 .20 .18 .03 105 .01 .12 .07 .08 .03 .09 .06 .04 4 104 .31 .36 .34 .04 .20 .23 .22 .02 106 .26 .46 .36 .14 .17 .32 .25 .11 Table 13 shows the intraclinician diagnostic statement agre ment statistics for the diagnosis of four simulated reading cases. e- In general, the data indicate that the reading clinicians for this study showed only limited agreement with themselves on diagnostic statemen made for the same case. Additionally, it can be stated that althoug the overall mean intraclinician agreement Phi (.26) and Porter (.17) were limited, they were higher than the overall group mean or ts h B3 inter/intraclinician agreement Phi (.11) and Porter (.08) on state- ments. (See Table 10.) Thus, it appears that the reading clinicians in this study showed more agreement with themselves on diagnostic statements made for the same case than the group as a whole agreed with each other; i.e., the individual intra agreement was higher than the group inter/intra agreement. Table 13.--Means and standard deviations for intraclinician agreement for diagnostic statements on four cases. Case Subject Phi Coefficient Porter Index Number Number U. §2, U. §Q 103 .28 .17 l 107 .26 .27 .01 .17 .17 .00 102 .43 .29 2 108 .2] .32 .16 .14 .22 .11 101 .26 .16 3 105 .0] .14 .18 .03 .10 .09 104 .31 .20 4 106 .26 .29 .04 .17 .19 .02 Mean total = .26 Mean total = .17 SD = .08 SD = .05 In addition, when the values of intra Phi for statements (Table 13) are compared with the values of intra Phi for cues (Table 11), it can be seen that, on the average, subjects for this study agreed more with themselves on the cues they collected (.46) 84 for the same case (alternate forms) than they agreed with themselves on the diagnostic statements made for the same case (alternate forms). Looking at the data in Table 13 for each subject, it appears that Subject 105 had almost no agreement with himself on the diagnos- tic statements made for the same case at Time 1 and Time 2. Again, as with the intra agreement data on cues (Table 11), the low agree- ment on statements may be attributable to subject (clinician) dif- ferences or to case difficulty, since Subject 105 diagnosed the most difficult case. It should be noted that the lowest intra agreement Phi value (.05) for cues was for Subject 103, who diagnosed the first difficult case (Case 1), whereas the lowest intra agreement Phi value (.01) for statements was for Subject 105, who diagnosed the most difficult case (Case 3). The highest Phi value (.43) for intra agreement on statements was for Subject 102, who diagnosed the easiest case (Case 2). The highest Phi value (.87) for intra agreement on cues collected was for Subject 108, who also diagnosed the easiest case (Case 2). The Inquiry Theory states that the greater the similarity of clinical memory (problem, cue values, prescription and treatment descriptions, and the relations between them), the greater the agree— ment of diagnoses. One descriptive part of this theory represented by the agreement corollary states that the agreement in cues collected by the same individual diagnosing alternate forms of the same case is greater than or equal to comparisons made between individuals (the group) diagnosing alternate forms of the same case. (See Chapter 1.) Therefore, if the corollary holds, the intraclinician-agreement 85 measures should yield equal or higher values than the inter/intra- clinician-agreement measures. A summary of the agreement statistics on cues collected for four cases is presented in Table 14. The data indicated that when the inter/intraclinician Phi was compared by case to the intraclini- cian Phi, the intraclinician values were higher for Cases 2, 3, and 4. However, Case 1 showed a higher mean Phi for inter/intraclinician agreement (.30) than for intraclinician agreement (.24). Therefore, it would appear that for cue agreement, the corollary held for Cases 2, 3, and 4 but did hp; hold for Case 1. In terms of the Inquiry Theory, it would seem that the clinicians diagnosing Case 1 did not share a memory for cues. Whether or not this variability is a result of case difficulty and/0r subject differences is not clear. One might ques- tion what is unique about Case 1 and/or the subjects diagnosing that case. The second corollary of the Inquiry Theory postulates that the common elements between diagnostic statements made for two forms of the same case by the same clinician should be greater than or equal to diagnoses prepared by different clinicians on the same case. This would indicate that the intra agreement for diagnostic statements should be equal to or higher than the inter/intraclinician agreement. A summary of the agreement statistics for diagnostic state- ments on four cases is presented in Table 15. As the data indicate, the intraclinician (individual) agreement for diagnostic statements was higher than the inter/intraclinician (group) agreement for 86 diagnostic statements on all four cases. Therefore, the corollary held and the clinicians shared a memory, albeit a limited one, for diagnostic statements. Table 14.-~Agreement statistics on cues collected for four simulated reading cases. Statistic for CUES COIIECIEd Average Over Agreement Case 1 Case 2 Case 3 Case 4 Four Cases No. clinicians 4 4 4 4 No. diagnoses 6 6 6 6 Commonality score Mean .71 .74 .77 .77 .75 Std. dev. .12 .24 .20 .11 .03 Inter/intra.diagnosis correlation (Phi) Mean .30 .35 .38 .42 .36 Std. dev. .18 .28 .15 .11 .05 Inter/intra diagnosis (Porter) Mean .28 .36 .33 .37 .34 Std. dev. .12 .20 .12 .08 .04 Intra-diagnosis correlation (Phi) Mean .24 .69 .41 .50 .46 Std. dev. .26 .26 .04 .15 .19 Intra-diagnosis (Porter) Mean .25 .63 .35 .44 .42 Std. dev. .16 .29 .04 .11 .16 87 Table 15.--Agreement statistics for diagnostic statements made on four simulated reading cases. Statistic for Diagnostic Statements Made Average Over Agreement Case 1 Case 2 Case 3 Case 4 . Four cases No. clinicians 4 4 4 4 No. subjects 6 6 6 6 Commonality score Mean .55 .54 .48 .62 .55 Std. dev. .16 .15 .15 .12 .06 Inter/intra diagnosis correlation (Phi) Mean .10 .12 .07 .16 .11 Std. dev. .08 .ll .08 .07 .04 Inter/intra diagnosis (Porter) Mean .07 .09 .06 .11 .08 Std. dev. .05 .07 .04 .05 .02 Intra-diagnosis correlation (Phi) Mean .27 .32 .14 .29 .26 Std. dev. .01 .16 .18 .04 -08 Intra-diagnosis (Porter) Mean .17 .22 .10 .19 .17 Std. dev. .00 .ll .09 .02 .05 Product Measures: Summary The product measures data presented in this chapter seem to suggest the following: 88 l. Twenty-three percent of the same cues from the total cues available on the four cases were collected by three or more (P.A. = .50-1.0) of the six clinicians diagnosing each case. 2. Fifty-two percent of the cues available were collected across all four cases. 3. One percent of the gang diagnostic statements in the check list were checked by three or more (P.A. = .50-1.0) of the six clinicians diagnosing each case. 4. Twenty percent of the diagnostic statements in the check list were checked for the four cases. 5. The commonality scores indicated that the clinicians for this study were consistently high, 75 percent, in their agreement on cues collected across all cases and all subjects. 6. The commonality scores also indicated that the subjects ,for this study agreed, on the average, 55 percent of the time on the diagnostic statements made across all cases and all subjects. 7. The inter/intraclinician agreement Phi for cues indicated little variability between cases on cues collected (M_= .36, §Q_= .05). 8. The inter/intraclinician agreement Phi for diagnostic statements indicated little variability between cases (h_= .11, §D_= .04) but very low agreement. 9. The intraclinician agreement Phi for cues was not consis- tent, varying from .24 for Case 1 to .69 for Case 2. Totals were h: .46, pp: .19. 89 10. The intraclinician agreement Phi for diagnostic state- ments was not consistent, varying from .14 for Case 3 to .32 for Case 2. Totals were h_= .26, SQ.= .08. 11. The intraclinician agreement Phi on cues collected was higher than the inter/intraclinician agreement Phi on cues collected for Cases 2, 3, and 4 but hpt_for Case 1. 12. The intraclinician agreement Phi on diagnostic state- ments was higher than the inter/intraclinician agreement Phi on diag- nostic statements made for all four cases. 13. With two exceptions, the Porter Index yielded slightly lower scores than did the Phi for both inter/intraclinician and intraclinician agreement on cues and diagnostic statements. The exceptions were the lower inter/intra Phi on cues for Case 2 and the lower intra Phi on cues for Case 1. 14. The clinicians for this study were lower in agreement (both Phi and Porter) on diagnostic statements made than on cues col- lected. Process Measures In the clinical encounter there are key behaviors that govern the interaction between a clinician and a case. The process measures deal with the behavioral dynamics of the clinical interaction in terms of the dependent, time-related variables. These variables include such time-dependent data as (l) the length of time for inter- action with a case, (2) the number of cues collected and diagnostic statements made, (3) when in the session the cues are collected and 90 diagnostic statements are made, (4) the average time that cues and statements are made in the interaction, and (5) what relationships, if any, exist between cues collected and statements of hypotheses made regarding diagnosis. The process measures thus reflect the HEX. in which reading clinicians collect information on a case and subse- quently arrive at diagnostic decisions. In terms of the Inquiry Theory, process measures deal with the strategy used by a clinician in diagnosing a case. The analysis measures for process are based on the data obtained in the observational and debriefing parts of the clinical sessions. Results of process measures data analysis will be presented according to three major types of descriptive statistics. These were developed to answer questions relative to clinical problem-solving strategy, or the manner in which a clinician arrived at a diagnosis. The three types of statistics and the results presented are: 1. Basic process statistics, which summarize data relevant to the times that cues and statements first appeared in the clinical interaction for individual sessions and for combinations of sessions. 2. Process-agreement statistics, including correlation and partitioned Phi coefficients, which are intended to reflect clini- cians' agreement on the time/order in which cues are requested and diagnostic statements made. 3. Cue-to-statement relationship statistics, which are intended to indicate the degree of relationship between cues and statements of hypotheses as suggested by the observational session 91 and debriefing data (e.g., whether certain cues are more frequently used to confirm certain hypotheses). Basic Process Statistics The basic process statistics are calculated to provide a summary of the dependent variables for each clinical session. The basic process statistics reported for this study include: 1. total elapsed time of session in minutes; 2. total number of cues collected; 3. average time that cues were collected in a given session, shown as a fraction of the total elapsed time of the session; 4. the standard deviation of the average time cues were collected in a given session; 5. total number of diagnostic statements mentioned; 6. average time that statements were made in a given session, shown as a fraction of the total elapsed time of the session; and 7. the standard deviation of the average time that state- ments were made in a given session. Table 16 shows the basic process statistics data for four simulated cases in reading. The first column in the table lists the case numbers. The second column indicates the statistics (mean and true standard deviation) that are reported. Column 3 indicates the mean and true standard deviation for total elapsed time of the sessions in minutes for each of the four cases. The fourth and fifth columns show the mean and true standard deviation of the total num- ber of cues collected and statements made, respectively, for each of 92 memeu m _ e N <11111121211111_ opseeecwo one: omeemea epeem sopsewecwo emeu so. 40. mo. mo. e~.e no.“ om.PF u aw, .1 Pm. me. On. No. ~_._m me.FN “p.8m u a Aenev 4 mo. mo. mo. mo. mp.op cm.m om.o ".mw .m Pm. mm. Fm. mm. oo.e~ mm.e~ om.em u 2 Am- v m mo. so. mo. mo. ep.e mm.e em.e ".mm .m 3. 3. mm. S. 8.8 D.- 2.3” u z 3. v N mo. No. Po. mo. we.m Ne.. _e.m ".mw -m om. mm. .m. me. Ne.PN ~_.mp mm.e~ 1.2 fie- v F mac “my ARV MES Amv Rev Amy ANV A_v ewe meme deep s_e acmeamU—m ucmsmu mum mac 0:0 mucwfiwwmum PWWMW FMWWW mu mum Scum swam—””2 .>mo .uum mmmcm>< .>mo .npm wmmcw>< .mcwuwmc c? mmmmo umum_=ewm Lao» com sumo muwumwuepm mmmoocq meem11.mp m~amh 93 the four cases. In columns 6 and 7, the first rows for each case indicate the mean (average) and variation (shown as standard devia- tion) (Hi the average time that cues were collected in the specific sessions for each of the four cases; the second rows for each case indicate the true standard deviation for each value in the two col- umns. The values in the sixth column indicate the fraction of the total mean elapsed time for cues (and the true standard deviation of the mean time for cues) of the specific sessions for each case. For example, if the mean of the average cue fractional time was .50, then the mean of the average cue-collection time was exactly halfway through the session. This would indicate that, on the average, the same number of cues was collected by the clinicians for a particu- lar case in the first half as in the last half of a session. In columns 8 and 9, the first rows for each case indicate the mean (average) and variation (shown as standard deviation) of the average time that statements were made in the specific sessions for each of the four cases; the second rows for each case indicate the true standard deviation for each mean in the two columns. The values in the eighth column indicate the fraction of the total mean elapsed time for diagnostic statements (and the true standard deviation of the mean time for diagnostic statements) of the specific sessions for each case. For example, if the mean of the average statement frac- tional time was .25, then the mean of the average statement collection time was one quarter of the way through the session. This would indicate that, on the average, the clinicians made more statements 94 for a particular case in the first quarter than in the last three quarters of a session. The average total time of the session, approximately 27 minutes, was less for Case 1 than for the other three cases, all of which had similar mean values. Case 1 also had, on the average, fewer cues collected and fewer diagnostic statements made than the other three cases. The lower mean totals for time, cues, and state- ments could possibly be a result of case differences. Before data col- lection, Case 1 had been determined to be the second most difficult of the four cases. (See Chapter III.) The means of the average cue frac- tional times for Case 1 and Case 2 were .43 and .49, respectively. This would indicate that, on the average, more cues were collected in the first half than in the second half of a session for Case 1 and Case 2. The means for average cue time for Case 3 and Case 4 indicate that, on the average, more cues were collected in the second half than in the first half of a session for both Case 3 and Case 4, with average cue times of .55 and .62, respectively. Case 1, the second most difficult case, had the lowest value for the four cases on average cue time. The mean of the average statement fractional time for Case 1 (.39) was the lowest value of the four cases. Process-Agreement Statistics One of the assumptions of the Inquiry Theory, other factors held constant, is that clinicians whose memories and strategies are held in common should have common behavior in terms of cues collected and diagnostic statements made. The process~agreement statistics 95 are designed to reflect those behavioral agreements when several clinicians diagnose the same SIMCASE or when one clinician diagnoses an alternate form of the same SIMCASE. Three types of statistics comprise process measurement: correlation statistics, partitioned Phi coefficients, and cue-to- statement relationship statistics. Correlation.--The statistics that are calculated to deter- mine the agreement between an individual and a group are based on the correlation coefficients for each session: (1) between one clini- cian's cue times and the average of the other clinicians' cue times and (2) between one clinician's statement times and the average of the other clinicians' statement times. The statistical measure used to determine agreement was the Pearson pr0duct-moment correlation coef- ficient (rxy). The r_to Z_transformation was not used because data were based on a normal distribution. All correlation coefficients were calculated to include the times of those cues (and statements) that one clinician collected and no other clinician collected (or mentioned) for the same case, i.e., missing data. Table 17 shows average r_and standard deviation for cue times, including cues that one clinician collected and no other clinician collected (i.e., missing data) for four cases. The table indicates that the highest mean correlation for cue times occurred for Case 4, the value being .45. Case 4 was the second easiest case. The mean correlations for each of the other three cases had somewhat lower but more consistent values. The tables for each case are pre- sented in Appendix H. (See Tables H10, H11, H12, and H13.) The 96 data indicate that the magnitude of the relationship between the average individual cue time and average group cue time, including those cues collected by the individual but not the group, was low mh=.%L Table 17.--Means and standard deviations for value of r_on ppg_times, including missing data, for four cases. Nfigggr M“: §Q l .30 .17 2 .32 .15 3 .36 .18 4 .45 .13 Mean total = .36 §Q_= .07 Table 18, which is based on the data from Tables H14, H15, H16, and H17 for each case (see Appendix H) shows the average h_and standard deviation for diagnostic statement times, including diagnos- tic statements that one clinician made and no other clinician made (i.e., missing data) on the same case. Data are presented for four cases. As the table indicates, the overall mean correlation value for diagnostic statement time was fairly consistent across all cases. The data indicate that the magnitude of the relationship between individual statement time and group statement time, includ- ing those statements made by the individual but not by the group, was low (h_r_= .24). However, the direction of the relationship was 97 positive. The highest correlation for diagnostic statement time was h_h_= .30 for Case 4. Looking at the correlation for diagnostic state- ment time (Table 18) and the correlation for cue time (Table 17), it can be seen that the individual clinicians compared with a group of clinicians diagnosing the same case showed greater magnitude in relationship for cue time than for statement time. Table 18.--Means and standard deviations for value of r.on diagnostic statement times, including missing data, for four cases. Nfigggr M-r’ §9— 1 .20 .07 2 .25 .07 3 .22 .09 4 .30 .07 Mean total = .24 SQ = .04 Partitioned Phi coefficients.--The partitioned Phi coefficient reflects agreement among clinicians on the time/order in which diag- nostic statements made by one clinician are compared to diagnostic statements made by other clinicians on the same case; i.e., inter/intra- clinician agreement. The coefficients for partitioned Phi are calcu- lated in a manner similar to that used in the product analysis (see Inter/Intraclinician Agreement) except that for partitioned Phi coefficients the contingency tables are calculated within four time periods or quarters instead of across the whole session. For example, 98 if a clinician mentioned a particular diagnostic statement during the first quarter of any session, the time of that diagnostic state- ment was included in the calculations for the first quarter. The four partitions represent clock time into the session; i.e., Parti- tion 1 includes the first 25 percent or first quarter of the time of the session and Partition 2 includes 50 percent or half of the time of the session. It should be noted that the data used for time partitions were based on subjective decisions regarding the diagnostic statements the clinicians made on the H/OCL. (See Procedures in Chapter III.) The researcher identified and coded each response that was considered a diagnostic statement (the identification of a factor or variable that helps determine the state or condition of a student's reading performance) using the RDCL. Diagnostic statements for which no value judgment (strength or weakness) was made (e.g., "I was looking at instant word recognition.") were coded as observation statements. An attempt was made to represent the clinician's intent by referring to the way in which the clinician matched his own written diagnosis to the RDCL. Table 19 shows the mean and standard deviation of Phi for four time partitions across four cases. As indicated in the table, the agreement among clinicians in the time/order that statements were made in the session was very low, showing mostly negative or no agreement. The low values appeared to be consistent across all cases for the four time partitions. In view of the previously reported 10w agreement of clinicians on diagnostic statements, it might not be 99 surprising to find low partitioned Phi coefficients. However, the degree to which the clinicians showed lack of agreement on the par- titioned Phi might to some extent be accounted for by the subjective coding used in analyzing the data. Table 19.--Inter/intraclinician Phi on four time partitions for diagnostic statements and four cases. Case Partition 1 Partition 2 Partition 3 Partition 4 Number 11 59 u 99 .11 s_0 u 99 l .01 .17 .02 .26 .01 .23 .03 .22 2 -.05 .23 -.01 .14 -.01 .20 -.02 .15 3 -.05 .20 -.01 .20 -.01 .18 -.05 .ll 4 -.O6 .15 .00 .21 -.00 .17 .01 .15 Cue-to-Statement Relationship Statistics To compare the cue-to-statement relationship statistics across sessions required classification of the relationship between cues collected and diagnostic statements of hypotheses made. The classification method used for this study can be seen in column 2 (Relationship) of Table 20. The cue-to-statement relationship sta- tistics were used to determine if there was a pattern in the rela- tionship between the cues collected by a given clinician on a specific case and the statement of hypotheses (or other diagnostic statements) made by that same clinician on the same case. Unlike research involving physicians, in research of reading clinicians in clinical diagnosis it is very difficult to distinguish 100 between statements of hypotheses (i.e., statements of high utility in reaching a problem solution) and simple observational statements (i.e., statements of low or zero utility in reaching a problem solu- tion). Whereas physicians are accustomed to dealing with a more precise and standardized vocabulary and more exact data (i.e., blood tests, urinalysis, etc.), reading clinicians lack a standardized vocabulary for diagnosis, and their diagnostic data often include informal, subjective tests. In reading, predictions concerning the effect of clinical memory and clinical strategy on the generation and use of hypotheses to direct clinical inquiry are still conjectural in terms of explication of the Inquiry Theory. Therefore, the use of early hypothesis generation to determine patterns of clinical strategy (hypothetico-deductive approach), as suggested in the expla- nation of the Inquiry Theory in Chapter I (see Hypothesis-Generation Corollary), are still under investigation in the field of reading. However, although this researcher did not intend to investi- gate in depth the use of the hypothetico-deductive approach in the diagnostic inquiry of reading clinicians, it was of interest and import to provide preliminary data in this area to facilitate future research efforts. The method chosen in this study for initial investigation of the hypothesis-generation corollary of the Inquiry Theory in reading involved a number of subjective decisions. Those decisions included (1) selecting and coding relationship responses on the H/OCL, (2) com- bining responses to fit the number allowed by the computer program, 101 and (3) classifying the relationship responses into two major cate- gories for analysis. The data for cue-to-statement relationship statistics repre- sent average values on a scale of O to 100 for percentage of times a particular relationship appeared for cues or diagnostic statements. For example, in Table 20, Subject 103, Run 3, indicated that he had a "hunch" 12.5 percent of the time for cues on Case 1. On the other hand, Subject 107, Run 3, only indicated that he had a "hunch" 5.56 percent of the time for cues on the same case. The relationship statistics were designed to reflect clinical strategy as defined by the Inquiry Theory. The relationship analyses used in this study concerned (1) the pattern of relationship between cues (or diagnostic statements) and hypotheses or observations made by the clinician during the clinical session and (2) the average of percentage of number of times each relationship occurred for a given group of cues (or diagnostic statements). Data for cues are presented first. Vinsonhaler (1979b) reported that two types of strategies appear to characterize the behavior of clinicians in the clinical encounter: first, the strategy in which problem solvers tend to direct the inquiry process by the use of hypotheses about the problems of a case--i.e., cues or information is gathered to tg§t_the specific hypotheses; and second, the strategy in which problem solvers use cue collection to direct their inquiry--i.e., certain types of cues are collected and thgh_statements (including diagnostic statements) are made. The data for the relationships of cues and diagnostic 102 statements to statements of hypotheses were interpreted in light of these two types of strategies. On the H/OCL, which the clinician subjects were given in the debriefing part of the clinical session, only relationship numbers 1-8 appeared as choices, and subjects were permitted to check more than one number. Later, when the program for analyzing the data was developed, it was necessary to assign additional numbers (9 through 14) in those instances in which more than one relationship had been checked. Subjects had not checked more than two relationship numbers within the two groups (relationship numbers 1-4 and 5-8) of the original eight relationship numbers. The relationship responses on the H/OCL that were used in this research to be indicative of using a variable set of hypotheses to direct inquiry were Hunch (#1), Confirm Hunch (#5), Disconfirm Hunch (#6), Hunch and Confirmed (#9), and Hunch and Disconfirmed (#10). The relationship responses that were determined to characterize cue- directed or discovery-type inquiry included Just Wanted Information (#2), Usually Get Information (#3), Other (#4), Suggest Hunch (#7), Other (#8), Wanted Information and Confirmed (#11), Wanted Information and Suggested Hunch (#12), Usually Get Information and It Confirmed Hunch (#13), and Usually Get Information and It Suggested Hunch (#14). Table 20 is an example, using Case 1, to show the average of percentage of number of times each relationship from the H/OCL appeared for cues collected across all statements during the clinical session. The percentages for hypothesis-directed inquiry and cue- directed inquiry should add to 100 jf_a relationship occurred for 1133 .cc_ C0 C CO w_000 :0 000000 III -yl. I'll-.- .. I 1.0 l.'1IIIIIII. I 1 1 1 I I I- I1II-.IIII 00.0 00.0 00.0_ 0.0 0.0 0.0 0.0 00.0 00.0 00000 000000000 0 000 0.00000 A0+0000 00.0 00. 00.0 0.0 0.0 0.0 00.0 0.0 0.0 0000: 000000000 0 000 00.0000 00.000— 00.0 00.. 00.0 0.0 0.0 0.0 0.0 0.0 00.0 00000 000000000 0 .000_ A0+000_ 00.0 00.0 00.0. 0.0 0.0 00.0 00.0 00.0 0.0 000000000 0 .000_ 000000 00000.. 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 000000000000 0 00000 00+0000 00.0 00.0 .0.00 00.00 00.00 0.0 00.0 0.0 0.0 000000000 0 00000 A0+_00 00.0 00.0 .0... 0.0 00.0. 0.0 0.0 0.0 0.0 00000 0 00.00 00.00 00.00 00.0 00.0_ 00.0 0.0 00.00 00.00 00000 0000000 0 00.0 00.0 00.00 0.0 00.0 00.0 00.0 0.0 0.0 00000 0000000000 0 00.0. 00.00 00.000 00.00 00.0 00.00 _0.00 0.0 00.0 00000 0000000 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 00000 0 00.0 00.00 00.00. 00.0. 00.0. 00.00 00.00 00.00 00.00 .0000 000 00.0000 0 00.0 00.0 00.00 00.0 00.00 00.00 0.0 00.0 0.0 .0000 000000 0000 0 00.0 00.0 00.00 00.00 00.0 00.0 00.0 00.00 0.0 0000: _ cw z —0000 m cam m cam m cum 0 cam m czm _ cam a_:0:owuo—mm cwaE=z 000 00. 0000000 00. 00_ 0000000 00. 0000000 000 0000000 000 0000000 000 0000000 IMIIIIIHIIIRF. I hllhdllfllh "Ft“.."bhhlrt I. "M” rnvrhvnlr 0.0 m00u--0000 000 00000000 0000:0000000 some 0050» Co 0005:: 00 momucmocma 0o moecm><11.om 0.000 Idn").114 "1‘ H '4 I .1 H.“. 104 each cue and each diagnostic statement mentioned. However, in this study there was not always a one-to-one match or relationship between cues and hypotheses (or other statements). In Tables 21 and 23, those percentages out of lOO for which relationships occurred are shown in the last column of each table. A look at Table 20, by relationships, indicates that for Case 1 the most frequently used relationship for cues was relationship number 3, Usually Get Infor- mation. The second most frequently used relationship was number 5, Confirm Hunch. Looking at the data in the table by subject, it appears that when subjects were asked why_they collected certain information (relationship numbers l-4), in general, most of them indicated that they' either just wanted the information or usually got the informa- tion for a case they were diagnosing. This would seem to indicate that, in general, the subjects diagnosing Case l said that they used a strategy of cue collection in which certain cues were collected and then statements made (including diagnostic statements) concerning the information provided by the cues. When asked what_the information told them (relationship numbers 5-8), in general, most of the clini- cians diagnosing Case 1 responded that the information confirmed a hunch or suggested a hunch. Tables for Cases 2, 3, and 4 appear in Appendix H. (See Tables Hl8, Hl9, and H20). By combining relationships into two categories of inquiry, it can more clearly be shown how the individual subjects responded to questions on the two types of strategies used to characterize the clinical inquiry of the eight clinicians in this study. Table 21 105 is a summary by subjects across all runs for all cases, using means and standard deviations, for average of percentage of number of times relationships appeared for gu§§_in hypothesis-directed inquiry and cue-directed inquiry responses. The responses for relationship numbers 1, 5, 6, 9, and 10 were used to characterize hypothesis- directed inquiry appearing for cues. The responses for relationship numbers 2, 3, 4, 7, 8, ll, 12, l3, and T4 were used to characterize cue-directed inquiry appearing for cues. Looking at Table 2l for the average of percentage of number of times relationships appeared for cues, Subject l0l indicated the greatest use of hypothesis-directed inquiry. However, Subject l05 indicated the second highest use of hypothesis-directed inquiry and was somewhat more consistent across all three session runs. In the same table, the mean cue-directed inquiry responses for the average of percentage of number of times relationships appeared for cues was highest for Subject l03, indicating an information-gathering approach in clinical diagnosis. The other subjects seemed to indicate use of combinations of hypothesis-directed and cue-directed inquiry on rela- tionships appearing for cues. Case differences for the two types of strategies can be shown by finding the overall average by case on responses for (l) hypothesis- directed inquiry items and (2) cue-directed inquiry items. Using a scale of O to l00, the average or mean for each subject on each case was computed for the hypothesis-directed inquiry responses and the cue-directed inquiry responses. Table 22 shows means and standard deviations across all subjects and cases for average of percentage 106 Table 21.--Average of percentage of number of times relationships appeared for cues using hypothesis-directed and cue-directed inquiry responses across all subjects and all cases.a Average Average . Hypothesis- for 3 Cue- for 3 fl in Subject Case Run . . . . . b Directed Ses51ons Directed Sess1ons Relat1on- Number Number Number Inquiry & Inquiry & ship (if!) (§2) lOl 3 1 58.33 33.33 91.66 181 3' 3 68.11 ?;'(g) 27.54 fg'gg) 95.65 181 4 2 76.67 ° 23.33 ' lO0.00 182 2 1 42.59 31.47 74.86 102 2' 3 41.67 ?;'32) 58.33 (#1';;) 188.88 l02 1 2 58.14 ' 33.53 ' 91.67 103 1 1 6.25 62.50 68.75 103 1' 3 12.50 (12°g?) 81.26 ($2'43) 93.76 183 3 2 34.62 ° 57.70 ' 92.32 104 4 1 51.66 28.33 79.99 184 4' 3 50.59 (#3';;) 49.42 (33';g) 188.81 104 2 2 27.27 ' 72.72 ' 99.99 105 3 1 53.45 29.31 82.76 105 3' 3 54.83 ?§'g§) 37.28 ffi'gg) 92.11 .105 4 2 59.85 ' 37.13 ' 96.98 106 4 1 35.26 60.89 - 96.15 l06 4' 3 48.96 fg'gg) 41.58 (§g'}§) 8.46 186 2 2 31.82 ' 68.99 ' 188.81 107 1 1 51.95 48.03 0 99.98 187 1' 3 44.45 ?Z'8§) 55.56 ?g'55) 188.81 187 3 2 45.63 ' 44.84 - 90.57 188 2 1 31.25 68.42 9l.67 . 35.35 . 58.41 . :6 l08 2 3 48.14 (11.3,) 48.15 (9.4:) 99':: 108 l 2 26.66 66.66 93.3- -....H-._.——..- -—-—.—--——-_._._.—__._-.——_... -0- ...—~-—..........-.. ._-- - - . _ _.- - . . - - . 8Based on the average of percentage of number of times each relationship appeared for cues. (See Table 20 and Tables H18, Hl9, and H20 in Appendix H.) bl', 2', etc. = alternate forms of Case l, Case 2, etc. 107 of number of times relationships for hypothesis-directed and cue- directed inquiry responses appeared for cues. (See Row Total column for Table 20.) Table 22.--Means and standard deviations across all subjects and cases for average of percentage of number of times rela- tionships for hypothesis-directed and cue-directed inquiry responses appeared for cues. Case Hypothezgagairected Cue-Directed Number y an1ny L4. 5.9 .11 59. l (gf6) 33.33 21.44 57.92 16.37 2 (flf5) 36.99 8.25 56.68 15.06 3 (366) 52.50 11.41 38.33 11.33 4 (géG) 53.83 13.72 40.10 13.78 Mean total = 44.16 Mean total = 48.26 §DO= 10.52 §D.= 10.48 Data in Table 22 suggest that when subject responses using relationships characterizing hypothesis-directed inquiry for cues were compared by case to subject responses using relationships char- acterizing cue-directed inquiry for cues, subjects diagnosing Cases 1 and 2, on the average, indicated by their responses that they used more cue-directed inquiry than hypothesis-directed inquiry. This would imply that they collected cues based on a fixed strategy. Conversely, the subjects for Cases 3 and 4, on the average, indicated by their responses that they used hypothesis-directed inquiry, imply— ing that they collected cues based on a variable strategy; i.e., their 108 hypotheses (hunches) determined the information they collected. (See Appendix H.) Table 23 shows the average of percentage of number of times each relationship from the H/OCL appeared for diagnostic statements made for Case 1. A look at the table indicates that the most fre- quently used relationship for statements on Case 1 was relationship number 5, Confirm Hunch. As mentioned earlier, by combining relationships into cate- gories of hypothesis-directed and cue-directed inquiry, it can more clearly be shown how the individual subjects responded to questions fonnulating the two types of inquiry. Table 24 is a summary for average of percentage of number of times relationships appeared for diagnostic statements using hypothesis-directed and cue-directed inquiry responses. The same relationship responses (numbers 1, 5, 6, 9, and 10) that were used to characterize hypothesis-directed inquiry appearing for cues also apply to diagnostic statements. Likewise, the same relationship responses (numbers 2, 3, 4, 7, 8, ll, 12, 13, and 14) that characterized cue-directed strategy appearing for cues apply to diagnostic statements. Means and standard deviations were computed using the same procedures appearing for the previously described cue data. The tables for statement computations include Table 23 and Tables H21, H22, and H23 in Appendix H. As shown by data in Table 23, inconsistencies in subject-reported strategy become apparent, e.g., Subjects 103 and 108. .co. cu e do mpmom co vmmmmm . (' l‘."'.l|ll.|.l'lllllluvll)l'l'l 1'1- ‘Il. '40 Iv-1 1 l ’7.)|.. r>i||‘.'||‘1II‘.I|‘ ' It'll): 0|.I. all! I.-.‘- (Ill. ll -II’I . '..'.lu‘.¢ ‘1- l I )-1- 41.111 1 A. 1 -1 -IOII. '1'"- (IIIIIIIIIII 1(39 I I O. | . . 0 H ".h U1”.‘u11fll,fl 0 com. mmo. mm.m o.o o.o o.o o.o wo.~ mN._ sacs: naumaoasm w “mu >._6=m= AN.MV¢_ $4.. mNK. mm.q 0.0 o.c o.o mm.q o.o 0.0 sacs: aaetwccou a “mu »._6=m: Am+mvm_ “0.. on. mo.~ 0.0 o.o o.o o.o o.o mo.~ goes: oaumaomam a .oc=_ A~+NVN_ Fo._ mm._ ..w o.o o.o mw.m ~_.N mo.~ o.o amet_ceou a .6c=_ 666:8: Am+mv_. 0.0 o.o o.o 0.9 o.o o.o o.o 0.0 o.o caecwceaamwo a 26:5: Ao+_vo_ om.m om.v No.NN No.6. oo.o o.o mm.q o.o o.o vasc_c=oo a goes: Am+_va NN.m mm.~ oo.q_ o.o oo.e_ o.o o.o o.o o.o cacao m mo.m_ mm._m mm.- __.__ oo.o~ m~.o o.o oa.me mo.mm coca: awmmasm A m~.e em.m oo.o~ 0.0 co.~ em.~_ mm.o o.o o.o gags: Ecpccoamwo o mm.m_ mm.om om._mp oe.mm oo.o mm.mm m_.mm o.o ¢~.m_ goes: sc_c=ou m o.o 0.0 o.o o.o o.o o.o o.o o.o o.o cagao a co.o_ mm._m No.2mp .N.mm oo.e~ co.mm m_.mm mN.w_ ek.eq .oc=~ yaw »_.~=m= m 60.6 .m.o oo.mm om._ 30.0. No.m o.o om.~_ o.o .oc=~ 666:6: “as: N No.m mm.o m_.mm mm.¢ oo.~ we.“ mm.e mc.m_ o.o sacs: P am 2 .mwmc ~o_mowwwasm mo_Namwwasm No_mumwwasm Nop_6”mwa=m mo_mouwwn=m mo__u”wwgsm awgmcowampmm thMmz “I. ..|.h ...Il- ""1”".hbhflihwnud [H.i'. H“""."“ N EIWhulN. I o._ ammo--ww:uamwmwwzmwawqcaoflv Loy coccmqam gwcmcowuepmc comm mme_u 60 Logan: $0 mmmucmucma mo mmmgm><--.mm m_nmh 110 Table 24.--Average of percentage of number of times relationships appeared for diagnostic statements using hypothesis-directed and cue—directed inquiry responses across all'subjects and all cases.6 -..- *— .—-.— .—_— Average Average . Hypothesis- for 3 Cue- for 3 i in gzggggt Nfigggr ngger Directed Sessions Directed Sessions Relation- Inqu1ry 8 Inqu1ry & sh1p (£9) (.52) 101 3 1 78.57 21.43 100.08 101 3' 3 76.37 (2°38) 23.63 fi'gg) 100.08 101 4 2 66.03 ' 30.13 ' 96.16 102 2 1 71.16 28.51 96.67 102 2' 3 52.10 (??'§§) 47.90 (#1'3g) 100.00 102 1 2 51.59 ' 48.11 ° 99.70 103 1 1 15.79 84.21 100 00 103 1' 3 15.63 (ig'gg) 84.37 (23°33) 100 00 103 3 2 47.41 ' 49.01 ° 96.42 104 4 1 44.93 55.07 100 00 104 4' 3 34.35 (ig'g?) 63.21 (?g'9?) 97.56 104 2 2 20.00 ' 80.00 ' 100 00 105 3 1 73.44 26.57 99.67 105 3' 3 52.35 (?g 80‘ 28.90 (ff'}§) 81.25 105 4 2 53.13 ° ’ 46.88 ' 100.01 106 4 1 31.31 68.69 100.00 106 4' 3 43.12 ?§'§g) 56.87 f; 3;) 99.99 106 2 2 29.55 ' 70.45 ' 100 00 107 1 1 54.35 45.65 100 00 107 1' 3 54.81 E; g?) 45.19 ??'?g) 100 00 107 3 2 46.77 ' 43.55 ‘ 90.32 108 2 1 27.68 72.32 100.00 108 2' 3 47.52 (fg'gg) 52.48 ($g'gg) 100 00 108 1 2 16.00 ' 80 00 ' 96.00 _H.-._.-__-_—________.-_-_- .. --.. -..-_. ...-.-_- -.- - - - -.- . - ..._- - . .. - _ .—. -... aBased on the average of percentage of number of times each relationship appeared for statements. (See Table 23 and Tables H21, H22, and H23 in Appendix H.) b1', 2', etc. = alternate forms of Case 1, Case 2, etc. 111 Case differences for the two types of strategies can be shown by finding the overall average by case on responses for hypothesis- directed and cue-directed inquiry. Table 25 shows means and standard deviations across all subjects and cases for average of percentage of number of times relationships for hypothesis-directed and cue—directed inquiry responses appeared for diagnostic statements. column for Table 23.) (See Row Total Table 25.--Means and standard deviations across all subjects and cases for average of percentage of number of times relationships for hypothesis-directed and cue-directed inquiry responses appeared for diagnostic statements. Hypothesis-Directed Cue-Directed Nfimggr Inquiry Inquiry 14. 59. 14. 99 l (né6) 34.70 20.72 64.59 20.10 2 (gf6) 41.34 19.10 58.61 19.20 3 (flf5) 63.43 14.70 32.18 11.34 4 (_é6) 45.48 12.74 53.48 13.63 Mean total = 46.25 Mean total = 52.22 §Q_= 12.32 §D_= 14.11 Data in the table suggest that when subject responses using relationships characterizing hypothesis-directed inquiry for diagnos- tic statements are compared by case to subject responses using rela- tionships characterizing cue-directed inquiry for diagnostic statements, subjects diagnosing Case 3, the most difficult case, on the average indicated by their responses that they used hypothesis-directed 112 inquiry to a greater extent than did subjects for the other cases. It also appears that Cases 1, 2, and 4 showed more responses indi- cating use of cue-directed than hypothesis-directed inquiry. Over- all, the subjects indicated more use of information-gathering strategy, or cue-directed inquiry, than hypothesis-directed inquiry in making diagnostic statements about the cases. Process Measures: Summary The process measures deal with clinical problem-solving strategy, or the manner in which a clinician arrives at a diagnosis. The behavioral dynamics of the clinical interaction, which includes the clinician's problem-solving strategy, are measured statistically in terms of the dependent, time-related variables, i.e., length of time for case interaction, number of cues collected, and number of diagnostic statements made. Results of the formal process data analysis are presented according to three major types of descriptive statistics: (1) basic process statistics, which summarize data relevant to times cues and statements first appeared in the clinical interaction for individual and combinations of sessions; (2) process-agreement statistics, which include correlation and partitioned Phi coefficients, and are intended to reflect agreement among clinicians on the time/order in which cues are collected and diagnostic statements are made; and (3) cue—to- statement relationship statistics, which are intended to indicate the degree of relationship between cues and statements of hypotheses as suggested by the H/OCL (debriefing) data. 113 The process-measures data seem to suggest the following: 1. Case 1, the second most difficult case, had the lowest average total time of the clinical sessions-~an average of approxi- mately 27.minutes total elapsed time. 2. Case 1 had, on the average, the fewest cues collected; average cues totaled 15. 3. Case 1 had, on the average, the fewest diagnostic state- ments made; average statements totaled 22. 4. Case 1 had, on the average, the lowest average cue frac- tional time; more cues were collected in the fjr§t_ha1f than in the second half of a session. The mean of the average cue time for Case 1 was M.= .43, SQ,= .05. 5. Case 1 had, on the average, the lowest average statement fractional time. The mean of the average statement time for Case 1 was M_= .39, §Q_= .07. 6. Case 2, the easiest of the four cases, had, on the average, more cues collected in the first half than in the second half of a session. The mean average cue time for Case 2 was M_= .49, §D_= .05. 7. Case 3, the most difficult case, and Case 4, the second easiest case, had, on the average, more cues collected in the seggnd half than in the first half of the session. The mean average cue time for Case 3 was M_= .55, §Q_= .09; and for Case 4 M,= .62, §Q.= .05. 8. The highest mean correlation for cue times occurred for Case 4, the second easiest case, at Mr; .45, _S__D_= .13, with the other three cases having somewhat lower but more consistent mean correla- tions. 114 9. The overall mean correlations for statement time were fairly consistent, with the highest value for Case 4 at j_r_= .30, §Q_= .07. 10. The average individual clinician compared with a group of clinicians diagnosing the same case showed greater magnitude in relationship for cue time (M r_= .36, §Q_= .07) than for diagnostic statement time (Q r_= .24, §D_= .04). 11. The agreement among clinicians on the time/order in which diagnostic statements were made by one clinician when compared to other clinicians on the same case (inter/intraclinician agreement), as reflected by the partitioned Phi coefficients for statements, indicated very low, mostly negative, or no agreement; the range of the means was -.003 (Case 4) to .03 (Case 1). 12. In general, the subjects diagnosing the four cases indicated by their responses concerning why_they asked for certain information that they used some combination of hypothesis-directed and cue-directed inquiry when collecting information on a reading case. The mean total for the average of percentage of number of times relationships appeared for cues was M = 44.16, §D_= 10.52 for hypothesis-directed inquiry and M_= 48.26, §D_= 10.48 for cue- directed inquiry. (See Table 22.) 13. Subject 101, Run 2, Case 4, with the highest average of percentage of number of times relationships appeared for cues (76.67 percent), indicated that he used mostly a hypothesis-directed approach or strategy in making decisions about that case. (See Table 21.) 115 14. In general, the clinicians in this study indicated by their reSponses to the question of why_they asked for certain infor- mation on a reading case that they used primarily cue-directed inquiry in making diagnostic statements about the cases. The mean total for average of percentage of number of times relationships appeared for diagnostic statements was M.= 46.25, §D_= 12.32 for hypothesis- directed inquiry and M_= 52.22, §0'= 14.11 for cue-directed inquiry. (See Table 25.) 15. Subject 101, Run 1, Case 3, with the highest average of percentage of number of times relationships appeared for statements (78.6 percent), indicated that he used mostly a hypothesis-directed strategy in diagnosing that case. (See Table 24.) The Informal Product/Process Measures It may be recalled that the second principle of the inquiry theory describes those factors that govern the clinician's behavior during the clinical encounter. This second principle states that those events or behaviors that occur in the clinical interaction are determined probabilistically by the case and the clinician's memory and strategy. The purpose of the informal product/process measurement was to gain further insights into the clinical behavior of reading clinicians, the clinical behavior being defined in terms of the data base repre- sented in clinical memory and the diagnostic routines represented in clinical strategy. Additionally, it was important to attempt to determine the source (schema) that allowed the clinician's data base 116 (e.g., cue collection) to be translated into an action (e.g., hypothe- sis generation). It was recognized that extended questioning in the debrief- ing might overburden the clinician subjects, but at the same time it was desirable to investigate their clinical/diagnostic behavior through measures of informal self-reporting. Therefore, the informal product/process portion of the third session was limited to six ques- tions. One of the questions was designed to elicit the clinicians' ideas about the content of a diagnosis or the data base (memory). Another question was designed to elicit information on the way in which the diagnosis was conducted, that is, the routine used (strategy). Four questions were designed to generate responses about the source of the data base or that which allows the clinician to generate a diag- nosis. The six questions, in the form used by the examiner when listening to tapes of the clinicians' responses, appear in Appendix C. These questions were generated from a diagnostic model of reading and learning (Sherman et al., 1978). The informal questioning followed the last session for each clinician in the study and was used to guide his thinking about the third case with which he worked. The paraphrased responses of'representativeclinicians (1048.106)to the questions in the informal portion of the session appear in Tables 26-28. The complete, unedited dialogue for Subject 104 appears in Appendix D. Also in Appendix D are the paraphrased responses of the other clinicians in the study. In looking at the data base (memory) reported by the clini- cian, the interest was in determining what were the descriptors of 117 the data base (i.e., What information should be included in a "good" diagnosis?). In the diagnostic routine, the concern was with the task of information gathering or h9w_the data base (memory) was translated into action (strategy) (i.e., How do you usually go about a diag- nosis?). The third important aspect of the informal product/process measures involved hypothesis generation or the source (schema) of the data base that allowed memory to be translated into strategy (i.e., How did you know . . . that the reading fluency was low?). Table 26 shows one sample of responses to informal questions regarding memory and strategy. The sample reflects one kind of material that clinicians said was contained in the data base (Ques- tion 1) and the diagnostic routine. Sample 1, Question 1, appears to reflect a straightforward type of data collection, saying, in effect, "I do this in order to find out this and then I do this"-- a cue-directed process. Table 27 shows another sample of responses to the same questions as Sample 1. However, the second sample is more reflective of data collection for the purpose of making a com- parison: "Depending on what I found I would do this or this"--a hypothesis-directed process. Interestingly, some of the subjects answered the question about what_should be included in a good diagnosis in behavioral or process terms of h9w_rather than in content or product terms. Others used a combination of behavioral and content- type responses. This latter observation also held true for the second question regarding the "how" of diagnosis. It appears that, based 118 Table 26.--Sample 1 of responses to informal questions relating to memory (1) and strategy (2)--Subject 104. What information should be included How do y9g_go about a in a "good" diagnosis? diagnosis? (1) (2) 1. Unusual factors such as: l. Informal tests Physical (sight & hearing) Emotional (child abuse) 2. Sight words Environmental (abnormal) 3. Phonics 2. What kid knows about words 4. Structural analysis 3. Informal oral reading, usually child's choice of material, to get hunch of "where he is" 0‘1 . Application 6. Comprehension (limited 4. Slosson Oral Reading Test to assessment) get grade score 5. Ekwall Inventory for miscues on words in isolation to determine understanding of phonics system 6. More formal phonics test to see if he knows initial and final consonants, blends, etc. 7. Instant sight-word recog- nition 8. Oral reading to determine if word-by-word or phrase reader (fluency) 9. Listening comprehension 119 Table 27.--Sample 2 of responses to informal questions relating to memory (1) and strategy (2)--Subject 106. What infbrmation should be included in a "good" diagnosis? 1 How do ygg_go about a diagnosis? 2 l. The Wechsler or some kind of an intel- lectual assessment of their potential strengths and weaknesses. 2. Comparison of oral and silent reading to find out what kinds of decoding problems the child is having as well as the oral and silent comprehension. 3. Listening comprehension to compare with the WISC. 4. Word recognition test to compare with kinds of errors made on the Durrell oral reading. 5. Word analysis to compare with a Dolch and to get an indication as to whether or not they are sight-word readers or have analysis or decoding skills. 6. Definitely a visual and hearing screening.(I give these early to eliminate physical problems.) 7. Family and school history to under- stand whole child and find out if there are physical problems relating to reading difficulty or if familiar kinds of problems affecting attitude and motivation; also to check exces- sive absences. 8. Then depending on what I found, I would go to more specific things like checking auditory blending and dis- crimination and check digit span on WISC and if he didn't get it, I would give auditory memory and visual mem- ory from Durrell. 9. Child's comments in an interview are also important to find out how he feels about reading and what he thinks his reading problems are. . I would start with family background and school background. . I would talk with the teacher. . I would check the physical for hearing and if a problem I would pursue the auditory. . Then I would take care of other things I men- tioned in a "good" diagnosis. 120 .uw apooo o» 5?; pomoxo u.ooo om Eopmxm oooum lemon: u.:mooe .coooo o ocozuzouocoz Homoo mom 8:3 H zpwcozuo<.11 oucowcmoxm 3.8: v... oz me, u. .cowmcozocosoo mowooumwp cos» Lozo— meow» ozu _oco oco pompwm ”mocoum ._owc logos mowcmamwp moocm zp$_$ ocozocosoo coo poo mowoozo so mocoz zoox p.cmooo .mEop imam z_ooo p.=ou ”omcommom .658 .ll zpwgozu=<.11 oucowgmoxm 38: v. 2.2.5 meccaomwo a Paco oz UH. mo> UH. mzoox :o» oo 3o: «go, zoom=_w mcmoooc mo: mzoox so» on 3o: mzop oo_mcozoco5oo momoooc mo: .8 .3 .8 an .mmoomw: .uoooomooo _oomm go: “on Fowuwcw mpoo .muwcozo .ooocm mo movocoumcooco omooo zucooz ow m.o; mm moocm -mmooz oomomcou "mmcommmz unmocoom commopm nomcommoz cozpo.kr szuo.11 zawcozuo<.11 zuwcozuo<.kv moowwcmoxm.11 11. 11. mocmwgmoxm.11 .11 11. —oooz oz mo> \ Foooz oz mm> \ wzocz oo» oo 3oz .om «Zop m-_zm mwmzpocouocoz mom: 0mm wzoox oo» oo 3o: .o_ ozop oowpwomouoc ocoz acoumcw no: .o_ .o omou .m com .oop poonoomuumowoooc mo mpooomo ovowooom mcwogomoc mcowpmooo Loom ow momcoomoc wo oposomuu.mm m—ooh 121 on the informal data, clinicians in this study in general did not share a clinical memory, in terms of a data base, i.e., information that should be included in a "good" diagnosis. Likewise, it appears that these same clinicians did not share a clinical strategy in terms of a diagnostic routine, i.e., how to go about a diagnosis. Table 29 is a tabulation of responses by eight clinicians to specific questions regarding reading skills for four cases, two clinicians per case. The results suggest that all subjects for all cases employed a schema in making determinations about specific skill areas of reading. This schema appears to reflect some theoretical process based on cause/effect relationships; authority, based on test data or teacher report, etc.; and other, when it was not based on process, experience, or authority. No responses were coded "experience." It appears that although any one or all of the clinicians may as a normal practice use theoretic process as a source of their data base or memory, the subjects' responses did not always reflect that assumption. Therefore, it could be that some of the clinicians in this study did not consistently employ a standard or model for diagnosis, or it could be that the questions themselves failed to elicit the kinds of responses that would reflect the use of a formal theoretic process model (i.e., cause/effect relationship). Also, it can be seen in Table 29 that clinicians diagnosing the same case were not always in complete agreement with each other 122 =.z_ooooco= mo; cowomooo ou mmooomoz8 Zoo: \. EB: \. bfio \. b .2652 x 2: e poooz \ poooz \ \ Lofio \ zu_cozu:< \ cop o Foooz \ zumgozoo< \ zuwcozuoz xx zuwcozo=< 5\ mo_ m _oooz \ Foooz \ _oooz \ _oooz \ _o_ m Lofio \ _oooz \. \ L23o \ \ zuwcozuo< \ wo_ N cozuo \. \ Foooz \. \ cozuo \ _mooz \ mop m cozuo \ Poooz \ Lozpo \ zowcozuo< \ Bop _ Foooz \ zumcozuo< \ zuwcozu=< \ zuwcozuo< \ mo_ P olz 81%. mm W16» mm m.» mm mm . ~3o_ . wzop co_m . ~3o_ m_p_zm . ~zo— oo_u_o . . ascozzwoz zooms—w ozwcxzmoz -oozocosoo ozmcxzwox mmmxpoco ozoczzmoz -moooc woo: .non ommz owo : mcwoooc mo: o.o : movoooc mo: o.o : ocoz ocoz ovo : acoumop mo: .o m u .oo .oo .om .om .om .om .op .o— .momoo goow cow mowoooc 6o mooco coo; mcwocomoc moomumooo owwvumom op mcowowc_po uzmwo zo momcoomoc wo mu—omoz--.mm «Pooh 123 even when asked about the same specific reading skills. The indi- vidual responses of the clinicians shed light on why it might appear that reading clinicians disagree'ifasked to answer "Yes" or "No" to diagnostic questions. For example, Subject 101's response to the question, "Was instant word recognition low?" was: "Not that low. He's in 7th grade and his score was 6.8 on instant word recognition. But the school district has the reputation for having children doing well and so many of his peers are probably above him. He's conceiv- ing himself as being poor, so in that case he's probably low." (See TableeDll,Appendix D.) This response might be marked "No" for low instant word recognition and left untreated. However, considering environmental factors, it might be wise to supplement what could be a strength for that student. Informal Product/Process Measures: Summary The informal product/process measures were designed to reflect in a less formal way than was reflected in the statistical measures the second principle of the Inquiry Theory: the probabilistic deter- mination of behavioral interaction between a case and the clinician, represented by the clinician's memory and strategy. Six questions based on Sherman's Model of Reading and Learning (1978) were asked orally to the eight clinicians in this study following the third or last clinic session for each clinician. The questions were designed to (l) elicit information regarding the clinician's conceptions of the content or data base of a "good" diagnosis; (2) determine how the data base was translated into action, i.e., the diagnostic 124 routine; and (3) identify the source or schema that allowed the data base (memory) to be translated into action (strategy). In general, the clinicians' responses to the informal product/ process questions seemed to indicate that the "expert" clinicians in this study did not share a clinical memory in terms of a data base, i.e., information that should be included in a "good" diagnosis. Likewise, the responses of those same clinicians seemed to indicate that they did not share a clinical strategy in terms of a diagnostic routine or how they went about a diagnosis. Additionally, results of the clinicians' responses to informal questions suggested that all subjects employed a schema in making determinations about four areas of reading diagnosis: instant word recognition, word analysis, reading comprehension, and reading fluency. This schema appeared to reflect theoretical process models based on cause/effect relation- ships; authority models based on specific test data, teacher report, etc.; and some other model, reflecting a guess, an intuition, or some combination of nebulous factors. The informal data do seem, in general, to support the statis- tical findings and additionally provide insights into the inter- relationships among diagnostic reading factors, which make the application of statistical analysis to clinical diagnosis of read- ing a very difficult though seemingly justifiable pursuit. 125 21mm. The analysis of the data was presented in three parts: (1) formal product measures, (2) formal process measures, and (3) informal product/process measures. The major finding indicated by the product or outcome measures was that there was little agreement among experienced, highly trained reading clinicians, using simulated reading cases, on the data they collected and the diagnostic statements they made for specific read- ing problems. This finding was supported by an analysis of the results of several statistical measures, including proportional agree- ment, commonality scores, inter/intra and intraclinician agreement Phi coefficient, and the Porter statistic. The major finding suggested by the process measures was that the experienced, highly trained reading clinicians, using simulated reading cases, lacked an extensive and systematic method of collect- ing data and evaluating information about specific reading problems. This finding was supported by an analysis of the results of several statistical measures, including basic statistics (time of cue collec- tion, number of cues collected and diagnostic statements made), correlation, partitioned Phi coefficient, and cue-to-statement rela- tionship statistics. The major conclusions suggested by the informal product/ process data were that (1) although experienced, highly trained read- ing clinicians, using simulated reading cases, may as a normal practice use theoretic process as a source of their data base in clinical diag- nosis, they do not consistently employ a standard or model for 126 diagnosis and (2) although experienced, highly trained reading clini- cians, using simulated reading cases, may have their own individual way of conducting a diagnosis, these reading clinicians do ppt_use a common systematic or comprehensive diagnostic routine or clinical strategy when diagnosing specific reading problems. CHAPTER V SUMMARY, DISCUSSION, CONCLUSIONS, AND RECOMMENDATIONS Summary Introduction The response of reading researchers to the challenge of more effective and efficient reading diagnosis and treatment has shown little effort to investigate the way in which reading clinicians think about their students' reading problems (Shulman & Elstein, 1975). If one assumes that correct diagnosis is a prerequisite of effective remediation, then it would seem to follow that one approach to improving remedial practices would be to improve the diagnostic skills and training of reading clinicians (Hoffmeyer & Bader, 1978). Recent studies in medicine have demonstrated how clinicians might be taught to seek answers to pertinent questions and thereby improve their clinical skills and subsequently their diagnostic and thera- peutic competence (Barrows et al., 1976; Elstein, Shulman, Sprafkaenzal, 1978; Vinsonhaler, Wagner, & Elstein, 1977). Similar research is needed in the field of reading. The need is threefold: First, there is practical value in having a well-understood theoretic base, such as the one developed in medicine, for research on clinical problem solving in reading; second, there is efficacy in using scientifically based methods to examine clinical problem solving in reading; and 127 128 third, there is augmentative value when research studies share a common methodology (Vinsonhaler, 1979a). To move in the direction of improving problem solving among reading clinicians, the first step might be the observation and study of the decision-making and problem-solving behavior of "expert" reading clinicians for the purpose of distinguishing those reasoning skills that characterize them as "experts." Then, as in medical studies, these behaviors might be used as models in teaching effective and efficient clinical diagnosis in reading. The Problem The general purpose of this study was to use the Inquiry Theory of Clinical Problem Solving as a theoretic base in determining ppw_eight experienced reading clinicians diagnosed specific reading problem cases (process), and whgt information was used in reaching a diagnosis (product or outcome). Specifically, the purpose of this study was formally to test three basic components of the Inquiry Theory. These components were (1) the agreement of reading clinicians in collecting data (cues) on a specific case in order to diagnose a reading problem, (2) the agreement of these same clinicians in making diagnostic statements, and (3) the reading clinicians' use of hypothe- ses to direct their diagnostic clinical inquiry. Additional informal assessment was made regarding (1) the clinicians' agreement on the information that should be included in a "good" diagnosis, (2) the clinicians' agreement on how to go about a diagnosis, and (3) the 129 schema employed by the clinicians in making diagnostic determinations about specific skill areas of reading for a particular case. Review of the Literature The Inquiry Theory was developed by a team of researchers at Michigan State University to provide a formal theoretic structure that would integrate and account for the numerous concepts and empiri- cal findings on clinical problem solving. The theory postulates that the clinical encounter involves a case and a clinician, and is char- acterized by the interaction that occurs between the case and the clinician's memory (problem, cue, cue value, diagnosis, treatment, and the relations between them) and strategy (information-gathering and information-processing tasks that translate memory into action). Although the study of how information is processed has been confused and complicated by conceptually unclear terminology, research procedures in problem solving have shared a common relationship in tasks relating to realistic environments. Such procedures include, among others, (1) total-task or process-tracing studies, which are concerned with the sequential character of gathering information in order to make a decision or judgment; (2) in-basket studies, in which the decision maker receives information "input" for making a decision; and (3) tab-item methods, which provide objective, reliable data through predesignation or choices of action, such as troubleshooting electronic equipment performance failure. 0f major import to inves- tigators using these information approaches is the consideration of human behavior in actual task environments and the observation of the 130 process of thinking and judgment in these environments. In the inves- tigation of educational problems, studies dealing with thinking, human judgment, and decision making are lacking. A major area in education that lends itself to the study of thinking, human judgment, and decision making is reading (Shulman & Elstein, 1975). However, as in other areas involving the processing of information, the field of reading likewise has its inherent problems. Some of the concerns related to problem solving in the diagnostic process of reading include (1) the etiological and therapeutic aspects and procedural steps in diagnosis; (2) the causal debate, singular versus pluralis- tic and etiological versus therapeutic; and (3) the lack of stan- dardized terms. These and other concerns are important because disagreements over such issues have complicated communication both within the reading profession and between fields relating to reading, such as learning disability and psychology. One attempt to explicate clinical diagnosis in medicine and subsequently in reading has been the research relating to the Inquiry Theory of Clinical Problem Solving. A number of interrelated studies have been undertaken in the research on clinical diagnosis in reading. Results of these studies suggest the need and provide the impetus for the improvement of instruction, performance, and evaluation of reading clinicians. Design and Procedures The major objective of this study was to determine the nature and extent of agreement of eight experienced reading clinicians in 131 terms of (l) the data they collected on specific cases in reading in order to make diagnostic decisions about those cases, (2) the diag- nostic statements made for specific cases, and (3) the use of hypothe- sis generation in order to reach a diagnosis on specific reading problem cases. The eight subjects in this research were chosen by univer- sity faculty recommendation from a list of reading clinicians who had taught the summer institute courses in reading diagnosis and remediation offered by Michigan State University. From among those recommended, the eight highly trained and experienced clinicians who volunteered included two male and six female clinicians. All of the clinicians were paid at a professional rate for each of the three approximately three-hour sessions (spaced no less than one week nor more than four weeks apart) in which they agreed to participate. The clinician subjects, taken one subject at a time, were asked to obtain materials from a SIMCASE (a SIMulated CASE or set of data representing a child with a reading problem). SIMCASE materials were contained in a box, and the examiner provided the information by handing the material to the subject as the subject requested it. The clinician did not know specifically what material was in the SIMCASE; however, he was told that the information could be provided in five fonns as test scores, examiner's comments, test booklets, audio recordings, and test directions. Examples were shown for each. The clinician subject was given 45 minutes in which to collect and study the SIMCASE materials. The subject was free to take notes and to retain all items of information requested. 132 Following the SIMCASE interaction, the clinician subject was given 25 minutes in which to write a diagnosis and an additional 25 minutes in which to write a remediation. After a short break, the subject was asked to (1) transfer his written diagnosis to the Reading Diagnostic Check List (RDCL) and (2) indicate if each of the reading factor statements in his written diagnosis was a strength, a weakness, or just an observation. Next the subject was asked to com- plete the Hypothesis/Observation Check List (H/OCL) by responding to the questions of why he had asked for each piece of information (cue) and what the infonnation had told him. The subject was asked to write a brief explanation for each response to the wpgt_question. The above procedures were followed for each of the three clinical sessions. Additional procedures for the third or last ses- sion only included questions designed to (l) elicit information regarding the clinician's conception of the content or data base of a "good" diagnosis, (2) determine the clinician's diagnostic routine, and (3) determine the source or schema that allowed the data base (memory) to be translated into action (strategy); i.e., there were questions dealing with four areas of reading and how they could be defined for the third or last case each subject diagnosed. The four areas were (1) instant word recognition, (2) word analysis, (3) reading fluency, and (4) reading comprehension (Sherman, 1979). All stimulus materials used in the study, including SIMCASEs and equivalent forms, were subjected to counterbalancing to minimize systematic effects. Subjects were randomly assigned to the case order. 133 Subject reliability on the use of the RDCL, a previously unused instrument, was measured using a test—retest procedure. The Pearson product-moment correlation coefficient (5) was calculated as the measure of relationship. The value of 3 ranged from .15 to .69. The Porter statistic, 31%;6-(see Wilcox, 1977), an index of the pro- portion of agreement that excludes clinician agreement not to select a cue or make a diagnostic statement, was also calculated on the same data. The Porter values ranged from .10 to .54. The RDCL of reading factors was designed to objectify the data collection and analysis. Clinicians were asked to convert their written diagnoses to the check list, indicating strengths, weaknesses, or observations for diagnostic statements on the list corresponding to their written statements. Analysis Measures and Results Because of the limited sample size used in this study and because the methodology and statistical measures are still open to Speculation, all findings and conclusions offered in this disserta- tion must be considered tentative and should not be generalized beyond this study, pending further evidence. The analysis of the data was presented in three parts: (1) formal product measures (including proportional agreement, com- monality scores,inter/intra-and intraclinician agreement Phi corre- lation, and the Porter statistic), (2) formal process measures (using correlation, partitioned Phi coefficients, and cue-to-statement 134 relationship statistics, and (3) informal product/process measures (using Sherman's Model of Reading and Learning to Read). The major findings related to product or outcome of the clinical interaction between a reading clinician and a case (SIMCASE) include the following: 1. Twenty-three percent of the same cues from the total cues available on the four cases were collected by three or more (P.A. = .50-1.0) of the six clinicians diagnosing each case. 2. Fifty-two percent of the total cues available were col- lected across all four cases. 3. One percent of the same diagnostic statements in the check list were checked by three or more (P.A. = .50-1.0) of the six clinicians diagnosing each case. 4. Twenty percent of the diagnostic statements in the check list were checked for the four cases. 5. The commonality scores indicated that the clinicians for this study were consistent, 75 percent, in their agreement on cues collected across all cases. 6. The commonality scores also indicated that the subjects for this study agreed, on the average, 55 percent of the time on the diagnostic statements made across all cases and all subjects. 7. The inter/intraclinician agreement Phi indicated low agreement, with little variability between cases on cues collected and diagnostic statements made. 8. The intraclinician agreement Phi for cues and diagnostic statements was not consistent, and agreement was low. 135 9. The intraclinician agreement Phi on cues collected was higher than the inter/intraclinician agreement Phi on cues collected for three of the four cases. 10. The intraclinician agreement Phi on diagnostic statements made was higher than the inter/intraclinician agreement Phi on diag- nostic statements made for all four cases. 11. With two exceptions, the Porter Index yielded slightly lower scores than did the Phi for both intra- and inter/intraclinician agreement on cues and diagnostic statements. The exceptions were the lower inter/intra Phi on cues for Case 2 and the lower intra Phi on cues for Case 1. 12. The clinicians for this study were lower in agreement (both Phi and Porter) on diagnostic statements made than on cues collected. The major findings related to the process or the way in which the clinician diagnosed or behaved in the clinical interaction include the following. 1. The highest mean correlation for cue times occurred for Case 4, with the other three cases having somewhat lower but more consistent mean correlations. 2. The overall mean correlations for diagnostic statement time were fairly consistent, with the highest value for Case 4. 3. The average individual clinician compared with a group of clinicians diagnosing the same case showed greater magnitude in relationship for cue time than for diagnostic statement time. 136 4. The agreement among clinicians on the time/order in which diagnostic statements were made, as reflected by partitioned Phi coef- ficients, indicated very low, mostly negative, or no agreement. 5. The subjects diagnosing the four cases indicated that they used slightly more cue-directed inquiry than hypothesis- directed inquiry in data (cue) collection. 6. The subjects diagnosing the four cases indicated that they used more cue-directed inquiry than hypothesis-directed inquiry in making diagnostic statements about the cases. 7. The subjects diagnosing the four cases indicated greater use of cue-directed inquiry for diagnostic statements than for cues. The informal product/process measures were designed to reflect, in a less formal way than in the more formal statistical measures, ,the second principle of the Inquiry Theory. That principle states that the behavioral interaction that occurs between a case and a clinician is determined in some probabilistic manner by the clini- cian's memory and strategy. To explore that principle informally, six questions based on Sherman's Model of Reading and Learning to Read (1978) were asked orally to the eight clinicians following the third or last clinic session for each clinician. The purpose of these questions was (1) to gain insight into the clinical diagnostic behavior of reading clinicians being defined in terms of (a) the data base represented in clinical memory and (b) the diagnostic routines represented in clinical strategy, and (2) to attempt to determine the source or schema that allowed the clinician's data 137 base or cue collection to be translated into an action, e.g., the clinician's generation of hypotheses regarding the reading problem. Results of responses to the informal product7process questions seemed to indicate the following: 1. In general, the clinicians in this study did not share a clinical memory in terms of what information should be included in a "good" diagnosis. 2. In general, the clinicians in this study did not share a clinical strategy in terms of a diagnostic routine or how they went about a diagnosis. 3. The informal data generally seemed to support the statis- tical findings. Discussion Because of the exploratory nature of this study and the rather complicated interplay of the features of the Inquiry Theory with the actual diagnostic behavior of the clinicians, discussion will attempt to provide further understanding of the results obtained in this research. Study of Product In Chapter I, it was noted that three components comprising the Inquiry Theory directed the present investigation of clini- cal problem solving among reading clinicians. The three components, referred to as corollaries, are (l) the agreement of reading clini- cians in collecting cues in order to reach a diagnosis, (2) the agreement of reading clinicians in making diagnostic statements, and 138 (3) the hypothesis-generation strategy or inquiry of reading clini- cians in making a diagnosis. The first two corollaries enabled the investigation of whgt_highly trained, experienced reading clinicians judged to be important material for diagnosing a reading problem. These results or the product of diagnosis are represented in the Inquiry Theory as clinical memory. The third corollary, dealing with hypothesis generation, provided the means for investigating the diag- nostic routine or hpp_reading clinicians go about a diagnosis. This process of diagnosis is represented in the Inquiry Theory as clinical strategy. Previous research closely related to this study (Vinsonhaler, 1979a) indicatedfjndings that can be further substantiated by the results obtained in the present study. When results of the Observa- tional Studies of 1977 (OS '77) are compared with the results of this study (OS '78.3), several observations can be made regarding reading clinician agreement. (See Table 30.) It appears that, on the average, reading clinicians for both studies showed some degree of commonality in terms of cues collected for a given case (SIMCASE). Likewise, clinicians for both studies showed, on the average, a higher agreement with themselves (intra— clinician agreement) than with each other (inter/intraclinician agree- ment) in terms of cues collected for a given case (SIMCASE). Thus, it appears that the implications for cue agreement as postulated by the cue-agreement corollary (intra higher than inter/intra) do hold for this study as well as for the one to which it is compared. 139 .omoco>o ozu mo oowuow>mo ocoooopm mos“ ".mm «m n Azooum Loo momocmopo popop oo .o Azooum coov oo:_o mo .oz mo. mo. 0.. mo. :owpow>mo ocoooopm up. mm. No. Po. coo: Nooucoov mpmocmowo occh mo. op. mp. mp. oowpow>oo ocooooum mm. mp. mo. mm. coo: szov :owuopoccoo m_mocmowo ocuoH mo. Po. oo. mo. oowuow>oo ogoocoum mo. NP. om. Nm. coo: Acaocoao mwmoomowo ocu:w\couoH oo. m9 m9 N9 cowuow>oo ocooooom F_. N9 1 mm. mp. coo: z_;ao :o_oapaccou mwmonmowo ocucw\goch m9 m9 m9 m9 cowuow>oo ocoocopm mm. mm. m“. «N. coo: ocoom xuwpocoesou sosom m.o2. mo zospm “a. mo zosam m.wu. mo sosom as. mo ocoeaaco< mucosououm owumocmoma moou Low owpmwuopm .mwmoo movoooc oouopoepm Loom mopmo morooum o3» yo zoom op moopopoppo pzmpo an woos mucosououm oppmoomoPo oco oopooF—oo mooo co mopumppopm pooEoocm Fo * Fm>o4 moocw mgooF Fo z < . z .muuonoom :oFuFoFFo mcFoooc co :oFuoELoFcF oooocmxoomuu.F< oFooh 161 :oFuocumFoFEo< a A.o.:a co xgoz FacoFoFooa 6mFoLooo .o.4 o .oo .aaam omF mo z Fm>o4 moocw mgoo> mo * < . z .ooooFuoou--.F< oFoop APPENDIX B CLINICAL SETTING AND INSTRUCTIONS 162 163 Tape Recorder One-Way Glass Exper. Subject Figure Bl.--Diagram of clinical setting. 164 INSTRUCTIONS TO SUBJECTS FOR CLINICAL SESSION This research is being carried out by the Institute for Research on Teaching at Michigan State University. The Institute needs to observe a number of representative reading clinicians in order to develop theories and computer simulations of how clinicians diagnose reading problems. You have been chosen as one of those who will be observed during our study. Because this work will take sev- eral hours, the IRT will pay you as a consultant for the College of Education. A check will be mailed to you after the session. Before I explain the observational session, I should emphasize that all personal information regarding this session will be kept confidential. Your name will not be part of Institute permanent records. Instead, a number will be used. We are required by law to protect your privacy by keeping confidential your name, social security number, etc. Second, I should emphasize that we are not evaluating you in any way. We are merely interested in understanding how you usually go about determining the most probable reading problems of a given client. Now I will explain what we will be doing. The session will be divided into three parts. In Part I you will be asked to interact with materials to analyze a case of reading difficulty. The case materials with which you will be working will be in this file box (indicate box). The case information can be provided in these forms (show list of Forms of Information to subject): (1) test scores (show example), (2) test booklets (show example), and (3) examiner's coments 165 (show example). (4) There are also audio recordings of reading test sessions (show example). For Part II, you will be asked to write a diagnosis for this case and then to write suggestions for appropriate treatment or reme- diation based on your diagnostic findings. In the final part of the session, Part 111, you will be asked to transfer your written diagnosis to a Diagnostic Check List. Also during Part III, you will be asked to attempt to recall what you were thinking about as you worked on the diagnosis. Here is an overview of what we will be doing for the three parts of this session. (Indi- cate Session Overview.) 1 will explain each part in more detail as we come to it. We will begin now with Part 1. Your task is to request the information about a case which will be used to determine the most likely diagnosis and to suggest a general program of remediation. There is no right or wrong amount of information to request for your diagnosis. I would like you to diagnose this case in much the same manner you would use in diagnosing a real case. Assume that you are working with the child in a one-to-one setting. You will be given 45 minutes in which to request information on the case. When you request an item of information, I will give it to you. You may keep all items of information throughout the session. Request items in the order in which you normally collect such information. You may take notes if you wish. (Indicate note pad.) I will begin timing when you make your first request for information and I will let you know when there are 20 minutes of the time remaining. 166 To review, here is a summary of the instructions for Part I. (Give subject Summary of Part I Instructions.): 1. As a consultant, you have been called in to examine and analyze a reading case. 2. Ask for needed information as you would normally collect data. 3. You may take notes if you wish. 4. You will have 45 minutes to reach a decision on the diagnosis. 5. If you reach your diagnosis before the time is called, indicate that to me. Are there any questions? I will have a tape recorder turned on during the session just in case there is something to which I might need to refer at a later time. Now here are some initial items of information on the case .before we begin timing. (Give picture and initial contact material. Start tape of initial contact. Make sure tape is at starting point.) You may begin your request for further information on the case when you are ready. What information would you like first? (Begin timing before and after IC and when subject makes first request for information.) (Record time of cue request under column heading labeled CUE REQUEST TIME on large blue sheet--Hypothesis/Observation Check List.) (Write cue name of cue requested under column heading labeled CUE REQUEST ORDER on large blue sheet--Hypothesis/Observation Check List.) 167 (Twenty-five minutes after first cue request, remind subject he has 20 minutes of time remaining--say, "You have 20 minutes left.") (After subject has completed his work or time is called, give him NCR Blank Form paper, two sheets, and instruct him as follows.) Now for Part II, I would like you to summarize your judgments in written form. Please briefly state your diagnostic opinions on this special carbon paper. (Indicate double sheet of paper on clip- board--DIAGNOSIS.) Write only on the top sheet. Additional paper is available should you need it. In writing your diagnosis, please write as clearly as possible and double space between each line. Be as spe- cific as you can and use complete sentences. In writing your diagnosis, assume that the report will be used by a clinician with training simi- lar to yours who will work one-to-one with the student. You will have 25 minutes to write your diagnosis. Here is a copy of the instruc- tions. (Give subject copy of the Instructions for Writinggthe Diag: ppsjs, Allow time for him to read instructions.) Are there any ques- tions? You may begin. (Begin timing.) (When subject has finished or time is called, place his name, the date, and the SIMCASE name on the Diagnosis write-up.) Now please write your suggestions for remediation here on the special carbon paper. (Indicate Remediation clipboard.) You may use whatever form is convenient for you in writing your remediation. You will have another 25 minutes in which to complete this task. Are there any questions? 168 (When the subject has finished writing the remediation or time is called, place his name, the date, and the SIMCASE name on the remediation write-up.) (Give the subject a lO-minute pggpk,) (During the break the Examiner should put all cues in the order in which they were requested by the subject. Separate the double sheets of carbon paper for the diagnosis write-up and number the pages in correct order. File one complete copy of the diagnosis in the sub- ject's folder and leave the other copy on the table.) We will now begin Part III and the final part of this session. In order to help me objectify data and to make sure that I understand your diagnosis, I would like for you to transfer your writ- ten diagnosis to a check list. This list is made up of possible diag- nostic statements which might apply to students with reading problems. The statements in the check list are restatements of actual statements made by reading diagnosticians that have been put into a standard vocabulary. The statements are divided into several categories such as Developmental/Physical Factors, Perception/Perceptual Motor/Language Factors, Reading-Isolated Instant Word Recognition Factors, etc. These categories listed on a cover sheet and having section tabs should make it easier for you to locate statements for transferring your diagnosis. There are more statements on the check list than you will need and you might not have statements in all of the categories. Please take a few minutes now to look over the check list and then I will give you further instructions. (Give subject sample check list. Allow time for the subject to look over the check list.) 169 Now, to make sure that you understand what you are to do, I will first show you an example of how to transfer a written diagnosis to a sample portion of a check list. Then I will give you a chance to practice a transfer using another example. This is a sample portion of an actual written diagnosis for a reading case. (Show Sample #l--Diagnosis.) It is not meant to be a model or to indicate how a diagnosis should be written. Notice now that circles have been drawn around diagnostic statements. A diagnos- tic statement is the identification of a factor or variable which helps determine the state or condition of a student's reading performance. It may be one sentence or several and it may also just be part of a sentence. In addition, the diagnostic statement may be a strength, a weakness, or just an observation. The circled diagnostic statements or reading factors are numbered, as you can see in the sample. Next you can see how the circled and numbered factors have been transferred to a sample part of the diagnostic check list. (Indicate Check List Sample #1.) The number of the diagnostic statement in the written diagnosis is placed to the left of the matching diagnostic statement on the check list and under the appr0priate column to indicate that the statement or factor represents either a strength, a weakness, or is just an observation. (Indicate number on Sample Check List #1.) Are there any questions? (Collect sample materials.) Here now is another sample from a written diagnosis. (Indicate Practice Sample #l--Diagnosis.) Please locate, circle, and number one diagnostic or reading factor statement in this sample. (Allow time for task.) Here is a copy of the Diagnostic Check List. (Present 170 practice copy of Diagnostic Check List.) Now, using the check list, locate the main category into which you believe the circled statement from the diagnosis will best fit. Use the cover sheet of the check list to help you decide upon and locate the appropriate main category of the check list. (Allow time for task.) Next locate under the main category heading which you have chosen the statement which most nearly matches the statement from the written diagnosis. (Allow time for task.) Put the number of the circled diagnostic statements to the left of the corresponding statement on the check list and under the appropriate column heading--strength, weakness, or observation. (Allow time for task.) Note that space is provided within each main category for "Other Statements." You may copy any statements from the written diagnosis under "Other Statements" if you are able to determine the main category but are unable to find a matching statement on the check list in that same category. However, statements in the written diag- nosis for which you can not determine a category should be copied under the main category Roman Numeral #10 OTHER FACTORS in the check list. (Indicate X. OTHER FACTORS.) Please make as limited use as possible of the statements and the category called Other. Are there any ques- tions? (Collect practice materials.) Here is a review of the instructions. (Give copy to subject.) 1. Locate all of the diagnostic or reading factor statements in your written diagnosis. 2. Circle and number the diagnostic statements. 3. Locate on the cover sheet of the Diagnostic Check List the main category into which your diagnostic statement would most likely fit. 171 4. Find that main category in the check list and locate a statement within it which best matches your circled statement. 5. Decide if your statement indicates a strength, a weakness, ("'lSjUSt an observation which you made; then place the number of your circled statement under one of the column headings. 6. If you recall information that you did not include in your written diagnosis, write that information on the Stimulated Recall sheet, then transfer it to the check list using A, B, C, D, etc. In transferring your diagnosis, it is important that you do not add diagnostic statements from memory to the check list even if some- thing should come to mind. You may, however, write those thoughts on this sheet of paper. (Indicate Stimulated Recall tablet.) Here is a copy of the Diagnostic Check List and your written diagnosis. I would like for you to transfer your written diagnosis to the check list according to the instructions we just reviewed. You will not be timed on this task, but work as rapidly as possible. You may begin. (If subject adds diagnostic statements to the Stimulated Recall sheet, remind him that those statements are to be added to the check list by using A, B, C, 0, etc. in the columns.) To complete our work, I would like to have you clarify for me the way you went about making your decisions for the diagnosis. To help you with recall, I have recorded the information you requested in the order in which you requested it here on this sheet. (Indicate large blue sheet--Hypothesis Observation Check List.) I will tell you the name of the information and then I would like for you to complete two main statements by checking and then explaining your responses. 172 Here is a sample (indicate Sample #2--Hypothesis/Observation) of the Hypothesis/Observation Check List. It is not intended to repre- sent a model. In other words, the statements in the sample under the columns EXPLAIN are not to suggest the way your statements should be worded. Notice that in some instances more than one answer has been checked. (Indicate fifth column for third example.) You may check more than one answer whenever you feel it is appropriate. Be sure to explain each time you check a response. 00 you have any questions about the sample? Now I will give you a chance to practice using the Hypothesis/ Observation Check List form. (Give subject practice Hypothesis/ Observation Check List.) Three items of possible information on a reading case have been filled in under column 2. Assume that you requested ppg of the pieces of information listed and complete the practice for a hypothetical case using that one piece of information. If you have questions, be sure to ask them. (Allow time for task.) Now I will give you your list of requested information. (Give subject his own Hypothesis/Observation Check List.) I will show you the information in the order in which you requested it. You are to check your response in column 3, then explain it in column 4. Then check your response in column 5 and explain it in column 6. Use the same procedures for each piece of information you requested when diag- nosing the case. You will not be timed on this task. (Allow time for task.) That completes this session. (See Unedited Dialogue in Appendix D for example of additional format followed in each third or last session.) APPENDIX C CLINICAL OBSERVATION INSTRUMENTATION 173 174 READING DIAGNOSTIC CHECK LIST* DEVELOPMENTAL/PHYSICAL FACTORS I. SOCIAL/PSYCHOLOGICAL FACTORS II. EDUCATIONAL FACTORS III. PERCEPTION/PERCEPTUAL MOTOR/LANGUAGE FACTORS VI. READING-CONTEXTUAL FACTORS V. READING—ISOLATED INSTANT WORD RECOGNITION FACTORS VI. READING-WORD ANALYSIS IN ISOLATION FACTORS VII. READING-ORAL/SILENT FACTORS VIII. READING-COMPREHENSION FACTORS IX. OTHER FACTORS X. *Developed by Linda Patriarca, Joel VanRoekel, George Sherman, and Ethelyn Hoffmeyer. Strength Weakness Observation 175 I. DEVELOPMENTAL/PHYSICAL FACTORS General Health General Physical Development Physical Coordination Physical Activity Level Physical Activity-Sports Vision-General Statement Vision-Acuity Hearing-General Statement Hearing-Acuity Allergies Birth Process Neurologic Speech Production Other Statements: II. SOCIAL/PSYCHOLOGICAL FACTORS Intellectual/Educational Potential-General/Overall Verbal Intellectual Potential Nonverbal Intellectual Potential Verbal Performance Compared to Nonverbal Performance Potential for Grade Level Work- General Statement Potential for Grade Level Work- Reading Strength Weakness Observation 176 Home Background-General Statement Home Background-Sibling Relationships Home Environment-Influences on Academics Home Environment-Influences on Reading Attitude Toward School-General Attitude Toward Reading- Instructional Attitude Toward Reading- Independent Classroom Behavior Relationship With Peers Cooperation in Group Activities Ability to Work Independently Ability to Work in a One-to- One Situation Social Adjustment-General Level of Responsibility Aggressiveness Emotional Adjustment-General Confidence in Own Ability- Academic Maturation Variety of Interests Willingness to Participate in Competitive Activities Ability to Deal With New Situations Ability to Retain Information Attending Behavior-General Statement Attending Behavior-Getting to Attention Attending Behavior-Selecting and Organizing Strength Weakness Observation 177 Attending Behavior-Pausing anngeflectinq Attending Behavior-Maintaining and Sustaining. Appropriateness of Verbali- zations Socio-Economic Status English as a Second Language Other Statements: III. EDUCATIONAL FACTORS Grade Level Placement-General Statement Motivation-For Academic Work Motivation-Reading Progress in School-General Statement. Progress in School-Reading Quality of Instruction Instructional Materials- Appropriateness-Genera] Statement Instructional Materials- Appr0priatenesseReadjng. Completion of Assignments Rate of Work Accuracy of Work Amount of Practice Required in Subject MatterzAreas Level of General Information Other Statements: Strength Weakness Observation 178 IV. PERCEPTION/PERCEPTUAL MOTOR/ LANGUAGE FACTORS Perception-General Perception-As Related to Academic Growth Auditory Memory-General Auditory-Sounds Auditory Memory-Words Auditory Memory-Sentences Auditory Sequencing Auditory Discrimination-General Auditory Discrimination-Sounds in Words Auditory Discrimination-Whole Words Visual Perception-General Visual Memory-General Visual Memory of Words-Recognition Visual Sequential Memory Visual Sequencing Visual Discrimination-General Visual Discrimination-Shapes Visual Discrimination-Letters Visual Discrimination-Whole Words Visual Association-General Visual Skills in Relation to Auditory Motor Development-General Motor Development-Onset of Walking Gross Motor Coordination Skills Strength Weakness Observation 179 Fine Motor Coordination Skills Ability on Paper-Pencil Tasks Language-General Verbal Skills (Syntax) Articulation Vocabulary-Oral Vocabulary-Reading Listening-Receptive Language- Generg] Li steni ng-Abi l ity to Comprehend Spoken Language at Grade Level Other Statements: . READING-CONTEXTUAL FACTORS Oral Reading-General Oral Reading-Score Oral Reading-Accuracy/General Oral Reading-Hesitations Oral Reading-Insertions Contex- tually Acceptable Oral Reading-Miscues Contextually Acceptable Oral Reading-Omissions Contex- tual 13L Acceptable Oral Reading-Punctuation Self-Correction of Oral Readinq,Errors Silent Reading-General Silent Reading-Score Strength Weakness Observation 180 Word Recognition-Contextual Word Analysis-Contextual Independent Reading Level Instructional Reading Level Frustration Reading Level Frequency of Independent Reading Application of Isolated Skills to Contextual Readinq Reading Performance Relative to Grade Placement Other Statements: VI. READING-ISOLATED INSTANT WORD RECOGNITION FACTORS Word Recognition-General Word Recognition-Basic Sight Words Basic Sight Word Score Utilization of Whole Word Approach Ability to Deal With Irregular Spelling;£atterns Consistency of Error Patterns in Word Identification Other Statements: II. READING-WORD ANALYSIS IN .ISDLAIION.FACTORS Word Analysis-General Strength Weakness Observation 181 Phonetic Analysis-General Use of Initial Consonant Sounds Use of Final Consonant Sounds Use of Blends-General Use of Blends-Specific Use of Digraphs-General Use of Digraphs-Specific Use of Vowels-General Use of Vowels-Specific Use of Vowel Pattern Use of Vowel Variant Pattern Structural Analysis-General Use of Prefixes Use of Suffixes Use of Word Chunks or Roots Use of Word Families Use of Phonograms Use of Syllables Ability to Decode Polysyllabic Words Ability to Blend Component Parts Ability to Blend Component Parts Auditorallv Integration of Analysis Skills Ability to Pronounce Nonsense Words Other Statements: Strength Weakness Observation 182 VIII. READING-ORAL/SILENT FACTORS Oral Reading-General Silent Reading-General Fluency in Oral Reading Fluency in Silent Reading Rate of Reading-General Rate of Reading-Oral Rate of Reading-Silent Rate of Oral Reading With Respect to Accuracy Use of Context-General Use of Cfintext to Determifie Word Pronunciation Use of Context to Determine Word Meaningg Influence of Sight Vocabulary on Reading Rate Influence of Decoding Ability on Readinngate TAppropriateness 0f'1ntonations (phrases 81 clauses) in Oral Reading Other Statements: IX. READING-COMPREHENSION FACTORS Comprehension-General Comprehension Of Grade Level Materials-Oral ComprehensTon of’Grade LeveTTF Materials-Silent ‘Influence of Decoding on Comprehension Influence of Knowledge of Word Meanings on Comprehension 183 Strength Weakness Observation Influence of Interest on Comprehension Comparison of Oral to Silent Reading Performance-General Comparison of Oral to Silent Reading,Performance-Score Recall of Sequential Information Use of Specific Strategies for Comprehension Other Statements: X. OTHER FACTORS 184 Ncozuo Nzocoz o umomaom Nzoooz o ELFoooomFo Nzucoz o ELchou -...lilllilll fl. Ncozoo Nzoooz o Fmocmom Nzoooz o ELFmooomFo Nzucoz o ELchou O | I-III.III.IIF|I11| O 1 Afil‘llllli'|101-1111111 I Ncozao Nzooo; o Fmoomom Nzucoz o ELFFooomFo Nzocoz o ELFFcoo Ncozoo --- NcoSoEoFcF 25 ppm AFFoom: no» --- 23:58.... 35 ooucoz Fmon so» --- Nzocoz o no: so» 1.. Ncazoo --- N53996:... $5 pom AFFoomo so» .1. NooFuoEcoF:F szu omuooz goon no» --- Nzoooz o no; so» 1.. Ncozoo Nzocoz o Fmooaom Nzoooz o ELFFooumFo Nzocoz o ELFFcoo .cFoFoxN a: moon ---:oFFoscoFoF szu :3 mmou coFuFumocooFo #11 Ncmzoo --- NooFuoELoFcF szF Foo mFFoomo 2oz --- NcoFFoscowoF szu ooucoz goon 3oz --- Nzuczz o to: ac» --- Ncacoo --- NooFuoscoF:F szu “om zFFoomo oox --- NooFooacoFoF szu ooucoz umon no» 1.. Nzocoz o no; ooz --- .cFoFoxN F3 ---omooooo :oFuoEcoF:F szu coF zoo 3oz oFo E Fmooooz oou goose FNF mEFF umooooz wow 3 hsz zuuzu zth<>mummo\mme:h0a>z 185 FORM FOR QUESTIONS ASKED FOLLOWING LAST CASE IN INFORMAL DEBRIEFING Before we end the session, I have a few questions I would like to ask you about your opinion regarding some aspects of reading diagnosis. 1. In your opinion what kinds of information should be included in a "good" diagnosis? 2. How do you usually go about a diagnosis? Thinking in terms of the case with which you have just worked-- 3a. 00 you think the student's instant word recognition was low? Yes No b. How do you know? Reasoned in terms of relationships (cause/effect--pupils who have -——-this problem do this [behavior]). ____Reasoned in terms of experience with other students. Reasoned in terms of authority (general reference to authority ——— book, course, etc.). ____Other. 4a. 00 you think the student's word analysis skills were low? Yes No b. How do you know? Reasoned in terms of relationships (cause/effect--pupils who have --this problem do this [behavior]). ____Reasoned in terms of experience with other students. Reasoned in terms of authority (general reference to authority --'book, course, etc.). ___.Other. 186 5a. Do you think the student's reading comprehension was low? Yes No b. How do you know? Reasoned in terms of relationships (cause/effect--pupils who have -—-this problem do this [behavior]). -——— Reasoned in terms of experience with other students. Reasoned in terms of authority (general reference to authority -——-book, course, etc.). -—- Other. 6a. 00 you think the student's fluency was low? Yes No b. How do you know? Reasoned in terms of relationships (cause/effect--pupils who have -——— this problem do this [behavior]). -——— Reasoned in terms of experience with other students. Reasoned in terms of authority (general reference to authority -——— book, course, etc.). — Other. APPENDIX D CLINICIAN RESPONSES ON FORMAL AND INFORMAL MEASURES 187 188 DATA REQUESTED AND NOT AVAILABLE FOR FOUR CASES Sppggpt Nggggr Data Requested, Not Available 101 3 None 101 4 None 101 3 None 102 2 Ekwall Phonics Inventory or Woodcock or Gates-McKillup 102 1 None 102 2 Gates-McKillup Nonsense Word List Bader's Phonics Test 103 1 None 103 3 Psychological report ITPA 103 1 Iowa or Stanford Phonetic analysis for Durrell Gates-McKillup, sounds and their relationships 104 4 None 104 2 None 104 4 None 105 3 None 105 4 None 105 3 Botel Dolch Word List Memory Battery of Woodcock Botel (2nd request) Creative Writing ITPA Speech report Counselor report Kottmeyer Miscue Inventory Subject Number 106 106 106 Case Number 4 2 Run Number 1 2 189 Data Requested, Not Available Detroit or visual perception Auditory discrimination Health record ~ Gates-McKillup oral reading and examiner's comments Cover sheet of Durrell or summary profile Durrell Spelling or Gates-McKillup Weschler full-page profile Gates-McKillup 107 107 107 Stanford Achievement Durrell Spelling Test Handwriting sample Peabody vocabulary and comprehension Motor ability test Informal Reading Inventory Peabody score interpretation Wepman auditory blending 108 108 108 Psychological tests Informal reading inventory Kottmeyer ITPA or auditory memory Expressive language indication Visual acuity Visual discrimination Left-right dominance or laterality Writing sample or writing sample on spelling test of Durrell Drawing test or Bender-Gestalt Dictated sentences Visual/motor coordination skills Wepman Reversals test Examiner's comments for Individual Reading Analysis Durrell Visual Memory, intermediate level 190 .momFF FFo ooFcu oz pogo oouo: on oFoozm FF oFoFmoom zFFocooom mgoccm .moooom uuocooocF ooooz ooo .ooooo .oouquo .ooFquomF oF moooz ooNFomouog oz 3oz oz» :F oopooFFoc ocoz mommocxooz zoooFo=< .coFmooz logosoo oF acoFo>Fooo ooocm m.m ooo zLoFooouo> :F pooFo>Fooo oooom m.m ooooom oz .onmchozumopou .muooso>onoo Foozom Foouoo oco mpmocoqu ooozuoo monmcoFquoo ozo poooo mooFuooocoomFE no; oozoooF .coooo ocu eocF mozocF m moxo sz Foox o; mo oooo mo: ocoumoo .mFFFxm moFoooo oFmoo Fo zoo; .umFF ooogm ocm oz» co oooz uoooooo ooo AFoo on: .mooo: czocxoo oooooooco op ooz LocFonm .coFmoozoooEou moFo; uxoocoo oo oocoFFom .zFFooFuocozo mocoz oooooo op poeouoo Foo oFo .mocoom moo< .moooom mquo .oFoooLu 2F; o>om ucFoo oFFuFF ooo .moooz oFFuFF .mogoz mFm mucosououm uFumoomoFo =mcozuo= co» poossou EFuoooo> 3 3 3 FF> 3 F> 3 3 FFF m m FoF 3 3 x 3 3 F> m 3 > N v FoF 3 3 3 >F F m FoF No.3.mv Loosoz Loosoz Loosoz Loosoz ooFo> zgooopou com omoo Foonoom hsz xumzu thmozw m N NoF .ooFooo =oF= z FF> N F NoF .coFmoozoLoEoo moFoousz m xF F N NoF .ooFmoozooosoo mchomeo 3 xF Fo.o:oov .ooFuFomoooc ocoz :o moFxLoz zFooooco mo: 3 FFF> m m FoF mucosououm oFumommoFo =moozoo= Low ucoeeou EFpooLo> Fo.3.m Loosoz LooEoz Foosoz goosoz ooFo> zoomooou com omou uoowoom 192 Fo>oF oooom poo .moooom poooooocF .mocoom mooo .moooom muFEouumcFFFoom .mconFpooooulmoFoooo Fooo .moFmoozo oooouimcFoooo Fooo .ooooFoosF oFomosuooFmF> .oFoouoooooo: ooosz 33333 NoF NoF .mooossoo oz .Fooooom monmooFquoo FoFuoom .moFEooooo oF oooosoooooo op ooquoo ocooom ooomFF zoouFoo< .FoF zuFFFoov mooooooo Fo ooFoooFFoo< .zoFooz op ooo moooomoo pooooooo .coFmoozoooEoo moFooomFo 033 U) >F >F FF xF moF moF moF .uooeo>onoo moFuFozoco: .ocoso>onoo moFFFoom .ooFmoozoooEoo co ooomFe oooz oo oooooFooF .ooFmoozoooEoo oo ooFuFomoooo oooz oo oooooFFoF .Fooo Foo .FcFoo :F mooo3 mucosouoom oFumocmmFo =moozuo= ooo Fooseou EFuoooo> 33cc 3 xF F> Ao.3.mv ooFo> ooosoz zoomooou m oooEoz com m oooszm omou moF ooosoz uuonnzm 193 .oFFFooo ooFoomFo mcFooooo o x .ooFquoooo moFoooo Fooo 3 > .oonooFFosoo zoooooFomoz o F m N NoF .Fcossoo oz F F NoF .muFooooo FouooE op Fooso>onoo .mcFooamFF Fo :oFooFoz 3 FF F N NoF .FooanFuucoFmoozooosou 3 xF .oFoooooouooo 3FFo=Fxouooo ooosz 3 > N F NoF mucosouopm oFumoomoFo =moozuo= ooo FooEEou stoooo> .1o 3 m 1moos=z oooEoz oooEoz 1Uoo232 o F > zoomopoo com omou poonoom 194 UNEDITED DIALOGUE OF INFORMAL DEBRIEFING FOR SUBJECT 104, RUN 3, CASE 4 Before we end this session I have some questions I'd like to ask you regarding your opinions about some aspects of reading diag- nosis. In your opinion, what kinds of information should be included in a "good" diagnosis? Well, the first thing that I'm concerned about is to make sure that there are no unusual factors such as hearing deficit or hard of hearing or partially sighted or some emotional factors that I'm not aware of such as child abuse or some other factor which is affect- ing the child either physically or emotionally. But most of the kids that I see that are in the regular school rooms, the so-called typical kid, most of them come from so-called normal environments and they're not wearing glasses and they don't appear to be hard of hearing. Once that's out of the way, I kind of discount any emotional effect on the child or any abnormal physical effects on the child. The first thing I want to know about the kid is what does he know about words; how many words does he know; and I usually check this out by--I usually have kind of an informal-- I pick up something quite easy for them to read, have something they want to read to me and the kid normally picks up something that he can read. Occasionally he'll pick something quite diffi- cult, then I have to kind of gear them toward something else. After having them read to me orally, I get kind of a hunch where they are already but then I give them the Slosson test, the Slosson Oral Reading Test. You get some kind of an approximate grade score or an idea of how many words they know by sight. Then I usually follow that up with some type of inventory. I like to use the Ekwall because it has some nouns in it, other than just the verbs or other words that Dolch has, to see which words they miscue on when they read words in isolation so I can see if there are some consistencies in their miscues. I also find out which words they know. That's going to be helpful to me later on when I start working with them in remediation. Taking words that they know for example that the regular words that they follow some type of word family. I might be able to show them some phonic skills in the area of word families or whatever I feel they're weak in-- blends, digraphs, or word endings. I take the words they already know and try to work out some word patterns or some systematic approach. So the kid says, "Yeah, there is some regularity to our language." After getting an idea of what they know about words in isolation and how they attack words in isolation that they don't know, that they don't know instantly, I get some kind of a clue into their understanding of the phonic system, and I usually try to administer either an informal or a phonic or a more formal 195 phonics test to see if they know initial and final consonants, blends, diagraphs, multisyllabic words, vowel rules. I usually don't get too carried away with it. The kids I work with are usually in about 4-5th grade, and they're down reading about 2nd, 3rd grade level. About the only thing I'm really concerned about in the area of vowels is do they understand the word families and short vowels, and do they understand the vowel rules that have high frequency of application of which I think there are very, very few. Then I'm kind of concerned about what do they do with the knowledge of the words that they have, instant sight word recognition and their knowledge of phonics. How do they apply this? Are they still word-by-word plodders? I've run into some kids that know a lot of words or they'll know 200 Dolch words or 210. They'll have a pretty good understanding of the phonics rules, they can sound out most any word that they see in isolation and yet when they--when you ask them to read it's almost word- -by- word plodding. So, I go to the oral reading to see how they apply this if they do read in phrases. Do they read, "under - the - or "under the. " It's got to be "under the something. " And some of the kids just won't do that. Or it'll be "under the bridge of the cow," and they don't apply our punctuation system. 50 kids that know a lot of basic sight words and kids that know the phonics system, but don't apply it are really kind of confusing because they give you the appearance of being able to read fluently. If you don't listen to them read orally you don't really have an under- standing that they are reading word-by-word. Particularly in the upper grades some of the kids can survive with this word-by-word reading. And they can still get satisfactory scores so the teacher doesn't get too upset. And there's not a lot of oral reading going on in the upper grades. But I think that the oral reading is the method that we should use for diagnosis to find out, "Hey, what is the kid doing right and what's he doing wrong?" Where I find a lot of breakdown is in the application of systems. Either they don't know the systems or even if they know, if they know the systems, they don't apply them in their reading. I don't get too carried away with comprehension. Most kids that I work with that are deficient in phonics and structural analysis skills and basic sight word knowledge, they still can understand anything written at their grade level. The stories aren't that complicated and if they can read it they can usually give you back the factual, literal recall. I still think we're really shaky on what compre- hension is other than the literal recall. I think we're shaky in the cognitive understanding of what happens in the reading act. There are some basic comprehension skills that I assess other than literal recall, but most of the time I don't get too carried away with it because I find the kids will understand what they're read- ing if they only know the system and apply the systems. Because most of the kids understand if you read to them. That's why I like to give this listening comprehension. For example, the kid we worked on today, oral and silent reading--they mirrored one another. We couldn't hear his silent reading but I suspect that he read 3b. 196 silently just like he read orally, word-by-word, made the same miscues on those high frequency words "there" for "three," "then" for "when" and "when" for "then" on different occasions. But yet, when he was read to, when you asked him for literal recall, he understands. Kids understand. In fact, most of them are quite s0phisticated in their understanding of the spoken words. They understand what they hear on television. They understand what their friends say to them. But if what was said on television was reduced to a script, such as it was in this Walton's Pearl Harbor thing, and there was one other one that they did, once you put it down visually, p10p it in front of them, their comprehen- sion goes all to hell; they don't know the systems; they don't know the words; they don't know the phonics, or they just don't put it together. Visually they just get nothing. Yet when they see it on T.V. and you ask them the questions about it, they've got it. So, I don't get too carried away with comprehension. I think the kids in the elementary grades, the normal kid, doesn't have any trouble comprehending the material that he's reading. I think you've kind of answered the next question, which is: How do you usually go about a diagnosis? Did you finish that? Let's see--we got mathe informal. Yeah, the informal. I check sight words. I check phonics. I check structural analysis, and then I check application of the systems and then I do some limited assessment in comprehension. And you feel that that's pretty much in answer to the first ques- tion too, like what do you think a good diagnosis should include? That's what I think it should include. . Thinking in terms of the case with which you just worked, do you think that 's instant word recognition skills were low? Yes, definitely. And how did you know that? By his scores on the Slosson. I think it was 42, which indicated about the beginning 2nd grade. What about 's word analysis skills? 3c. 3d. 197 Very poor because he has just a confused, hodge-podge understand- ing of phonics. He seems to know initial consonants, but he doesn't look at word endings. His miscues are--well, you could find probably half a dozen patterns of miscues. He just doesn't seem to do anything right other than maybe clue in on that ini- tial consonant. Sometimes he doesn't even do that when he says "saw" for "was" or "yes" for "say." Do you think that 's comprehension skills were low? Oral and silent reading. And the reason they were low in oral and silent is because the kid can read, can't apply the systems, doesn't know the words, and doesn't know the phonics. But when he was read to with the Durrell, he scored very well up to 5th grade, answered 7 of 9 on the 5th grade one. So you're saying his listening comprehension is good? I could read to him 5th grade material. He understands it. So, you've determined that the oral and silent were low and that the listening was ok. How did you know this? By the scores and the reason I like to get these scores. I think that difference between listening comprehension and silent--oral in his case which were 2 years apart, this gives you an idea of where you can bring that kid; his potential. Let's say he got 9 out of 9 at the 7th grade, then that indicates that the kid's really got a lot on the ball; something's screwed up. The kid understands up to 7th grade, who's only reading on 2nd grade level. If we can only straighten out where his problem is we should be able to bring him up to 7th grade level. On one of the other children that we had when we did the other ones. Sometimes we get a little confused when we say well how far can we bring this kid along. We find that their oral reading comprehension, their silent reading comprehension, their listening comprehension are about all the same. Then I start saying, "Hey, maybe the kid's working up to grade level even though he's in the 4th grade reading at 2nd grade level. Maybe we shouldn't be pushing this kid any harder." That's--but when you see the big discrepancies. . . . You see the big discrepancy. This gives you an idea of where you can bring the kid toThis potential. What about Brian's reading fluency? Do you think that was low? Yes, very definitely and it goes back to the application of the systems. He doesn't understand the systems, so we can't expect 198 him to apply the systems. He's just a word-by-word plodder. If he miscues the words in isolation, he's still going to miscue them in his oral reading. Although he made attempts, he realized that "Hey, this just does'nt make sense if I say 'then' instead of 'when'" and twice in his oral reading, after he completed a whole sentence, he went back knowing that "Hey, it can't be. It didn't make sense." So he went back and plugged in the right word. Oh, he said "next to" for "near." Well, "next to" and "near" are pretty close in meaning. So, he knows that reading has to make sense, and he's bothered when it doesn't make sense. He just doesn't continue right on. There was one time, he kind of went on because it didn't make any sense. But you can almost sense in his reading when he miscued and there was a loss, a real loss in comprehension, he went back and reread the whole sentence. Well, do you have any other thoughts concerning reading diagnosis and remediation that you'd like to share with me? Just kind of off the cuff, any concerns you have? I am a little. The thing like this kid we did today, you know, strong auditory learner in his Spelling of nonsense words--"car- plite." You could see him putting in the "c" and the "r" and the "p" and the "i"--maybe mixing them up because he doesn't apply it. But that kid heard the words and could find them on the hearing sounds in words. He got 28 out of 29. When you show him the word visually, he doesn't know how to look at that word. He just-~I think he scored 1.5 on that. So, when he sees something, he has all kinds of trouble. But with the combination of hearing and seeing I think that kid can be remediated. He can be helped a lot. And along that line, when you say you think he can be remediated, and thinking maybe more in terms of college students which you might be teaching, do you have any suggestions for improvement of instructing reading teachers, maybe thinking in terms of diagnosis and remediation? Yeah, one of the things that I think that, first of all, some of the courses that I've taught other than 830C, the beginning course. They don't seem to know the scope and the sequence of phonics skills. The college students? The college students taking the graduate level reading courses, they don't seem to know the scape and sequencing of phonics skills and the other thing that they seem to be weak in is when they look at the words in isolation and they're looking for error patterns they seem to have trouble seeing that "Hey, the kid's got a vowel problem" or that the kid is miscuing on words that are highly 199 similar in appearance. They'll look at the words "there" and "three" and say that the kid doesn't know his vowels. Well, when I look at "there" and "three" t-h-e-r-e and t-h-r-e-e, I call those highly similar appearing words and that's not a vowel problem. "Want" and "went" they'll say is a vowel problem. Those are both, well "want" in particular being an irregular word, it isn't a vowel problem if a kid doesn't know "want“ when he sees it he says “went." How might you suggest that we improve this in teaching? Oh, I could talk for a long time. There are the things I see that teachers do. Maybe you should light on one that bothers you the most. Well, another thing that does bother me is we spend so much time on words in isolation and working on helping kids read words in isolation. They so infrequently read words in isolation. I think we have to tie our instruction in phonic skills, in structural analysis skills and teaching of basic sight words, we could be tieing this back into the context. In the beginning, context with noun phrases, verb phrases, prepositional phrases, and as soon as possible sticking it into whole sentence context. I see too much instruction on an isolated word basis and like light I think for example, the word light, he gives you light, doesn't give you a hell of a lot out of isolation or out of context. It can mean so many different things. Or words like "want," “need," or whatever. Pick any of those isolated sight words. They don't mean much out of isolation. So I think we should be making an attempt to teach the phonics skills, the structural analysis skills, and basic sight words in context as soon as possible because then this is going to improve the fluency and is going to get the kid to applying the system, the very systems that belong to him. How do you see diagnosis and remediation related or interrelated? It should be a constant, ongoing process. How you make your ini- tial diagnosis, and you start working on it, you're going to find out other things that you missed in your initial diagnosis and then as you're working with the kids and you see progress in various areas, you kind of start working on the strengths and con- centrating on the deficits where the kids really have problems, but the diagnosis and remediation have to be continual then. Do you think the diagnosis is important as such? You know, well, let's put it this way. Do you see yourself being able to success- fully remediate a student without going to a formal kind of diag- nosis, maybe the one that you mentioned as being a good diagnosis? 200 Well, the one I talked about I think is a simple, well maybe a little bit too simplistic or too simplified, but just because of the problem of time and number of students I have to get realis- tic and maybe a little bit pragmatic about how I diagnose the kids. But I think you have, you know, you have to have an initial diagnosis. If you don't know where the kids are, you don't know where the hell you're going. Poor plan's better than no plan at all. So, as soon as possible in the first grade like at the grade level I teach, as soon as the kids get in there, I find out which letters they know, which letters they confuse, which sounds they know, do they know initial consonants, are these kids already reading, what words do they know, what is their knowledge of basic sight words, and how much do they know about phonics, and take off from there. But, if I just walk in and you know and assume that the kids know such and such and started teaching in different groups or something there would be just a hodge-podge. Do you think that the graduate students that you teach have an idea of how to go about a good diagnosis and remediation? Do you think they have a model? I think they will after the seminar. So you do work toward helping them develop a pattern of a good diagnosis and what's included in it--the diagnosis? Yeah, I try to do it again in a practical way. They don't have time to sit down and write a 3-hour diagnosis. In one diagnosis class that I worked with, they did one full-blown lO-12-l4-16 page diagnosis on their most difficult child and then in the last 2 days we sat down and did it in 15 minutes. I'd say, "Now in 15 minutes tell me what the kid knows about the sight words. Tell me about his flash recognition and his analysis of basic sight words, look at your Durrell, look at your SORT, look at your Ekwall list and your Dolch list and for 15 minutes tell me what the kid does right, tell me what he does wrong." And then we'd go into the phonics. I'd say, "All right, hey for the next 20 minutes you're going to say, 'Hey, what does the kid know about phonics?‘ Tell me about initial consonants, final consonants, blends, dia- graphs, vowels, suffixes, prefixes. Put it down. You've got 20 minutes, do it. Then tell the teacher next door who's got the other 2nd grade classroom about it." Then in another 15 minutes I'd say, "What does the kid know about word patterns? What does he know about punctuation? What does he do with noun phrases, verb phrases, prepositional phrases? How does he read when he reads orally? Write itin 15 minutes." And then, finally, I say, "OK, now comes the toughy. Tell me about comprehension. What does the kid do with comprehension? How is comprehension affected by his knowledge of phonics, sight words, his application system? 201 What is the student's comprehension? Write for 15 minutes." So, in four lS-minute periods they've pretty well covered some type of a--they draw some theoretical ground issues in those four ideas. This is what I hope they do when they get back to the classroom. Well, this leads me to my final question, which is how did you feel about the task that you had to do for the three sessions and what about the time pressure that you were under? Did you feel it was realistic or difficult? What did we do? We had how many minutes to diagnose? Forty-five minutes to request information on the case and then 25 minutes to write the diagnosis and 25 minutes for remediation. Yeah. That's realistic. It may be longer than the people out in the field would get in a class of 40 or maybe a class of 50. You've got 50 kids to work with in your caseload, you may not have that long. Now we were, I guess, most concerned about the time of the actual writing, but since it was a time-press situation, we wanted to see what people could do with limited time. Did you feel that you improved any in your writing between the first and last time? Oh, just for example I think the others I had 13-14 diagnostic comments that I had to identify and record in the book. Today I think I had 21. I think I performed better today than I did the other two times and that could be familiarity with what was expected of me--the process that I was going to be going through and less anxiety. Well, I certainly appreciate your time. This concludes our three clinic sessions. 202 Table Dl.--Subject responses to informal questions regarding memory (1) and strategy (2)--Subject 101, Case 3. What information should be included in a "good" diagnosis? 1 How do ypg_go about a dia nosis? 121 1. A look at behavior. 2. Recognition of words in isolation. Pre-reading skills strengths and weaknesses. 3. Ability to figure out words in reading situation--oral, silent and listening. 4. Oral reading for clue to compre- hension by way he figures out words. . Get initial information about child from his teacher and his parents; also speech teacher or social worker if that applies. . I might also observe child in classroom. . Then I interview the child, get acquainted and get self-report informa- tion. ' . I use a word list or quick measure of reading behavior and grade abil- ity; sometimes I use child's own book. 203 Table DZ.--Subject responses to informal questions regarding memory (1) and strategy (2)--Subject 102, Case 2. What information should be included How do ygg_go about a in a "good" diagnosis? dia nosis? 121 1. Instant word recognition. 1. Home background through telephone interview with mother. 2. Word-attack strategies. 2. Slosson Oral Reading 3. Fluency. Test 4- Rate. 3. Parts of the Durrell 5. Literal comprehension. 4. Informa] reading inven- tory for comprehension. 5. I.Q. test only if appears to be a problem. 204 Table D3.--Subject responses to informal questions regarding memory (1) and strategy (2)--Subject 103, Case 1. What information should be included How do ypp_go about a in a "good" diagnosis? diagnosis? 1 (2) 1. It is based on the individual I look at: child. But I guess I ask for information on the environment 1. Word recognition and maybe I.Q. 2. Word analysis 2. Information most helpful is look- ing at the child's oral reading 3. Comprehension, and performance. Oral reading gives hunches related to ability to 4 attack words--attempt to decode words that are not known. If he performs well there I wouldn't need to go to Dolch Word List or Slosson. . Attitude. 3. Should look at ability to use sound-symbol relationships. 4. Comprehension, including use of context clues. 5. Standardized test scores for grade placement. 6. Background information. 7. Why child referred. 205 Table D4.--Subject responses to informal questions regarding memory (1) and strategy (2)--Subject 105, Case 3. What information should be included in a "good" diagnosis? (1) How do ygg_go about a diagnosis? (2) 1. Focus on strengths of child. 2. Relate field of reading to child's other language arts skills. 3. Assurance on elementary skills, especially older children, such as basic Dolch sight words, sounds of letters or blends, word analysis. 4. What motivates a child--the affective process. 5. Self-diagnosis by the child. . Intensive interest inven- tory to get acquainted. . Screening for quick evi- dence of grade level and for basic skills-~SORT, Slosson, I.Q., Botel word opposites. . Gates-MacGinitie to con- firm reading level. . Look for divergence that would respond to whatever I begin to find out about child. 206 Table DS.--Subject responses to informal questions regarding memory (1) and strategy (2)--Subject 107, Case 1. What information should be included How do ypp_go about a in a "good" diagnosis? diagnosis? 1 (2) 1. You need silent and oral reading The way I just explained and compare the two. it. 2. You should use listening compre- hension or if there's a WISC available that would do for poten- tial. 3. I use the Slosson with older kids or Dolch with younger ones for sight word recognition and grade level score. 4. Most important thing is listening to reading. 5. An informal inventory to support a standardized test or for a quick check a group silent test. 6. An interview with student. 7. Background information from records, talk with teacher. 207 Table 06.--Subject responses to informal questions regarding memory (1) and strategy (2)--Subject 108, Case 2. What information should be included How do ypp_go about a in a "good" diagnosis? diagnosis? 1 (2)- 1. Listening comprehension test if It depends on the situation. not an I.Q. measure. If it's to confirm a teach- er's finding I would use 2. Extensive informal reading just the reading portion of inventory--oral and silent para- a diagnosis. If I have the graphs with comprehension checks time I prefer a more thorough and word recognition skills. diagnosis that gives the ad- This yields information as to vantage of checking visual child's performance on school- and/or auditory difficulties type tasks, tests, etc. and some measure of capacity and strength. 3. Depending on case, you would go into individual analysis of either specific reading skills or perhaps auditory areas or visual areas. Efficient diag- nosis should eliminate testing in areas such as visual unless indication of a problem. 208 .szoooo zF Eonooo »o:ooFF o m.ooozp »ozo pszo 3ozx so» om opoooooooF mF »ooF -ozooo> oooz pszm sz .sz -oooo mo: oz zozz mzonomeoz »ooE oooz ooozp "oncommoz 52.8 H 3:652 11 oozoFoooxu.11 Foooz \. Nzooz oo» oo 3o: .oooo wwwoommoz oz mo> \. NzoF »u:ooFF moFoooo mo: .mFFFzm oonFzmoooo oooz oooo Fo omoooom Homoommoz 8:8 H »pFoozp=<.kP oozoFoooxm.11 Foooz Nzoox so» on 3o: oz \ 66F NzoF :oFmoozoooeoo mcFoooo mo: .ze .oo .zm .om .mzonomeoz 3o oozsoz o zsz oooo oz .oonp .moooz .ooooom oonp o ooo FFoz oooom me Fo NFo »FchooF »ppooo moFoo mo: oz oe op »Fco oFoou oz mouoosoooooo oopooFooF sz» .mEoomooozo ooo3 zoFoo mo: ooFu oozpoc< Fozo> poozm ozo mFo3o> .pooeoooFo oooom sz zoFoz poozm sz .moooom pzozom oooe oo ooo» o poozo mo: -zoo FonFzF sz zozz oF oozoeoowooo sz .onFoFoooz oz meF oooz co "omzo mom -mopow po szzooz "omzommoz oozpo.11 oozpo.11 3 {9:2 mu .3 22:2 .\.1 oozoFoooxN.11 .11 .11 ouzoFoooxm.11 11. .11 Foooz oz \ mo> Foooz oz mo» \ Nxozz oo» oo 3oz .zN NzoF mFFFzm mFm»Fo:ouooo3 ooo3 .oN Nzozz oo» oo 3oz .zF -1111-.I1‘Ill NzoF :onchouoo oooz pzopsz mo: .oF .F omou .moF poonzom --moFoooo Fo mpooomo oFFFooom szooomoo moonmooo Fosoo3zF op momooomoo puowzom--.oa onoF 209 .mocsom FoFooe oooosz ooo mFomoo>oo osom ooz omFo o: .mpooo ooo3 Fooo uzom moms oz mpoooo ooo3 o ozop p.zmooo o: Homoommoz .zoFoo ozp co :Fomo os oozoFo F pozp mooooo oozpo ooo3 ooozh .zFFE zoFoo p.omooo :ozono < .omzom ozoa p.zoFo pF zozz oopm poz oFo o: =.zozono= =zopsz= _ooFFoo oz sz -oooo mo: oz zoz3 ”omzommom oozpo.kr oozpo.11 »p .Foozps< I »p .Foozps< D oozoFoooxm 11. 11. 11. oocoFoooxN 11. 11. 11. Foooz oz mo> \ Foooz oz mo> \ . NzoF mFFFzm . NzoF :onFzmoooo 63ozz so» oo 3o: mFm»Fozo1oooz ooo3 ozozz so» on 3oz ooo3 poopsz mo: .zN .oN .zF .oF .F ammo .NoF paowzsm 11szoooo Fo mpooomo oFoFooom szooomoo moonmoso FoEooFoF op momzoomoo puonzsm11.mo onoF 210 .oooo mo: szmoozo sz mo oeom ozo :onospozso oooosz omFo o: .»ozo 1sFF ooooFo>oo p.zmoz pmsn oz mooo: o ooo szzoooom m»o:Fo p.zmo: o: .ooo:1» -ooo: pmosFo m.pF "omoo mom oozpo.kn »pFoozps<.11 oozoFoooxm Foooz.11 . oz mo: pF ”omzommoz oz mo> \ .szoooo m.oz poz: szoooz mo pzoeoooowoFoo moooz oz pozp oE FFop oFso: pozp om .Fooo ozp mo ooom mo p.:mo: :oFmoozoooEou szoooo pzoFFm mF: .onopooooo ooo: ozo omzom ooos »ozp psz .ooo »o: ooo: mosomFe sz Fo oEom .pF ozsooo zoo: oFso: oz11omzom pmoz ozp ozos m»o:Fo p.=oFo oz zosozp =o>o ooom »FFozonoooxo mo: pF szoooo Fooo ozp zmsoozp »o: ozp FF< ”oooommoz oozpo H zpFoozps<.11 ooooFoooxm F656: \ .ooom »oo> »FFooo mo: pozp » mop 1mFooosm "omzo mom N:ooz so» oo :o: .zo N:oF »uoosFF moFoooo mo: .oz N:o:z so» oo :o: .zm N:oF zoFmoozooosoo szoooo mo: .om .ooschcoo--.oo onaF 211 .oopoooxo oozp ooppoz moFmoozo moonospooso o>oomzo poo mooo .mooo: ooo:poz momsoo Homoommoz oofio N zpFoozps<.11 ooooFoooxm Foooz.11 oz moz 1M1 .mzooomoooo moFoooo pooFFm ooo Fooo Homoommoz ES 1|1 zpFoozp=<.ov ooooFoooxm.11 Foooz .:oF mo poo moFoomeF ooo Fooo m:oF pooFFm Homoommom oz .ol 8F v.1 N:ooz so» oo :o: N:oF »ooosFF moFoooo mo: N:ooz so» on :o: N:oF ooFmoozooosoo ooFoooo mo: .oo .oz .oN .om .mFFFzm moFoozo :ooz p.ooFo oz omsoooz mo: oozpooo zuso .ooppoF pmoFo poo oo p.ooFo oz pooo .oozomeo p.ooFsoo mostosom .mmosm »oos »oo> monoE psozpF: oFooz oo zooz p.ooFo oz mmeF zoo poopo ozp zoooozp »om oFso: ”mooo: o~»Fooo op mpom oz ”pooh moFoooz .oFoF mo: posoppo p.omooo o: Homoommoz Fooo oommoFm Eooo Homoommom mF: Homoommoz oozpo.kr oozpo.11 »pFoozpoz zpFoozpoo ooooFoooxm I11 1 l1 ooooFoooxm H 1 1| Foooz oz mo> \_ Foooz \ oz \ moz N:ooz so» on :oz .zN N:oF mFFFzm mFm»Fooo1ooo: ooo3 .oN N:ooz so» oo :o: .zF N:oF oonFomoooo ooo: poopmoF mo: .oF .N omou .NoF puoozsm numoFoooo 3o mpooomo oFFFooom moFooomoo moonmoso FooooFoF op momooomoo poonzsm11.mo onoF 212 .oooo 1memmo ooo poooomoosoooo ooom zpF: mooo: oN»F 1ooo op ono oz op »oooo 1oop o oo:ozm oz Homoommoz .855 N »pFoozps<.11 ooooFoooxm Foooz I .moF noooo ooF »pFoooou FFooozo.mHm ooF :oF poo psz Fo>oF oooom sz oo» :o_ ooo: »ozp Homoo mom oz \ mo> \ .moFoooo Fooo ozp oF ooo pmop oonFo 1moooo ooo: ozp mpom oF mosumFs ooz o: Homoo mom .855 1|1 o. .2232 N. ooooFoooxN Foooz 1 oz 8» H N:ooz so» oo :o: .oN N:oF mFF.C_m mFm»Fooo1ooo: ooo3 .oN N:ooz so» oo :o: .zF N:oF oonFomoooo ooo: poopmoF mo3 .oF .N omoo .ooF pooooom 11moFoooo Fo mpooomo oFFFooom moFooomoo moonmoso FoEooFoF op momooomoopooozsm11zpzzono» 213 =.oopoF: FFo :oom ozp oF pso oooz m.pozp oopooop oFo .»pmso o ozFF moosom oz: .mon »5 FFop F m< .oopoo »oo> oooo p.omooo oz moooooo pF . ooo EFz op moFoomeF »m Homoo moz oozpo M »pFoozps< ooooFoooxm.11 $8: I .ooFmoozoooEou oo:oF puooxo oFso: so» pozp opoo -FooF oFso: oosp oF zon: .mFFFzm omosmooF oo:oF .oooum oooomFFFopoF oo:oF ozp monooFFo »Fszmoo mo oopooooopoF oz oFsoo zoo oopoom oopooFFoEou o m.pozz .moFoooFoFFoo »oopFoso ooom op oopoF 1oo m.pozp pooomsm F ooo .mpmop »pFooooo oo Fooooom oF commooooo mo: ooooooow 1ooo Fozoo> mF: Homoommoz oozpo H »pFoozps< ooooFoooxm 11. Foooz .xl oz moo .xl .pozp ooooo -oFooF oooz ozoFo moopooo oo>opoz: oo »pFoooou Fopooe sz zpF: poomem 1oou mo: pF pooomsm F .Fo>oF omo sz ooo :oF psz mFFFzm »oooFuFFwo moF 1oooo sz o>ozo mo: ooFmoozoooooo moFoooz "omoommoz oz N. ...oz w. N:ooz so» oo :oz .zo .:o »uoos omowoooo own N:ooz so» oo :o: .oo .8 N:oF ooFmoozoooEoo moFoooo mo3 .mm .oooonooo--.oFo oFooF 214 .moopzp »ooszso ozp oz p.ostoo oz zoo mFm»Fooo zoo: oF FoooFoz oz zFooz zoopFooz .monoFF»m opoF mzoo: ozF>Fz p.ooo o: .EoF -zooo moFoooz o Fo omsoooz mFFFzm oonFomoooo zoo: o>oz p.omooz oz pozp o>Fp 1Fmoo pmoEFo E.F "omoommoz oozpo H »pFoozps<.11 ooooFoooxm Foooz.m. oz mo>.m1 .:oF »onzooo m.oz omoo pozp oF om .oooo moFoz mo oFomon moF>Fooooo m.oz .EFz o>ozo »onzooo ooo moooo sz Fo »ooo om zoo FFo: moFoz ooozFon moF>oz ooo oonopsooo moz poFomez Foozom psz .oonFomoooo zoo: poopmoF oo m.z moooom z sz oF 11:oF pozp poz Homoommoz oozpo M zpFoozpo<.11 ooooFoooxm.11 Foooz N:ooz so» oz :o: .zN N:oF mFFFzm mFm»Fooo1zoo: ooo3 .oN N:ooz so» oz :o: .zF N:oF oonFoaoooo zoo: poopmoF mo3 Cap .m omoo .FoF poonzsm -1monooo Fo mpooomo oFFFooom monoomoo moonmoso FoEooooF op momooomoopooozsm11.FFo onok 215 .poF o pxopooo oms op zoz11mzoo: omozp FFo o~»Fooo p.ostoo oz .»pFosoomoF p.omo: pozp FF moFoozoo: E.F zoo FFomEFz poooooo zoo oopm op zoz o: .moonFpoooo zoo .moonomeoz Fo poF o ”zoom »oo> p ooo: pF monooo Fooo oF Homoommoz oozpo »pFoozps< ooooFoooxm Foooz.zn .»ozo ze oz .mzoo: moFNFomoooo mo: oz »o: ozp zoo EFz ooo psz o>oz o: pozp moooom poooo>onoo Fo>oF ozoom ozp op moF -zooooo :oF mo: pF Homoommoz oozpo 1|1 zpFoozps<.11 onFFFFosov 11. 11. ooooFoooxm.11 11. 11. oz moz \. Fozoz \ oz \ moz N:ooz so» oz :o: .zz N:oF »ooosFF . N:oF ooFmoozoooEoo monooo mo3 o:ooz so» oz :o: monooo mo3 .oz .om .oo .oooonooo--.FFo oFooF 216 .Fmonooo Fooo Fo ooopv ooFmoozoooooo zoo »ooosFF »oopmoz sto: pF onFooop om moFmoozo Homoommoz oozpo.11 »pFoozps.m1 .ooFmoozoooooo onFoFoooz oopoo zoooozp »FooFsonooo zoooFFooo mo: zoo ooFmoozoooooo oF :oF moF 1zooz Fooo FFoooso Homoommoz oozpo.11 »pFoozps<.m1 ooooFoooqu FozozI oz mo>.ufl N:ooz so» oz :o: N:oF »ooosFF monooo mo3 N:ooz so» oz :oz N:oF ooFmoozoooEoo monooo mo3 .zz .oz .zm .om .oooo pF zommFE .FFo:zm zoo ooF:p puooooo pF pom o FFoooso .osFFon31mopow oz ...Foszoom= zoo:11poop .» zozooo oooF FoFoopoz Homoommoz oozpoI »pFoozps<.kV ooooFooome Fooozl .»onzooo Homoommoz oz mo» UH. 1mFmooooF mo: o: .omoommoz oozpoI zpFoozps<.m1 ooooFoooqu FooozI "omoo moz oz mo>.ufl N:ooz so» oz :o: .zN N:oF mFFFzm me»Fooo1zoo: ooo3 .mN N:ooz so» oz :o: .zF N:oF oonFomoooo zoo: poopmoF mo3 .oF .m omou moF pooozsm Imonooo Fo mpuoomo oFFFooom monoomoo moonmoso FooFoooFoF op momooomoo poonzsmI. NE onoF 217 .mFm»Fooo Foosposopm oo mpooo zoo: Fo ooooFoo> oo mFo:o> Fo omzoF:ooz zopFEFF »Foooopxo »FFooo psz .ooozp zoo oooz zoon o FosFo poooomooo FooFF o zoms oz ozFF zozooF pF oFFz:o oF oooo .»Fpoop 1mFmooo zoos oz monp»oo oom p.oze F pozp oozp oozpo psz »Fpoomemooo »ppooo mzoon poooomooo FonFoF zoos o: ”omoommoz oozpo H zpFoozps<.11 ooooFoooxu.11 Fozoz oz mo> UH. .oozoom zpz ooF :oF mF zon: .ozoom zoN szz po pso oooo FFoooso zoo zoFoz zoo ozoom zpz m.oz Homoommoz oozpo H zpFoozps<.m1 ooooFoooxu 11. Fozoz oz moo H N:ooz so» oz :o: .zN N:oF mFFFzm mFm»Fooo1zoo: ooo3 .oN N:ooz so» oz :oz .zF N:oF oonFomouoo zoo: poopmoF mo: .oF .z omou .ooF poozzsm 11monooo Fo mpooomo oFFFooom monoomoo moonmoso FooooFoF op momooomoopooozsm11.mFo.onoF 218 .mFFFzm moF 1zoooz :oF sz Fo omsoooz pozp pooomsm sto: F psz .oozooo :oFm o mo: oz pozp mpooseoo m.oooFsoxo zoo m.oozooop oooz Homoommoz oozpo H b.2552 l ooooFoooxm Foooz 9 oz moo 1N.I .aoFoomeF zoo moF 1zooo FFoooso oo zomoz szp ”pozp oo FFo: »ppooo zoozooo 1ooo stoo oz pozp zooo op ono mo: oz pozp zpooF om zoo momoozo zoo ozoo: »oz ozp sooF oonoEooFoF zmsooo pom op ono mo: oz .mFFFzm monooo oozp ooppoz ooFmooz 1ooosoo monooz Homoommoz oozpo H »pFoozps<.11 ooooFoooxu .11 Fozoz.m1 oz mo>.m1 N:ooz so» oz :oz .zo N:oF »uoosFF monooo mo3 .oz N:oF ooFmoozoooEoo monooo mo: .zm .om N:ooz so» oz :oz .oooonooo--.oFo oFooF APPENDIX E EXAMINER'S NOTES FORM AND LETTER TO SUBJECTS 219 220 EXAMINER'S NOTES Clinician's Name: Case Name: Date: Time Session Began: Time Session Ended: Total Time of Session: Time Initial Contact Began: Time Initial Contact Ended: Total Time Initial Contact: Time of First Cue Request: Time Cue Interaction Ended: Total Time on Cues: Time Written Dx Began: Time Written Dx Finished: Total Time for Written Dx: Time Rx Began: Time Rx Finished: Total Time Rx: Time Dx Transfer Began: Time Dx Transfer Ended: Total Time for Dx Transfer: Time Hx/Ob Check List Began: Time Hx/Ob Check List Completed: Total Time on Hx/Ob Check List: File No. Allowed Cue Time Began: + 25 Remind: + 20 End: Allowed Dx Time Began: + 25 End: Allowed Rx Time Began: + 25 End: 221 Clinician: Case Name: Date: Problems with session: Clinician's reactions: Other: 222 MICHIGAN STAT}: L NIVERSI'I‘Y I\\Illl ll Hm K;\1\H1H11\ ll \1 HIM. .\N1 1\\\1\1. ~ \111 11.. \\ ~1uj. Iillllhlt"1111‘\ll-'\-|Hl(k\ll\ll\ll November Dear 29, 1978 Thank you again for agreeing to be a clinician in the reading research project for the Institute for Research on Teaching (IRT). As I discussed with you at our final session, I need to have you transfer each of your written diagnoses to the Diagnostic Check List for a second time. The purpose of having you repeat this task is to check the reliability of the Diagnostic Check List. Enclosed are copies of your three written diagnoses along with three Diagnostic Check Lists and a stimulated recall sheet. When you have completed all three transfers of your diagnoses to the check lists, please place all of the materials you were sent in the enclosed envelope and mail then promptly. Please follow these instructions: 1. Please do not rush with this task. You may want to work on each diagncsis on a different day. Begin with your written diagnosis for case number 1, . Do not look at the other diagnoses. Locate and circle all of the diagnostic or reading factor statements in your written diagnosis. (Note: A diagnostic statement is the identification of a factor or variable which helps determine the state or condition of a student's reading performance. It may be one sentence or several and it may also just be part of a sentence. In addition, the diagnostic statement may be a strength, a weakness or just an observation.) Circle and number the diagnostic statements or factors. Locate on the cover sheet of the Diagnostic Check List the main category into which your diagnostic statement woold most likely fit. 223 Page 2 November 29, 1978 (the: Space is provided within each main category for "Other Statements". You may copy any statements from the written diagnosis under "Other Statements" if you are able to determine the main category but are unable to find a matching statement on the check list in that same category. However, statements in the written diagnosis for which you can not determine a category should be copied under the main category "Other" in the check list. Please make as limited use as possible of the statements and the category called "Other".) 6. Find that main category in the check list and locate a statement within it which best matches your circled statement. 7. Decide if your statement indicates a strength, a weakness or is just an observation which you made; then place the number of your circled statement under one of the column headings. 8. If a stimulated recall sheet is included with your materials, circle the diagnostic statements in the stimulated recall as you did in the written diagnosis but use letters A, B, C, etc. rather than numbers to designate each statement. Next transfer the letters to the Diagnostic Check List as you did for the statements in the written diagnosis. 9. Check to make sure that you did not omit any statements from the check list. 10. Repeat procedures 2-8 for case number 2, . 11. Repeat procedures 2-8 for case number 3, . 12. Date each check list for the date you complete it. 13. Please be prompt in completing the check lists and in returning them. In the interest of protecting the work of the IRT, please do not discuss the nature of the tasks you have been doing with others in the field of reading. You should be receiving a check very soon for your work on this project. If you do not, please call me collect at (517) 332-2351. On behalf of the Institute for Research on Teaching, thank you for participating in this research project. Yours very truly, / . ~4/ \ 4”“. l a... . ”’4 {#/ '. Lynne Hoffmeyer ’ ' Enclosures: College of Education 3 Diagnostic Check Lists Michigan State University 3 Diagnoses 3 Stimulated Recalls (if you used them) 1 Envelope for return mailing APPENDIX F CUE DOMAIN SAMPLE FOR A CASE 224 \DmNOlU‘l-fiCUN-J BKG BKG BKG BKG DOL DOL DOL DOL DOL DUR DUR DUR DUR DUR DUR DUR DUR DUR DUR DUR DUR DUR DUR DUR DUR DUR DUR DUR DUR DUR DUR DUR DUR DUR DUR DUR DUR DUR EKW EKN EKN EKN GMG GMG GMG GMG GMG 04> SOCDNUl-th-HU‘I-th-‘N-HCDN xooouwm—atan-a 225 CUE DOMAIN SAMPLE FOR A CASE Background Information--Biographical Data--EC Background Information--Physical/Health--EC Background Information-- /Family--EC Background Information--Classroom Information--EC Dolch Basic Sight Vocabulary--TS Dolch Basic Sight Vocabulary--EC Dolch Basic Sight Vocabulary--TB Dolch Basic Sight Vocabulary--AR Dolch Basic Sight Vocabulary--TD Durrell Oral Reading--TS Durrell Oral Reading--EC Durrell Oral Reading--TB Durrell Oral Reading--AR Durrell Oral Reading--TD Durrell Silent Reading--TS Durrell Silent Reading--EC Durrell Silent Reading--TB Durrell Silent Reading--TD Durrell Listening--TS Durrell Listening—-EC Durrell Listening--TB Durrell Listening--TD Durrell Word Recognition and Word Analysis--TS Durrell Word Recognition and Word Analysis-~EC Durrell Word Recognition and Word Analysis--TB Durrell Word Recognition and Word Analysis--AR Durrell Word Recognition and Word Analysis--TD Durrell Visual Memory of Words-Primary--TS Durrell Visual Memory of Words-Primary--EC Durrell Visual Memory of Words-Primary--TB Durrell Visual Memory of Words-Primary--AR Durrell Hearing Sounds in Words--TD Durrell Hearing Sounds in Words--EC Durrell Hearing Sounds in Words--TB Durrell Hearing Sounds in Words--TD Durrell Sounds of Letters--TS Durrell Sounds of Letters--EC Durrell Sounds of Letters--TB Ekwall Phonics Survey--TS Ekwall Phonics Survey--EC Ekwall Phonics Survey--TB Ekwall Phonics Survey--TD Gates-MacGinitie-Vocabulary--TS Gates-MacGinitie-Vocabulary--EC Gates-MacGinitie-Vocabulary--TB Gates-MacGinitie-Comprehension--TS Gates-MacGinitie-Comprehension--EC Gates-MacGinitie-Comprehension--TB 226 49 ENG 13 Gates-MacGinitie-Speec & Accuracy--TS 50 GMG 15 Gates-MacGinitie-Speed & Accuracy--TB 51 GMK l Gates-McKillup Recognizing & Blending Common Word Parts--TS 52 GMK 2 Gates-McKillup Recognizing & Blending Common Word Parts--EC 53 GMK 3 Gates-McKillup Recognizing & Blending Common Word Parts—-TB 54 GMK 4 Gates-McKillup Recognizing & Blending Common Word Parts--AR 55 GMK 5 Gates-McKillup Recognizing & Blending Common Word Parts--TD 56 INF 2 Informal Oral Reading--EC 57 INF 3 Informal Oral Reading--TB 58 INF 4 Informal Oral Reading--AR 59 INF 5 Informal Oral Reading--TD 60 IRA 3 Individual Reading Analysis--TB 61 IRA 5 Individual Reading Analysis--TD 62 PEA l Peabody-Reading Recognition--TS 63 PEA 2 Peabody-Reading Recognition--EC 64 PEA 3 Peabody-Reading Recognition--TB 65 PEA 5 Peabody-Reading Recognition--TD 66 PEA 7 Peabody-Reading Comprehension--TS 67 PEA 8 Peabody-Reading Comprehension--EC 68 PEA 9 Peabody-Reading Comprehension--TB Weschler-Full Scale--TB Weschler-Verbal Scale--TS Weschler-Verbal Scale--EC 84 WISC ll Weschler-Verbal Scale--TD 85 WISC 13 Weschler-Performance Scale--TS 86 WISC 14 Weschler-Performance Scale--EC 87 NISC l7 Weschler-Performance Scale--TD 69 PEA 13 Peabody-Spelling--TS 7O PEA l4 Peabody-Spelling--EC 71 PEA l5 Peabody-Spelling--TB 72 PEA 19 Peabody-General Information--TS 73 PEA 21 Peabody-General Information--TB 74 SORT l Slosson Oral Reading Test--TS 75 SORT 2 Slosson Oral Reading Test--EC 76 SORT 3 Slosson Oral Reading Test--TB 77 SORT 4 Slosson Oral Reading Test--AR 78 SORT 5 Slosson Oral Reading Test-~10 79 WISC l Weschler-Full Scale--TS 80 WISC 2 Weschler-Full Scale-~EC 3 7 8 Key: T5 = Test Scores AR = Audio Recording EC = Examiner's Comments TD = Test Directions TB = Test Booklet APPENDIX G PROCEDURES FOR TWO STUDIES: OS '77 and OS '78.3 227 228 Table Gl.--Procedures for two studies of diagnostic decision making using simulated cases and eight clinicians. OS '77 OS '78.3 . Instructions and practice with sample simulated case. . Initial contact information given on simulated case. . Observational session directed by experimenter and recorded by a clinical observer and tape recording. Subjects col- lected data from "boxed" sim- ulated case using inventory of available data. Subjects were encouraged to verbalize think— ing. . Subjects prepared written diag- nostic and remedial report. . There was a debriefing session, directed by a clinical observer and aided by an experimenter, in which the subjects underwent stimulated recall. Each data item that had been requested was presented along with con- trolled interview questions: "Nhy did you request this? What did it tell you?" . Written diagnosis transferred to a diagnostic check list by independent judge agreement. 05 . Instructions and practice with sample simulated case. . Initial contact information given on simulated case. . Observational session directed and recorded by experimenter only and tape recording. Sub— jects collected data from "boxed" simulated case with gg_ inventory of available data. Subjects were ggt_instructed to verbalize thinking. . Subjects prepared written diag- nostic and remedial report. . Subjects were asked to trans- fer written diagnosis to standardized Reading Diagnos- tic Check List. . There was a debriefing session, in which the subjects underwent stimulated recall. Each data item that had been requested was listed alon with written check list: "(1 Did you ask for this information because ...you had a hunch? ...you just wanted this information? ...you usually get this information? ...other?" and "(2) Did this information ...confirm a hunch? ...disconfirm a hunch? ...suggest a hunch? ...other?" APPENDIX H TABLES 229 230 mcwucoomm o_c:< u m< mucmseou m.cmcm5mxm u om mcowuomgwa 9mm» u e. amt—com ummh u m._. mmsoum “hm... u m... 359. mm H 2 made sas=u< stooau=< m a=< om. am Amsv sne=u< _a=m_> m ma> om. mm Ammo muemstoceaa awesome: m. owe: as. we Away _aata> empeummz a ems: mm. mm Ammo «_aom Pass em_;omm3 P ems: om. Na Ashe seoseu=< eases: m am: om. om Amsv scep_aam m mam as. as make assumes _aeo commopm m meow mm. so made eoesmueaa__sm memspm=< m=_uamm _asua>_uca m_ euca a <¢H mm. am Amps maxecesm ceases mims_a=< sesammm Pasae>_uea m age as. _m Amsv :o_memeatasoo oeseeeaomz-mmnaw a use as. me Ammo stmpzaauo> menaceauaz-mmsaa _ use as. as Amps were: as seesaw septum: F_aet=o mm man as. am Ashe memspa=< use: new eoesecmouam are: __mttso Fm ass mm. mm Ashe :oemcmemtaeou assessmeo F_aet=a me «so mm. mm Aumv coamcmemtasou mcsemomaa spartan s_ mag om. em Amev accummm osmium F_aet=o a ass as. _N Amaesu< uaNPULaecasm om .mxm om. m mums sarcasm :o_sa=_a>m _easa ON .mxm cm. a Acme arouse Possum a, .mxm mm. a m=Fm> amassz mamz mag copucoaoca osu .N ammo .mmzo co ucmsmmcmm mo :owpconoea--._: m—aMF 231 mcgucoumm omua< u m< gucwEEoo g.em:msgxm u um meowpomewo amok u oh umpeoom umwp u mp gmeoum page u me name gN u z imam mgesgg eggmmm m gem gm. Nee Agme mgeggg masseuse N gee gm. gem imam ggggELoeegm eg_egmgz mm egg: mg. g._ mmme mgseg> egmegmgz N mmez Ng. mom mmee mggcggggm ggemmgmegm m meg gm. mm Amee game gee: ggugeg gomgomg m emog mm. mm mmee mgmmmezgegm m 24: gm. gg mmme memo: gaggmgoz gammmegz-mgegg mm ezg gm. mg mmme ages: gaggmgoz a=_mmegz-mgggg m eeg gm. mm mmgm merge ego: gaseog ggmgggem agemmegz-mgggg m ezg gm. mm emhe gommggegegsog gmgmgmgggz-ggggg m geg mg. Ne mmmm gommggegeaeog gagegmggaz-gggag N gzg mg. mg mmpe megmgmmgo> gmgmgmgggz-gggmg _ gzg mg. mg imam memmmgam segregg mm mmm mm. em emme gggmggeegggm were: me meosgz mggmm> _mgeesg NN egg mg. gm mmse goegggegegeog _geg __geegg m. egg mm. NN mmem ggmgggm segmmm Pageegm m meg mg. g. Agme ggmgggm eggmmm Pageegg m egg gm. N_ mmgmeg< ggNPgegggggm NN gem gm. g— Amhm mgeogm page gggeg>gmeg< egngeggggmm mN gem gm. m mgmm mggegoem .mmgm gggoegeugm gN gem gm. m mgme accuse mooegg ggsoegeggm gm gem gm. N mgmm atom egegggm ggsoegeggm m gem mg. g mgme atom gggega gcgoemeggm N gem mm. m m: —m> emnszz msgz mzu :omucogoea mag .m mggu .mmmm.:o mausomcmg do compeoaoeauu.mx spam» 232 memueoomm owuo< u m< mucosEou g.emcwsgxm u om geompumewo ammo u no um_moom ago» u mp mmeoom Home u we ”mom oN u z mmhe some oommmo g moo om. No, home gmgom _mom egmmgggo om mmmo om. No Ammo goggm __om eg_oomg3 mm mmmm mm. mm Ammo ooogseogegm tomogmgo N mom: oo.m om Ammo momego tomoomgm _ mgmm oo._ No Ammo ammo ooo: ogogeg ooeoogm Fooo oommomm m pmom mg. mg mmoo moooog mogogo oom>mg mommmeoz-mgogg mm ezg om. om Ammo oomoog_m meoomoom oo__meoz-moggg o ezg om. mg Ammo goeom ooo: ooesoo oomoogmm oog oomNmoooogm oommmeoz-momgg m ezg mg. oo Ammo _ ammo oommmgmm geomogoeogom mmgeeoo om moo om. mo Ammo megomem moeoz go meoEgz _gomeo _mgeeoo mm moo mm. _o Ammo sesamem moeo: om moooom oomeggm e.geeoo NN moo om. mm mmee mmmmmgom ooo: oog oomomoooogm ooo: emoteoo .N moo oo._ Nm mmhv oomoggm moo_mg __geeoo o moo oo.m mN Ammo oomoggm _oeo __geeoo o moo om. om Ammo oomoogm _geo Pagetoo m moo oo.m mm Ammo some oeoo megmoogooo :o_oo m moo mm. om mmoe oeoogm mooegm oomooseogom ooooeoeogm m_ gem om. m Amhv scam cogugmh comumecoecm ecoocmmogm m gem om. N Ammo smog mooegm ooemgseoeom ooooeoeogm m gem om. m wopo> eonsoz msgz moo compeoooem moo .o mmgu mwmmw co ucmswmemg we :owmeoooeouu.mz spam» .III'IOI'I.I'OII'IIIII'OVOI'OI'I ego mg Ame mm Ame mm momo mo_ Ammo mom 1": III, II":'I'IIII'I""""'|.|'I'.I-.Il""""III'II". 'l'tII'I'I- -10 -...lInIO ’OIII’I-1-"- Il'li'IIO Ammo mm. eNme meg moe eguoo o Ame mm Ame mg ego mm mNee mom Ame mm egNe mNN Ammo mmo m__ monoe m 3 ”m ; m_e me Ame mm moo mom Ame mm mom moe emem moN Ammo mNe om eguom N Ame mm Ame mm Ame mg ego mm meme mm. egem mm. Agog mNm mg eguom _ mo.eu.m.mm emm.n.o.mo mNg.u.m.mo mom.n.m.mm emm.u.m.mo mm..n.m.mo mo.u.m.mm mogmomommg g someomom_o m moomoeoeeo o mogmomom_o m someoeommo N omegmommg e omega—_oo gmooeeo>m tgoooz mo ogoog__og mo ogoggmmoo mo ogoog._oo mo ogoogm_oo mo ogooge_oo mo oggog__og .mmz mgog mgoo gmgg gmou mo gmou mo ggou mo gmou mo gmou mo gmou mo mo cozv_mm page» fozvmm fozemm 72:: 73:2. 72:3. fozegm -..u III! .gmggo coo» com gmoo co A.<.ov acmemmcmg mo :o_ueoooco _guou cm mmmgucooemo--.e: m—ao» 234 IIIDIOIIttoIluvIIIIIIIIIIIVIIII-IOIIIC.O.O.IO.I.IIIIIOII.OI.II‘.!IIIOIIIII.I¢I.I.III.IIIIO!IIIIIIIIIII.II.O|IDIIII..OaIIIII.I!.I!V0.OII moo mo Ame mo ANV mo Ame Ne mmNe m eNme em. moome gem mom egumo o moo mo mom mo Ame No moo m. mmNm mm m_ov mom mmmme mmm mom mgumo m mom mo moo mo mNe mo Ame m_ Ammo mo Ammo mN_ Ammoo eNm mom mowme N moo mo Ame mo mom m ego mm emee mm ANNV mom m__oo meg mom mgume _ mo._u.m.me mmm.n.<.me emg.u.m.mo mom.".<.me emm.n.<.me em_.u.m.me mo.n.<.me game eogeo m:g_o_:_Fu m geomomom_u m mcgmum:__u o geomo_:m_u m g:g_omcw_u m cowomcwpu _ cocmwmcmz . co em a: mo oooommogz mo ogoomoogo mo ogoomgogo mo oooomeogz mo ogoomoogm mo ogoomoogz goz mgogog om gmg z gucmsmugmm guemsmpgom geomemuoum mucmsmogum mucoemugum mucmsgomum mucmsmugum u m u moeoze m moeoze: Booze: cofoze: morozemm morozemm eomozemm :38 ll ll II I I..I.II.IIIIIIII,IIIIIn [IIIIII.IOIIII'nIII.IIlI.II.II.IlIIIII IIIIOII IIIIIIltIIIri IzIIllI I III. n III. I. III.I I I I I III I 1 . O II.I I o .l ....I: . I IIIII.I.I IqI.I.IVI,I I.III:I I .gmggo too» to» geomEmmgom omemommmwu co A.<.mv oomsmweam mo :o_oeoooeo _gHOH cm gmmgpcmoemo--.m: m_ngo 235 Table H6.--Commonality of agreement on cues collected in six diagnoses on each of four simulated reading cases. Commonality Frequency TROW Case 1 Case 2 Case 3 Case 4 otals .10-.20 0 0 0 0 0 .2I-.30 0 I 0 0 I .3I-.40 0 0 I 0 I .4I-.50 0 0 0 0 0 .5I-.60 I 0 0 I 2 .5I-.70 I 2 0 I 4 .7I-.80 4 0 3 I 8 .8I-.90 0 2 I 3 6 .9I-I.0 0 I I 0 2 I1” 6 6 6 6 g: .71 .74 .77 .77 Range = .48/.82 .30/.93 .39/.95 .62/.36 SD = .I3 .24 .20 .II Mean total = .75 §Q_= .03 236 Table H7.--Commonality of agreement on diagnostic statements made in six diagnoses on each of four simulated reading cases. . Fre uenc ROW Commonality Case 1 Case 2 q c:se 3 Case 4 Totals .lO-.20 O O 0 O O .21-.30 O l l O 2 .31-.40 l O 2 l 4 .41-.50 2 l O O 3 .51-.6O l 2 1 l 5 .61-.7O O 2 2 3 7 .71-.80 2 O O 1 3 Above .80 O O O O O g_= 6 6 6 6 M_= .55 .54 .47 .62 Range = .37/.76 .25/.66 .28/.64 .42/.78 §Q_= .16 .15 .15 .12 Mean total = .55 §Q_= .05 237 Table H8.--Inter/intrac1inician agreement (Phi and Porter) on cues collected on four simulated reading cases. Frequency Row Case 1 Case 2 Case 3 Case 4 Totals Phi Coefficient Negative values 1 l O O 2 .00-.20 4 4 2 1 ll .21-.30 4 l 3 l 9 .31-.40 6 l 4 7 18 .41-.50 O 4 4 4 12 .51-.6O O 3 1 2 6 Above .60 O 1 1 O 2 fl_= 15 15 15 15 M_= .30 .35 .38 .42 Range = -.O4/.55 -.18/.87 .04/.63 .21/.60 §Q_= .18 .28 .15 .ll Mean total = .36 §Q_= .05 Porter Index Negative values 0 O O O O .00-.20 5 5 3 O 13 .21-.30 5 O 5 4 l4 .31-.40 2 4 5 8 l9 .41-.50 3 5 l 3 12 .51-.60 O O 1 O 1 Above .60 O l O O 1 g_= 15 15 15 l5 M_= .28 .36 .33 .37 Range = .09/.46 .05/.83 .09/.54 .23/.52 §Q_= .12 .20 .12 .08 Mean total .34 _s_p_ .04 238 Table H9.--Inter/intrac1inician agreement (Phi and Porter) on diagnostic statements made on four simulated reading cases. Frequency Row Case 1 Case 2 Case 3 Case 4 Totals Phi Coefficient Negative values 0 1 2 O 3 3 .00-.20 13 13 12 13 50 50 .21-.30 2 O 1 2 5 5 .31-.40 O O O O O O .41-.50 O 1 O O 1 1 Above .50 O O O O O O n = 15 15 15 15 E .10 .12 .07 .16 Range .00/.28 -.03/.43 -.05/.26 .04/.31 §Q_= .08 .11 .08 .07 Mean total = .11 §Q_= .04 Porter Index Negative values 0 O O O O ..OO-.20 15 15 15 15 60 Above .20 O O O O O n = 15 15 15 15 E = .07 .O9 .O6 .11 Range = .02/.17 .00/.29 .00/.16 .05/.20 §Q_= .05 .07 .04 .05 Mean total = .08 §Q_= .02 239 Table HlO.--Correlation coefficient for Egg_times, including missing data, for Case 1. Subject Run Value Number Number of‘g .03 l .427 103 3 -.016 107 l .415 107 3 .232 108 2 .401 102 2 .364 Mean total = .30 §Q_= .17 Table H11.--Corre1ation coefficient for ggg_times, including missing data, for Case 2. Subject Run Value Number Number of.r 102 1 .390 102 3 .374 108 1 .461 108 3 .401 104 2 .260 106 2 .051 Mean total = .32 §Q_= .15 240 Table H12.--Corre1ation coefficient for Egg_times, including missing data, for Case 3. Subject Run Value Number Number of‘: 101 1 .234 101 3 .715 105 l .287 105 3 .404 103 2 .263 107 2 .266 Mean total = .36 SQ-= .18 Table H13.--Corre1ation coefficient for gyg_times, including missing data, for Case 4. Subject Run Value Number Number of.£ 104 1 .591 104 3 .441 106 1 .470 106 3 .223 101 2 .560 105 2 .439 Mean total = .45 §Q_= .13 241 Table Hl4.--Corre1ation coefficient for diagnostic statement times, including missing data, for Case 1. Subject Run Value Number Number of r“ 103 l .200 103 3 .144 107 1 .190 107 3 .317 108 2 .231 102 2 .132 Mean total = .20 §Q_= .07 Table H15.--Corre1ation coefficient for diagnostic statement times, including missing data, for Case 2. Subject Run Value Number Number of.£ 102 l .198 102 3 .227 108 1 .317 108 3 .292 104 2 .142 106 2 .303 Mean total = .25 .07 _sg 242 Table H16.--Correlation coefficient for diagnostic statement times, including missing data, for Case 3. Subject Run Value Number Number of“: 101 1 .133 101 3 .156 105 1 .213 105 3 .322 103 2 .156 107 2 .320 Mean total = .22 §Q_= .09 Table H17.--Corre1ation coefficient for diagnostic statement times, including missing data, for Case 4. Subject Run Value Number Number of‘g 104 l .199 104 3 .248 106 1 .296 106 3 .283 101 2 .339 105 2 .404 Mean total = .30 52 .07 24E! mm.o mg.N mo.m_ __.__ NN.N o.o o.o o.o om.m ooo:: oommgooom g goo meeoomo mm+meoe mm.o mm.N mo.N. __._m NN.N o.o o.o o.o om.m ooo:: ogaemeooo m egg m__oomo mm+mome o... go. mm.N mm.N o.o o.o o.o o.o o.o ooo:: ooomgooom g .oeom mm+NeNe oo.m oo.o mm.oN o.o mm.o o.o o.o oo.oN o.o ogatmeooo g .oeom ogooo: mm+Ne__ _m.e Ng. om.m o.o o.o o.o o.o o.o om.m ogotmeooogmo g oooom mg+eeo_ mm.m gm.._ o_.mm gN.o o.o Nm.m_ om.N_ NN.NN oo.m ogstmeoog g oooom mm+_eo mm.m mm._ oo.oN o.o o.o mg.o o.o o.o mm._ romeo m Nm.g gm.o_ mm.mm om._N .N._N _o.m mm.mm oo.o gN.o zooo: mmgooom N mN.N N_.N _N.N_ o.o o.o om.m o.o NN.N oN.g coco: otmeoogmmo g mm.m Ng.mm oo.oo go.m_ oo.mN _m.om om.o_ mg.__ om.o ooo:: someoog m om.N NN._ og.N mm._ o.o o.o mN.g o.o o.o romeo o mo.o mo.m_ om.mm om._N NN.NN _o.m mN.g Ng.g gN.o .oeoe mag m__gomo m mg.m_ mg.m_ oo.oo o.o m_.m_ om.mN mm.mN NN.NN oN.m .oeom ooooom mmoo N No.m mm.m oo.oo oN.m NN.N _m.o_ m_.o gm.m om.m_ ooo:: m ow 2 ENE N oom N com m com _ com m coo NS—uwfiwoom 3520522. 32.52 3om go. gogmoom oom gogooom mo. ogoooom wo— uomnnzm NoF uumnnam II .II 1.. r; In 11.... I,I¢ IIII IOIoI.I.. II.IIIIIIOI I:vIIh.IIVIsI.II|IIIIIIOII".IIII .N 33:33 toe ugeomoog 3:95.523 58 32.5 No cacao: mo mogecmucma mo mmocméimg ~22. 241I IIIIII I O II.I.III III-IIIlIIII II.III..II. IIIII.I IIIIIII mg.m mm.m oo.NN o.o mN.o_ o.o NN._ mo.e o.o ooo:: oggmoooom m egg m__oomo em+meoe NN.N mg._ oo.oe o.o o.o Nm._ NN._ o.o o.o moose oooeeeooo g gag m__oomo em+meme o.o o.o o.o o.o o.o o.o o.o o.o o.o oooom ogomgooog g .oeom mN+NmNm _o.m ...N mg.N_ gN.o o.o om.m o.o o.o o.o ogotmeoog g .oeom omega: em+Ne_. o.o o.o o.o o.o o.o o.o o.o o.o o.o ogEtmeooooeo g ooo:: eg+_eo. mm.m_ om.eN gN.oN_ gm.o mm.m mm.mm No.N_ oo.oN oo.oo ogotmeoog g egoom mm+mmo _m.m mN.N mm.gm mm.m o.o o.o mo.m mm.o o.o toemo g og.N_ mm.Ne mg.gm gm.o oo.oN mo.N mo.m No.o mm.mm oooom omgooom N Nm.o m_.o NN.oN Nm.o mm.m o.o mo.o o.o o.o ooo:: ateeoooaeo g mm... mo.m_ oo.omm oo.oN Nm.o om.o_ oN.o oo.mm mm.m oooom stmmoog m mm.e m_._ mo.m o.o o.o o.o mo.m Ng.m o.o toomo o mm.m _g.o mg.mm No.NN om.__ oo.e_ No.N_ o.o o.o .oeom egg m__oomo m mo.o mg.m _o.om mo.m mm.m oo.o_ me.m om.m o.o .oeom ogmooo omoo N og.g mm.m o_.om o.o mm.m_ mo.m mo.g_ gN.m gm.m oooom . on o _MMMN No_Nm”wwoom momNoummoom mommonmwo mommmmmwoog _o_mguwwoom Poe—onwwoom geomoomoo_om rowan: I.IinlII IIIII. IIIII. III! ..III III- I M II I...I III 3 Ii” ”11.31: .10 I.M<14szES 24J' I .l“'l.|.l'|tllyll..l.10.l.I "|l' I-Iu.||lll|l‘la’i 911.11 Ill'ol'.i;l. Il‘l.1|l 3_.3 33.N N3.3_ 3.3 _N.3_ 3.3 33.N 33._ 3.3 53:3: 333333333 3 333 3_—3333 AN+333. .3. N3. _N.N 3.3 3.3 3N.3 33._ 3.3 3.3 :3:3: 33::_3:33 3 333 3,—3333 33+333_ 3.3 3.3 3.3 3.3 3.3 3.3 3.3 3.3 3.3 :3:3: 333333333 3 .33:3 AN+N3N_ 33._ 33. 3N.3 3N.m 3.3 33.. 3.3 3.3 3.3 33::_3:33 3 .33:_ 333:3: A3+Nv__ 3.3 3.3 3.3 3.3 3.3 3.3 3.3 3.3 3.3 33:33::33333 3 53:3: A3+_333 33.3_ 33.N_ 3N.33_ 33.. 33.3 3_.N3 33.NN 3N.3N 33.N3 33:333:33 3 :3:3: A3+_33 N3.N 33.N 3N.N_ 33.3 3.3 3.3 N3.3 NN.3 3.3 33533 3 33.N .3.3 33.33 3N.m 33.3_ 33.. N3.3 N3.3 33._N 53:3: 3333333 N .3.3 _3.3 33.33 33.3 _N.3. 3.3 33.3. 3.3 3.3 :3:3: 3333:33333 3 33.3, 3N.3N 33.333 3_.N3 _N.3N 33.3N 33... 33.33 3_.N 53:3: 3:33:33 3 _3.N 33._ 3N.3 3.3 3.3 3.3 _N.3 33.3 3.3 53533 3 .3.3 3_.3 33.33 3_.3N N3.3_ 33.3 33.3 3.3 3.3 .33:_ 333 3.33333 3 N3.3 33.3 3N.33 33.3 3..N _3.3_ 33.. 33.3 3.3 .33:3 333:3: 3333 N 3_.3_ __.3_ 33.33 3.3 33.3, NN.FF 3N.3N 33.3. N3.3N 33:3: _ 33 : _MWMH N3_N3”wm333 33_N3uwma33 33333mww333 33.33umw333 33,33HMM333 _3_p3HHM333 3_53:3333_3: :www“: 'fl.lvl‘,1-l I.C"VUIQ‘1"I“. ‘-|.‘|D"n I -.‘L'tltli .m 3333--33:3533333 33333: "unfit u u u. :1.‘Pu.g 4£| ...- HI. I V I. '1' {'7' ? ’A'r",.ll ,' ”in! ’I‘kfl“ll I-EI.~F~I'N'N‘I 1 ”iii.“ "uln‘ :‘bll- ’rll E m333 no» 33333333 npgmcomuopmg 3333 mms_u 33 33323: 33 3333333333“:333333><--.NN: 3—333 ‘I 0 n ' ..-...“I."-y|,ll ' I .IIIIO'II.‘ 3 O ...--.n, 24%! N3.. 33.. 33.3 3..3 3.3 3.3 N3.. 3.3 33.3 53:3: 333333333 3 333 3..:333 .N+333. 3... 33. 33.3 3.3 3.3 3.3 .3.. 3.3 33.3 33:3: 33:33.:33 3 333 3..:333 .3+333. 33.. 33. 33.3 33.. 3.3 3.3 3.3 33.N 3.3 53:3: 333333333 3 .335. .N+N3N. 33.3 .3.N .3.3. N3.. 3.3 3.3 N3.N 3.3 N3.3 3323.3:33 3 .33:. 333:3: .3+N3.. 3.3 3.3 3.3 3.3 3.3 3.3 3.3 3.3 3.3 3333.3:333.3 3 53:3: .3+.33. 33.3 33.. 33.33 3.... N3.N. 3N.3 33.3 33.N 33.3 3333.3:33 3 53:3: .3+.33 3N.N N3.. 33.3 3.3 N3.. 33.3 3.3 3.3 3.3 33:33 3 N3.N 33.N. 3N.33. N3.3. 33.3N 33.3. 3..3. 33..N N3.3 53:3: 3333333 N 3N.3 33.3 33..N 3.3 N3.3 3N.3 33.3 3.3 3.3 53:3: E:.3:333.3 3 33.3 3N.3N 33.33. 33.3N 33.NN .3.NN 3..3. 33.3N 33..N 53:3: 3:33:33 3 N3. 3.. 3N.3 3N.3 3.3 3.3 3.3 3.3 3.3 33:33 3 N3.N. 33.3N 33.3N. 33.3 33.3 33.33 33.33 3N.3. 33.33 .:3:. 333 3..:333 3 .3.3 N..3 33.33 33.3. 3.3 33.N .3.3. N3... 33.3 .33:. 333:3: 3333 N 33.3 3N.3 3..33 3N.3 3...N .3.. 33.3 .3.3 33.3. :3:3: . 33 : .mwm. 33.N3umwn33 .3.N3HMW333 33.33MMW333 33..3”mw333 33.33umw333 33..3MMM333 3.53:3.33.3: :wwumz ..lvhl.hu'v.P.||Al.h'n'nnrrn.nl n‘fl’ lllllllllllllllllllllllllllllllll ”IIHPHNVHHUD I.!I ..u .‘v unI‘szhfiflrn.flulhvfl.wlh"WC.” .3 335-333.353.333 3333333333.? .333 33333333 33.33.553.33 .3333 33.3.5 3.3 333.33.. 33 3333333333 b.3 333333.16? 3.33 33333530 .33 .33 .33 .33 33 .3 33 33 33. 3.. 33 .3 3. 3. 33 33 :33333 :3 3:33. 33 .3: 33. 33. ... 33. .3. 3.. .3. 33. ... 33. 33. 3.. 33. .3. ... 33. .mw 3.. 3.. 33. ... 33. 33. 33. .3. 33. 33. 33. 33. 33. 33. 33. 33. .a ..mpgoav 03 3.3:. a. 2 33. 33. .3. 33. .3. ... 33. .3. 33. 3.. 33. 3.. .3. ... 3.. 3.. .mw ... 33. 33. .3. 3.. ... 3.. 3.. .3. 33. 33. 33. 33. 33. 33. 33. .m ..mugoav 3.3:3\gmuc. 3 3 3 . 3 3 3 . 3 3 3 . 3 3 3 . “33333 3:333 3.3.. 33 33333 ... 33 33333 3.3.. 33 33333 ... 33 3.:323.<.3 3.333z3<.3 3333 .3333333 ozp Low .mcvmson :3 3333533333 3333033333 3:3 3333 33 .3323: 33.33.333 .333333333 333.33 :3 3333333>mu 33333333 new camev 3333 3333 .3333>wu=.uu.3~: 3.333 APPENDIX I OBSERVATIONAL STUDY DATA-ANALYSIS SYSTEM (OSDAS) STATISTICS 250 251 OBSERVATIONAL STUDY DATA-ANALYSIS SYSTEM (OSDAS) STATISTICS The following statistics are all calculated by (l) a computer statistical-analysis system (Observational Study Data-Analysis System) developed and maintained by the Institute for Research on Teaching, Michigan State University (Wagner et al., l979) and (2) an inter- active statistical-analysis system (MIDAS) at the University of Michi- gan. A random sampling of data was computed by hand across numerous observational studies, and the system was found to be operating accu- rately. Some portions of the data, such as mean and standard devia- tion, were calculated with a Litronix 2270R hand calculator. Proportional Agreement Given a domain for cues or diagnostic statements (or remedia- tions) for a given case, proportional agreement is the proportion of clinicians who mentioned each cue or diagnostic statement. One pro- portion is computed for each cue or diagnostic statement in the domain. The statistic is bounded by 0 and l and is calculated by determining the number of clinicians who collected the same cues or made the same diagnostic statements for a given case. That number is divided by the total number of clinicians who interacted with the case. The resulting statistic indicates the proportion of clinicians who collected the same cues or made the same diagnostic statements for a specific case. The calculation is for the most frequently 252 collected cues and most frequently made diagnostic statements. The formal computation is as follows: C =_J_i P.A. C- J where Cji = number of clinicians mentioning the ith category Cj = total number of clinicians for a given case. For example, if two clinicians, of a total of six diagnosing the same case, mentioned the diagnostic statement "Contextual Word Recognition--Weakness," the proportional agreement would be: P A = no. of clinicians mentioning cue (statement) ° ’ total number of clinicians P.A. ll own: II (A) (JO Commonality Given a domain for cues or diagnostic statements for a given case, the commonality statistic is a measure of agreement between one clinical session and all other clinical sessions for a given case; e.g., an individual is compared with a group. The statistic is bounded by 0 and l, and only the proportional- agreement statistic is used in calculating an individual's score. A value of x_for a specific clinician implies that he has collected in his session, for a given case, roughly x_percent of those cues most frequently collected by the group for that same case. If, for example, a clinician has a commonality score of .47 it means that the clinician has collected in his session, for a given case, roughly 47 percent of 253 those cues most frequently collected by the group for the same case. The same analysis can be applied to diagnostic statements. Interclinician Agreement One of the indices used to describe interclinician agreement was the Phi coefficient. The Phi, denoted by ¢, is the traditional Pearson product-moment coefficient of correlation for nominal dichotomous data with no assumptions concerning the shape of the dis- tribution of scores. One Phi coefficient was computed for each pair of clinicians. To compute the interclinician correlation, two requirements were met. First, a domain of statements was defined. (See Appen- dix C.) Second, a determination was made as to which diagnostic statements were present in a diagnosis or absent from a diagnosis. When two clinicians were compared, for convenience, the data could be tabulated in 2 x 2 contingency tables showing the joint occur- rences of pairs of scores (+ 1) using frequencies. For this study, the cues (Cx) or diagnostic statements (Dx) mentioned by one clini- cian were compared with those mentioned by a second clinician for the same case. This comparison is illustrated in Figure Il. The calculation of o is derived from the contingency table as follows: (ad - bc) / (a+c)(b+d)(c+d)(a+b) Phi = (See Figure 12.) 254 Clinician A, SIMCASE Y PRESENT (+) ABSENT (~) :; 3; Frequency count of items Frequency count of items 2 I-z- in the domain present in present in clinician B's g a both clinicians' Cx/Dx Cx/Dx but not in clini- ; g a cian A's Cx/Dx O. m. A g ‘L Frequency count of items Frequency count of items -5 ._ in the domain present in in the domain absent in 'E E. clinician A's Cx/Dx but both clinicians' Dx/Dx -- ‘3 not in clinician B's Cx/Dx ES '1 c Figure Il.--Contingency table for calculation of Phi--Example l. Clinician A, SIMCASE Y + _ a (+ +) b (+ -) a + b + c (- +) d (- -) C + d Clin1c1an B, SIMCASE Y a + c u b + d u N Figure 12.--Contingency table for calculation of Phi--Example 2. 255 The Phi statistic is bounded by -l (when items are in cells b and c only) and l (when items are in cells a and d only), when the distributions in the marginals are equal. In all other instances, the maximum and minimum values will be less than 1 and greater than -l. An example of a completed table is shown in Figure 13. Clinician A + - ‘2 + 2 l 3 .2 a b .2 ----- .5 :3 - T 4 5 c d 3 I 5 | 8 Cues (C) of Cues (C)of Domain Clinician A Clinician B of SIMCASE Y SIMCASE Y Cues (C) Cl Cl C3 C2 C3 C3 C5 C4 C5 C5 C7 C6 C7 C8 Figure 13.--An example of a completed contingency table. 256 The second of the two indices used in this study to describe the interclinician agreement was A:%:C" which describes the propor- tion of agreement where the base was the total number of cues col- lected (or diagnostic statements made) for which one or the other or both clinicians collected the cue or made the diagnostic statement.* The upper bound of the index would be the value of the index‘3;%$%;fi-, which describes the proportion of agreement where the base was the total_number of cues in the particular cue domain (or diagnostic statements in the statement domain). The Porter Index, 33%:C" excludes clinician agreement not to request a cue or make a diagnostic statement, i.e., the "d" cell (- -) in the 2 x 2 contingency table. Intraclinician Agreement The same two statistical measures, the Phi and the Porter, that were used to determine interclinician agreement were used for intraclinician agreement. The intraclinician-agreement statistic reflects the agree- ment of one clinician‘s collection of cues for a specific case with his own collection of cues on an alternate form of the same case. It compares the presence or absence of certain cues collected by a given clinician (C) on a Specific case at Time 1 (T1) to the presence or absence of certain cues collected by that same clinician on the same case (alternate form) at a later time (T2). The same analysis can be used to compute the intraclinician agreement on *Statistic developed by Andrew Porter, Michigan State Univer- sity, for the National Day Care Study(Nilcox, l977, pp. 54-60). 257 diagnostic statements selected. One Phi coefficient is computed for each pair of sessions. This comparison is summarized in Figure 14. Clinician C SIMCASE Y, FORM l PRESENT (+) ABSENT (-) 3? Frequency count of items Frequency count of items “’ present in the domain in in the domain present in N 5 both sessions for FORM l the session for FORM 2 DE 3 and FORM 2 of SIMCASE Y SIMCASE but not in FORM l :8 E: SIMCASE Y to a a b '8:: 'Etu ,~ Frequency count of items Absent in both sessions ".22 J, present in the session for FORM 1 and FORM 2 “g ._ for FORM 1 SIMCASE but of SIMCASE Y 7: g not in FORM 2 SIMCASE Y 52 c d Figure I4.--Contingency table for calculation of intra Phi--Example l. The calculation of the Phi is derived from the preceding con- tingency table in the following manner: (ad - bc) /(a+c)(b+d)(c+d)(a+b) Phi = (See Figure 15.) 258 Clinician A SIMCASE Y, FORM 1 + .. N 3+ a<++) b<+-> a+b (ULL T3>3 ..... .534, 2355 - c (- +) d (- -) c + d 573 ..... I I a+c ' b+d l N Figure IS.--Contingency table for calculation of intra Phi--Example 2. The statistic is bounded by -l (when items are in cells b and c only) and l (when items are in cells a and d only), when the distributions in the marginals are equal. In all other instances, the maximum and minimum values will be less than l and greater than -l. An example of a completed contingency table is shown in Figure 16. Clinician A SIMCASE Y, FORM 2 Cues (C) of Clinician'A SIMCASE Y, FORM l C1 C3 C4 Figure 16.--An example of a completed contingency table. 4. 259 Clinician A SIMCASE Y, FORM I + — 2 2 I 3 I 3 ' 5 Cues (C) of Clinician A SIMCASE Y, FORM 2 CI C3 C6 C7 Domain Cues (C) CT C2 C3 C4 C5 C6 C7 C8 260 The Porter Index is also used to compute intraclinician agreement, where a clinician is compared with himself on cues col- lected or diagnostic statements made for alternate forms of the same case. Directions for Computing Relationship Responses The means and standard deviations were computed (for each run for each subject) on the hypothesis-directed inquiry responses by the following procedures, using a hand calculator: l. Compute the sums of the individual columns for relation- ship numbers l, 5, 6, 9, and 10 from each of the relationship tables. (See Table 20 and Appendix H.) 2. Compute the means (of the sums of the individual columns for relationship numbers 1, 5, 6, 9, and lO). 3. Compute the standard deviation (of the variability of the value for each run from the mean of the overall sum of relation- ships l, 5, 6, 9, and lO) from the relationship table. The means and standard deviations were computed for cue- directed inquiry on each run for each subject according to the fol- lowing procedures, using a hand calculator: l. Compute the sums of the individual columns for relation- ship numbers 2, 3, 4, 7, 8, ll, l2, l3, and 14 from each of the relationship tables. 2. Compute the means (of the sums of the individual columns for relationship numbers 2, 3, 4, 7, 8, ll, l2, l3, and l4) from the relationship tables. 261 3. Compute the standard deviation (of the variability of the value for each run from the mean of the overall sum of relation- ships 2, 3, 4, 7, 8, ll, l2, l3, and l4) from the relationship tables. The same two sets of procedures were used to compute hypothesis-directed inquiry and cue-directed inquiry for relation- ships appearing for diagnostic statements (Table 23, based on rela- tionship tables for diagnostic statements). BIBLIOGRAPHY 262 BIBLIOGRAPHY Barrows, H. 5.; Feightner, J. W.; Neufeld, V. R.; and Norman, G. R. Data Gathering Problem Solving and Management-Observation of the Clinical Encounter in Time Segments. Hamilton, Canada: McMasters University, 1976. Bell, A. E., and Aftanas, M. S. "Some Correlates of Reading Retardation." Perceptual and Motor Skills 35 (October 1972): 659-67. Bond, G. L., and Tinker, M. A. Reading Difficulties: Their Diagnosis and Correction. New York: Appleton-Century-Crofts, Inc., 1957. Brown, V., and Botel, M. Dyslexia: Definition or Treatment? Bloomington, Ind.: ERIC/CRIER, Reading Review Series, 1972. Burnett, R. "The Diagnostic Problem Solving Proficiency of Elemen- tary Teachers in Teaching Reading.“ Ph.D. dissertation, Indiana University, 1961. Carter, H. L. J. "Determining Consequential Factors in Diagnosis." In Readinngiagnosis and Evaluation, Proceedings of the Thirteenth Annual Convention of the International Reading Association. Edited by D. L. DeBoer. Newark, Delaware: International Reading Association, 1970. , and McGinnis, D. J. Diagnosis and Treatment of the Disabled Reader. London: The Macmillan Co., 1970. Chall, J. S. "A Decade of Research on Reading and Learning Dis- abilities." In What Research Has to Say,About Reading Instruction. Edited by S. J. Samuels. Newark, Delaware: International Reading Association, 1978. Clarkson, G. P. E. Portfolio Selection: A Simulation of Trust Investment. Englewood Cliffs, N.J.: Prentice-Hall, 1962. Cureton, E. E. "Note on PHI/PHI(MAX)." Psychometrica 24 (1959): 89-91. DeBoer. J. J., and Dallmann, M. The Teaching of Reading. New York: Holt, Rinehart and Winston, Inc., 1960. 263 264 DeDombal, F. T.; Horrocks, J. C.; Clamp, S. E.; and Storr, J. E. "Simulation Techniques and Computer-Aided Teaching of the Clinical Diagnostic Process: Five Years Experience." Medinfo 74 Proceedings, pp. 247-52. Amsterdam: North Holland Publishing Co., 1974. DeDombal, F. T.; Leaper, D. J.; Staniland, J. R.; McCann, A. P.; and Horrocks, J. C. "Computer-Aided Diagnosis of Acute Abdominal Pain." British Medical Journal 2 (1972): 9-13. DeGroot, A. D. "Perception and Memory Versus Thought: Some Old Ideas and Recent Findings. In Problem Solving: Research, Method, and Theory, pp. 19-20. Edited by B. Kleinmuntz. New York: John Wiley and Sons, 1966. Thought and Choice in Chess. The Hague: Mouton, 1965. Downey, N. M., and Heath, R. W. Basic Statistical Methods. New York: Harper and Row, 1965. Durrell, D. D. The Improvement of Basic Reading Abilities. Yonkers, N.Y.: World Book Co., 1940. Ekwall, E. Diagnosis and Remediation of the Disabled Reader. Boston: Allyn and Bacon, Inc., 1976. Elstein, A. S. "Clinical Judgment: Psychological Research and Medi- cal Practice." Science 194 (November 1976): 696-700. "Decision Analysis and the Psychology of Decision Making." East Lansing: Office of Medical Education Research and Development, Michigan State University, 1977. "A Process Tracing Analysis of Medical Problem Solving." Paper presented to the Annual Meeting of the American Psy- chological Association, Chicago, Illinois, September 3, 1975. ; Kagan, N.; Shulman, L. S.; Jason, H.; and Loupe, M. J. "Methods and Theory in the Study of Medical Inquiry." Journal of Medical Education 47 (1972): 85-92. Elstein, A. S.; Shulman, L. S.; Sprafka, S. H.; Jason, H.: Kagan, N.; Allal, L. K.; Gordon, M. J.; Loupe, M. J.: and Jordan, R. D. "An Analysis of the Medical Inquiry Process: Final Report to the Bureau of Health Manpower, OMERAD." East Lansing: College of Human Osteopathic Medicine, Michigan State Univer- sity, 1976. Medical Problem Solving: An Analysis of Clinical Reasoning. Cambridge: Harvard University Press, 1978. 265 Elstein, A.; Shulman, L.; Vinsonhaler, J.; Wagner, C.; and Bader, L. “Characteristics of the Clinical Problem Solving Model and Its Relevance to Education“ (Res. Ser. No. 27). East Lansing: Institute for Research on Teaching, Michigan State University, 1977. Feinstein, A. R. Clinical Judgment. Baltimore: Williams and Gil, C. Glaser, Wilkins, 1967) ; Hoffmeyer, E. M.; VanRoekel. J. L.; and Weinshank, A. B. "Clinical Problem Solving in Reading: Theory and Research" (Res. Ser. No. 45). East Lansing: Institute for Research on Teaching, Michigan State University, 1979. R.; Damrin, D. E.; and Gardner, F. M. “The Tab Item: A Technique for the Measurement of Proficiency in Diagnostic Problem Solving Tasks." Educational and Psychological Measurement 14 (1954): 283-93. Glass, G. V., and Stanley, J. C. Statistical Methods in Education. Englewood Cliffs, N.J.: Prentice-Hall, Inc., 1970. Gomez, M. R. "Specific Learning Disorders in Childhood." Psychi- Harris, Harris, atric Annals 2 (May 1972): 49-65. A. J. "Diagnosis of Reading Disabilities." In Corrective Reading in Classroom and Clinic, pp. 80-87. Supplementary Educational Monographs, No. 79) Edited by H. M. Robinson. Chicago: University of Chicago Press, December 1953; reprint ed., New York: Harper and Row, Publishers, 1972. How to Increase Reading Ability. 4th ed. New York: Longmans, Green, and Co., 1961. , and Roswell, F. G. “Clinical Diagnosis of Reading Dis- ability." Journal of Psychology 36 (1953): 323-40. A. J., and Sipay, E. R. How to Increase Reading Ability: A Guide to Developmental and Remedial Methods. 6th ed., rev. and enl. New York: David McKay Co., 1975. Hemphill, J. K.; Griffiths, D. E.; and Frederiksen, N. Adminis- trative Performance and Personality. New York: Bureau of Publications, Teachers College, Columbia University, 1962. Hoffmeyer, E. M. "A Study of the Process and Outcomes of Diagnostic Problem Solving Among Reading Clinicians." Unpublished prOposal for the degree of Ph.D., Michigan State University, October 1978. 266 , and Bader, L. A. "A Study of the Diagnostic Decision-Making and the Problem-Solving Behavior of Reading Clinicians as They Diagnose Reading Disability Cases." Unpublished paper, Michigan State University, May 1978. Jan-Tausch, E. "Reading Diagnosis--The Essential Ingredient in Teaching Every Child to Read." In Diagnostic Viewpoints in Readin . pp. 1-7. Edited by R. Leibert. Newark, De1.: International Reading Association, 1971. Kitchen, F.; Polin, R.; and Salters, D. Data Processor's Manual for the Clinical Studies Project of the Institute for Research on Teaching. East Lansing: Michigan State Uni- versity, February 1979; rev. ed. June 1979. Kleinmuntz, B. "The Processing of Clinical Information by Man and Machine." In Formal Representation of Human Judgment. Edited by B. Kleinmuntz. New York: Wiley, 1968. Lanier, J. E., and Shulman, L. 5. Technical Proposal submitted to and approved for funding by NIE. East Lansing: Insti- tute for Research on Teaching, College of Education, Michigan State University, April 1976. Lee, A., and Weinshank, A. “CLIPIR Pilot Observational Study of Reading Diagnosticians" (Res. Ser. No. 14). East Lansing: Institute for Research on Teaching, Michigan State Univer- sity, 1976. Norman, 6.; Barrows, H. S.; Feightner, J. W.; and Neufeld, V. R. "Measuring the Outcome of Clinical Problem Solving." Unpublished paper, McMaster University, Hamilton, Ontario, 1977. Penn, Julia M. "Reading Disability: A Neurolo ical Deficit?" Exceptional Children 33 (December 1966?: 243-48. Rimoldi, H. J. A. "A Technique for the Study of Problem Solving." Educational and Psychological Measurement 15 (1955): 450-61. . "The Test of Diagnostic Skills." Journal of Medical Education 36 (1961): 73-79. ; Devane, J.; and Haley, J. "Characterization of Processes." Educational and Psychological Measurement 21 (1961): 389-92. Robinson, H. M. "Clinical Procedures in Diagnosing Seriously Retarded Readers." In Better Readers for Our Times. pp. 152- 56. Conference Proceedings of the International Reading Association. New York: Scholastic Magazines, 1956. 267 Rosenthal, J. H. "Neurophysiology of Minimal Cerebral Dysfunctions." Academic Therapy 8 (Spring 1973): 291-94. Schwartz, W. B., et a1. "Decision Analysis and Clinical Judgment." American Journal of Medicine 55 (1973): 459-72. Sherman, G. B. "Training Reading Specialists in Diagnosis" (Res. Ser. No. 31). East Lansing: Institute for Research on Teaching, Michigan State University, 1979. Sherman, G.; Vinsonhaler, J.; and Weinshank, A. "A Model of Reading and Learning to Read." Personal correspondence to Clinical Studies Project Staff of the Institute for Research on Teaching, Michigan State University, 1978. Shulman, L. S. "Seeking Styles and Individual Differences in Patterns of Inquiry." School Review 73 (1965): 258-66. , and Elstein, A. "Studies of Problem Solving, Judgement and Decision Making: Implications for Educational Research." In Review of Research in Education, pp. 3-42. Edited by F. Kerlinger. New York: Peacock, 1975. Shulman, L. 5.; Loupe, M. J.; and Piper, R. M. Studies of the Inquiry Process: Inquiry_Patterns of Students in Teacher-Training Pro rams. Final report, Project No. 5-0597, Office of Education. East Lansing: Michigan State University, 1968. —Smith, H. P., and Dechant, E. V. Psychology in Teaching Reading. Englewood Cliffs, N.J.: Prentice-Hall, 1961. Spache, G. D. Diagnosing and Correcting Reading Disabilities. Boston: Allyn and Bacon, Inc., 1976. . Good Reading for Poor Readers. 5th rev. ed. Champaign, 111.: Gerrard Press, 1966. Sprafka, S. A., and Elstein, A. S. "What Do Physicians Do? An Analysis of Diagnostic Reasoning." Symposium presentation of the Conference on Research in Medical Judgment, Office of Medical Education Research and DevelOpment, Michigan State University, East Lansing, Michigan, May 1974. Stephens, E. C. A. "A Study of the Problem-Solving Behavior of Teachers as They Diagnose a Child's Reading Performance and the Effects of Experience and Training on That Behavior." Ph.D. dissertation, Michigan State University, 1978. 268 Taylor, W. C.; Skakun, E. N.; and Wilson, 0. R. "Computerized Medical Self-Assessment." Medinfo 77 Proceedings. Edited by D. B. Shires and H. Wolf. Amsterdam: North Holland Publishing Co., 1977. VanRoekel, J. L. "The Problem Solving Behavior of Teachers Trained in Reading and Teachers Trained in Learning Disabilities When Diagnosing Simulated Cases of Educational Difficulty in a Clinical Setting." Ph.D. dissertation, Michigan State University, in progress. Vinsonhaler, J. F. "Clinical Problem Solving Among Reading Special- ists: The Problem of Agreement on a Diagnosis." East Lansing: Institute for Research on Teaching, Michigan State University, April 1979a. "The Consistency of Reading Diagnosis." (Res. Ser. No. 28). East Lansing: Institute for Research on Teaching, Michigan State University, June 1979b. "Implications of an Interdisciplinary Theory of Clinical Inquiry for Diagnostic Efficacy." In Proceedings of AIMS Conference. Edited by F. Gremy and F. T. DeDombal. Amsterdam: North Holland Publishing Co., 1979c. "Theory." Personal Communication paper. East Lansing: Institute for Research on Teaching, Michigan State University, 1978. Vinsonhaler, J. F.; Wagner, C. C.; and Elstein, A. S. "The Inquiry Theory: An Information Processing Approach to Clinical Problem Solving Research and Application" (Res. Ser. No. 1). East Lansing: Institute for Research on Teaching, Michigan State University, 1977. Vinsonhaler, J. F.; Wagner, C. C.; Elstein, A. 5.; and Shulman, L. S. Wagner, "The Inquiry Theoretic Approach to Clinical Problem-Solving Research and Application: Basic Management Information System (BMIS) Technical Manual," Vol.1 (Res. Ser. No. 15). East Lansing: Institute for Research on Teaching, Michigan State University, 1977. C. C.; Kitchen, F.; Polin, R.; Vinsonhaler, J. F.; VanRoekel, J. L.; Gil, 0.; and Hoffmeyer, E. M. "Observational Studies Data Analysis System (OSDAS)." IRT Technical Report. East Lansing: The Institute for Research on Teaching, Michigan State University, 1979. 269 White, J., and White, M. "Perceptual and Psycholinguistic Factors in Retarded and Advanced Levels of Reading and Spelling Skills.“ Slow Learning Child: The Australian Journal on the Education of Backward Children 19 (July l972):7117-23. Whitecraft, C. J. "Approaching Cooperative Research in Learning Disabilities Through Psycholinguistics." Journal of Learning Disabilities 4 (December 1971): 568-71. Wilcox, M.; Alford, S.; Carew, J.; Gillis, G.; Porter, A.; and Stallings, J. National Day Care Home Study,_Phase 1 Pilot Study Report. Menlo Park, Calif.: Stanford Research Insti- tute, June 1977. Wilson, R. M. Diagnostic and Remedial Reading for Classroom and Clinic. Columbus, Ohio: Charles E. Merrill Publishing Co., 1977.