I. t it unit'- rr$n¢a14 u. :- . 1. 1 r.-. 03 4. ft 39?; n (.3? J18”,- ui if." 3;!“ 5 o 53 ; 0-... v1 . A v 01.31,: .J t. 3,. afldfiu’v... his! I“. :{f’ .‘3 fix" 7 4.117. VIII... '0: flit. : 5; .3 1.311%: ‘- .V c.) . 1.3.3 .v .3." .1... .1 v 2'. .f1.. II. c u ..u ..::.t. . I .b:'1lv 1" "Ivllliv: 3le 1(0) lvl [(9): I}! "OOIAIOD rl’n‘v. lit-it'll; 3...? p0:.. .vlhv . c. i .. ‘ i o f y I415! l. . SITY LIBRARI llslllllll 3 1293 00899 081 This is to certify that the dissertation entitled Influence of knowledge and clinical experience on clinical decision making of registered nurses and nursing students using interactive videodisc simulation presented by Joan Elaine Predko has been accepted towards fulfillment of the requirements for Ph.D. Educational degreein Psychology Date Z/BI/ya MS U i: an Affirmative Action/Equal Opportunity Institution 0-12771 _.- '~—-————+W’ LIBRARY Michigan State University K _._I' PLACE IN RETURN BOX to remove this checkout from your record. TO AVOID FINES return on or before date due. DATE DUE DATE DUE DATE DUE MSU Is An Affirmative Action/Equal Opportunity Institution cmmut INFLUENCE OF KNOWLEDGE AND CLINICAL EXPERIENCE ON CLINICAL DECISION MAKING OF REGISTERED NURSES AND NURSING STUDENTS USING INTERACTIVE VIDEO SIMULATION BY Joan Elaine Predko A DISSERTATION Submitted to Michigan State University in partial fulfillment of the requirements for the degree of DOCTOR OF PHILOSOPHY College of Education Department of Counseling, Educational Psychology, and Special Education 1992 ABSTRACT INFLUENCE OF KNOWLEDGE AND CLINICAL EXPERIENCE ON CLINICAL DECISION MAKING OF REGISTERED NURSES AND NURSING STUDENTS USING INTERACTIVE VIDEO SIMULATION BY Joan Elaine Predko Although an increasing number of researchers are examining clinical decision making, only a few, small descriptive studies have focused on critical care nursing. These studies have identified knowledge, experience, and education as being important variables; but they demonstrate conflicting results as to the effect of these variables on clinical decision making skills. Studies on simulation as a method to assess clinical decision making have yielded mixed psychometric results. No studies using interactive videodisc simulations have been reported. This descriptive correlational study examined the influence of theoretical knowledge, practical knowledge, and clinical experience on clinical decision making skills of 35 critical care nurses and 35 senior baccalaureate nursing students. All 70 subjects completed the cardiovascular subscale of the Basic Knowledge Assessment Tool (BKAT) as a measure of theory based knowledge, the Cardiovascular Self— Evaluation Tool (CST) of practical knowledge, and one interactive videodisc clinical simulation measuring decision making skills. Clinical experience was operationalized as a dichotomous variable with nursing students possessing Joan Elaine Predko minimal critical care experience and registered nurses (RNs) possessing at least one year. Correlational analyses supported a moderately strong positive linear relationship between theoretical knowledge and clinical decision making (r = .63), practical knowledge and clinical decision making (r = .47), and theoretical knowledge and practical knowledge (r = .73). All Pearson r correlations were significant (1 tailed, p < .0005). Multiple regression procedures, using theoretical knowledge, practical knowledge, and clinical experience as predictors, produced a significant equation that explained 51% of the variance in clinical decision making skill; however, multicollinearity caused theoretical and practical knowledge to be unstable predictors. MANCOVA was used to examine the effect of clinical experience on theoretical knowledge, practical knowledge, and clinical decision making with age as a covariate. The mean scores of the student and RN groups were significantly different for all three dependent measures (Simulation, BKAT, CST) with the RNs performing better than the students. These group differences provided support for construct validity of all three study instruments. A three-way ANCOVA revealed a significant interaction of practical knowledge and clinical experience. For students, mean simulation proficiency scores decreased as practical knowledge scores increased; Joan Elaine Predko and for RNs, mean simulation proficiency scores increased as practical knowledge scores increased. Decision making patterns were identified using proficiency and commission error scores. The competent approach was used most frequently by the RN group and by high scorers on the BKAT and CST. The nondiscriminating and random approach were used most frequently by the student group and the low BKAT and CST scorers. ACKNOWLEDGEMENTS It is with extreme gratitude and pleasure that I acknowledge those special people who have made this dissertation a reality. Dr. Stephen Yelon, my academic advisor and dissertation chairperson, provided unlimited understanding, advice, and support at every stage of my doctoral program. He allowed me the autonomy I desired for which I am especially grateful. His gentle nudging was both timely and administered with humor and caring. Dr. Yelon's demonstrated excellence as a teacher and scholar has been a continuous source of inspiration. Special appreciation is extended to the members of my dissertation committee, Dr. Lawrence Alexander, Dr. Marilyn Rothert, and Dr. Donald Hamachek, who each gave enthusiasticly of their time and expertise. My husband, James, has been both intellectual confidant and devoted supporter. I am profoundly grateful for his never ending and abundant support throughout these many years of graduate study. To my children, Christopher and Lisa, a special thanks for understanding a Mom who was always busy taking a class or working on a paper. I wish to thank my long distance friend and fellow researcher, Colleen Glavin-Spiehs, who was relentless in her efforts to keep me on track. Her enthusiasm, energy, and humor always came at the right time. My gratitude also extends to the many persons at the College of Nursing and Academic Computing Center of Michigan State University and at Sparrow Hospital who greatly facilitated the collection of data. Their enthusiasm and interest in this study made data collection enjoyable. I am thankful for the personal and professional support of Dr. Mary Anne Rizzolo, Program Director at the American Journal of Nursing. Data collection would not have been possible without access to multiple copies of the videodisc made possible by the American Journal of Nursing. A sincere thanks to Mary Garrett at Lansing Community College for her special efforts in securing the additional interactive video equipment needed for data collection. Special appreciation is extended to Denise Grimes, Marilyn Meinhardi, Alan O'Brien, and Kathy Ribbons who so willingly gave of their time and talent to serve on the panel of cardiovascular nursing experts. Finally, I wish to acknowledge that this investigation was partially supported by a grant from the Alpha Psi Chapter of Sigma Theta Tau International. vi TABLE OF CONTENTS List of Tables . . . . . . . . . . . . . . . . . . List Of Figures 0 O O O O O O O O O O O O O O O O 0 CHAPTER I - THE PROBLEM . . Introduction . . . . . Purpose . . . . . . . Conceptual Framework . . . Definitions & Operationalization Significance . . . . . . . . . . 000... CHAPTER II - LITERATURE REVIEW . . . . . . . . . . Overall Perspective . . . . . . . . . . f Study Variabls es Literature Related to Specific Study Variables and Methods . . . . . . . . . . Relationship Between Knowledge & Clinical Decision Making . . . . . Relationship Between Experience 8 Clinical Decision Making . . . . . . . . . . . . Relationship Between Process & Outcome of Decision Making . . . . . Validity of Simulations . . Critical Care Research . . . . . Summary . . . . . . . . . . . . CHAPTER III - DESIGN AND METHODOLOGY . Design . . . . . . . . . . . . . . .Research Questions and Hypotheses Data Collection Methodology . . Selection of Subjects . . . . . Procedure for Data Collection . . Protection of Human Subjects Student Data Collection . . . Registered Nurse Data Collectin All sample subjects . . . . . Data Analysis and Management . . . e e g e e e e e e e 0 CHAPTER IV - RESULTS . . . . . . . . . . . . . . Sample Subjects . . . . . . Reliability & Validity of Study Instruments Hypotheses Testing . . . . . . . . . . . . . Relationships Between Variables . . . . Differences Between Groups. . . . . . . Patterns of Decision Making in the Simulation. vii 18 21 23 24 26 29 31 31 32 33 41 42 43 43 45 46 46 48 48 54 70 7O 75 79 TABLE or CONTENTS (cont.) CHAPTER V - DISCUSSION . . . . Findings . . . . . Overall Significance . Limitations . . . . . Implications for Nursing Education and Practice Implications for Future Research . . . . . . BIBLIOGMPHY O O O O O O O O O O O O O O O O O O O 0 Appendix Appendix Appendix Appendix Appendix Appendix Appendix Appendix Appendix Appendix Appendix Appendix Appendix KENQHEQ MMUOU? Demographic Profiles . . . . . . . Basic Knowledge Assessment Test (BKAT) . Cardiovascular Self-Evaluation Tool (CST) Interactive Video Case Study Description Expert Panel Members . . . Directions for the Interactive Videodisc Simulation . . . . . . . . . . . . . Student Recruitment Flyer . . . . . . Student Consent Form . . . . . . . . . Explanatory Memo and Consent For RNs . Computer Program Scoring of Simulation Expert Panel Scoring of Simulation . . Simulation Feedback . . . . . . . . . Scatterplots . . . . . . . . . . . . . viii 85 85 94 97 100 102 104 112 116 121 124 125 126 127 128 129 130 134 138 139 Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table 8. 9. 10. 11. 12. 13. 14. 15. 16. LIST OF TABLES Comparison of Student 8 Registered Nurse Demographic Characteristics . . . . . . . . . Demographic Characteristics of RN Group from Categorical Variables. . . . . . . . . . Demographic Characteristics of RN Group from Continuous Variables . . . . . . . . . . Description Summary of BKAT Scores. . . . . . Description Summary of CST Scores . . . . . . Comparison of Local and AJN Panel Scoring of Simulation . . . . . . . . . . . . . . . . Description Summary of Simulation Proficiency scores 0 O O O O O O O O O O I O O O O O O 0 Comparison of Student & RN Simulation Scores. Mean Comparison of Efficiency, Commission Error, and Omission Error Scores. . . . . . . Frequency Responses of Students & RNs on Simulation Realism. . . . . . . . . . . . . . Frequency Responses of Students & RNs on Ease of Simulation Decision Making. . . . . . Frequency Responses of Students & RNs on Confidence in Simulation Decisions. . . . . . Frequency Responses of Students & RNs on Confusion of Simulation . . . . . . . . . . . Correlation Matrix of BKAT, CST, Simulation Scores 0 O O O O O I O O O O O O O O O O O O 0 Regression Models Using Independent Variables Analysis of Covariance for Effect of Experience on Simulation, BKAT, CST Scores. . ix 49 52 53 55 57 60 62 63 65 66 67 68 69 71 74 76 LIST OF TABLES (cont.) Table 17. Table 18. Table 19. 95% Confidence Intervals for Students 8 RNs on Simulation, BKAT, 8 CST. . . . . . . . . . 77 Analysis of Covariance Effect of Clinical Experience, CST, BKAT on SIM Proficiency. . . 78 Cell Means for Clinical Experience 8 CST. . . 79 Figure Figure Figure Figure Figure Association of Study Variables with Information Processing Concepts Comparison of Student 8 RN CST Median Scores LIST OF FIGURES Decision Making Patterns of Students 8 RNs . Decision Making Patterns of High and Low BKAT Scorers . Decision Making Patterns of High and Low CST Scorers xi 10 58 80 82 83 CHAPTER I THE PROBLEM Introduction Clinical decision making is the professional nurse's most vital skill. It is the basis on which all nursing care is given or not given. Clinical decisions are judgments made by the nurse that are directed toward helping the client promote, maintain, or regain health. These decisions include: (1) what to observe in the client situation, (2) inferences based on these observations, i.e. nursing diagnosis, and (3) nursing actions which will be implemented with or on behalf of the client, i.e. nursing management (Kelly, 1966; Tanner, 1987). The goal of the Joint Commission on Accreditation of Healthcare Organizations is to help health care organizations focus on improving the quality of care through key activities that directly affect patient care. In 1990, the Commission revised its nursing standards to "focus more directly on RN decision making" (Patterson, 1991). The quality of clinical nursing decisions is increasingly being recognized as a primary determinant of the quality of patient care. Although there is an expanding body of knowledge being 1 generated on clinical decision making in nursing, very few studies have focused on decision making of nurses in critical care areas. This paucity of studies is particularly unfortunate considering the prevalence of coronary heart disease and the corollary growth in number of hospital coronary care units. While the American Association of Critical-Care Nurses (AACN) has identified the use of the nursing process for clinical decision making as the basis for the delivery of nursing care (Sanford 8 Disch, 1989), the nursing process as the method of clinical decision making in nursing is being challenged by an increasing number of empirical studies which have not supported the linear sequence inherent in the nursing process or the assumption that there is only one type of process, i.e. the nursing process (Frederickson 8 Mayer, 1977; Broderick 8 Ammentorp, 1979; Pyles 8 Stern, 1983; Grier, 1984; Benner, 1984; Corcoran 1986a, 1986b; Westfall et al., 1986; Tanner et al., 1987, Itano, 1989, Grobe, Drew, 8 Fonteyn, 1991). While clinical decision making skills are considered inherent to critical care nursing, indeed to all of professional nursing; little is known about the underlying cognitive processes used by critical care nurses in making clinical decisions or how these skills are developed and used. This is of particular concern in light of the paradox 'that exists in the current staffing of hospitals, especially critical care units. Increased demands for the services of critical care, the nursing shortage, and high turnover rates of critical care nurses have resulted in the increased placement of new graduates and less experienced nurses in critical care units. The increase in the acuity of all hospitalized patients is causing most hospital units to function more like critical care units. Thus, while there is an increased need for nurses who can competently make decisions related to the acutely ill patient, there is a shortage of nurses with clinical expertise. The solution may lie in the ability to facilitate the transformation of the inexperienced novices into experts. The few investigations that have focused on clinical decision making in critical care nursing used inductive approaches with relatively small sample sizes (Baumann 8 Bourbonnais, 1982; Pyles 8 Stern, 1983; Thompson 8 Sutton, 1985; Smith, 1988; Baumann 8 Deber, 1989; Corcoran, Narayan, 8 Moreland (1988). However, findings from this small set of eclectic studies were consistent in identifying knowledge and experience as important factors effecting clinical decision making in critical care areas. Further investigation of the relationships between knowledge, experience, and clinical decision making processes are needed before we can begin to understand clinical expertise in nursing. Knowing the relationships among these variables ‘is the first step to answering questions such as: How does clinical expertise in decision making develop? What factors influence the transformation of novices to experts? Is the traditional master-apprentice model of clinical education used in many practice professions really the best method of teaching students how to make clinical decisions? Educational methodologists have successfully argued that the development of strategies to teach any task rests on an analysis of how competent individuals perform that task (Glaser, 1976). In addition, effective instruction is based on knowledge of the current state of the learner. Thus, an increased understanding of the cognitive processes employed by both novice and expert clinicians in making decisions should assist in the development of more efficient and effective instructional strategies to facilitate the novice to expert transformation. Since the early work of de Groot (1966) and Chase 8 Simon (1973) on specialized knowledge structures of master chess players, considerable research in cognitive psychology has been devoted to the study of expertise and has provided continuing evidence of a knowledge-competence dimension in a variety of domains. Clinical problem solving studies in medicine (Elstein et al., 1978; Kassirer 8 Gorry, 1978) and nursing (Tanner et al., 1987) have provided support for the idea that clinical expertise is critically dependent on task relevant knowledge and experience. Phenomenological studies on clinical judgment of nurses in actual practice (Benner 8 Wrubel, 1982; Benner, 1984; Brykczynski, 1989) have described two types of knowledge used in making clinical decisions: theoretical "knowing that" and practical "knowing how" (Kuhn, 1970; Polanyi, 1958). Benner (1983) states that clinical knowledge develops as both practical and theoretical knowledge are applied, refined, and extended in practice situations. These researchers believe that theoretical knowledge can be acquired in a decontextual fashion through lectures, reading, and discussion; but, the development of practical knowledge requires actual experience in a situation since it is contextual and transactional. Although nursing studies using written simulations and qualitative methods to compare novice and expert performance have been increasing, there is minimal cumulative information about the development of expertise in clinical judgment. It is apparent that larger sample sizes and more sensitive measurement tools are required before novice- expert differences will be adequately addressed. Purpose The purpose of this study is to examine the influence of theoretical knowledge, practical knowledge, and clinical experience on the clinical decision making process and outcome skills of critical care nurses. A comparison of inexperienced and experienced nurse clinicians will be used to answer the following questions: (1) What is the relationship between theoretical and practical knowledge? (2) How do these two types of knowledge relate to clinical decision making? (3) How do these two types of knowledge relate to the amount of previous clinical experience? (4) How does clinical experience relate to clinical decision making skill? (5) To what extent do knowledge and clinical experience explain the variance in clinical decision making skill? Conceptual Framework Elstein and colleagues have identified three major research paradigms used to study clinical reasoning: decision making, problem solving, and judgment (Elstein 8 Bordage, 1979; Elstein, Holmes, Ravitch, Rovner, Holzman, 8 Rothert, 1983). The decision making approach, based on decision theory, is prescriptive in nature and produces recommendations for optimal decision making while the problem solving approach, based on information processing theory, is descriptive and attempts to model human decision making behavior. The judgment paradigm, based on Brunswik's lens model, is both descriptive and prescriptive and focuses on the factors that influence judgments. Tanner (1983), in a review of nursing research on clinical judgment, identified the need for an additional research paradigm, the phenomenological approach: In this approach the researcher studies the judgment skills of experts in the naturalistic environment in order to explore 7 the contextual factors and to capture the gestalt view of the judgment process. LeBreck (1989) also recommended inductive studies based on theories of cognitive skill acquisition (Dreyfus 8 Dreyfus, 1986; Anderson, 1982) because previous theories have failed to provide a complete account of clinical judgment. The advantage of a single research paradigm is that it provides the conceptual and methodological framework to conduct an in-depth analysis of a particular aspect of clinical judgment. However, each paradigm has limitations. It may be possible to overcome the limitations of any one method by combining approaches in the study of a problem. Elstein and colleagues (Elstein et al., 1982) addressed this particular issue when they stated "The challenge lies in understanding how different methods can be used to complement and thereby strengthen each other to address the making of health care decisions" (p. 58). After a review of the literature, Tanner (1987) concluded that "in 20 years of research on clinical judgment in nursing, no single theory has been investigated sufficiently to conclude that the theory can be supported or refuted, or that it is in need of revision" (p. 158). This is not surprising considering the relatively small number of studies and the variety of theoretical perspectives being used. This investigation utilizes a conceptual framework based on an information processing theory of problem solving 8 and cognitive skill acquisition theory. A blending of these approaches produces a model of clinical decision making that can be tested using knowledge and clinical experience as primary determinants of clinical decision making expertise. Information processing theory evolved from the work of Newell 8 Simon on human problem solving (Newell, Shaw, 8 Simon, 1958; Newell 8 Simon, 1972). This theory conceptualizes problem solving behavior as an interaction between an information processing system (the problem solver) and a task environment (the task as described by the researcher). Solving a problem involves transforming an initial state by a set of operations into a goal state. The problem solver constructs a ”problem space" based on past experience and stored knowledge. The theory assumes that human information processing capacity is limited, i.e. "bounded rationality" (Newell 8 Simon, 1972). To be effective, the problem solver must employ strategies to adapt to these limitations and the demands of the task. In clinical decisions, the most relevant limit is the relatively small capacity of working memory (Miller, 1956) compared to the essentially infinite size of long term memory. Clinicians cannot, in a brief time, work efficiently with all that they know about a problem or all the data that could be collected. The clinician constructs a simplified "problem space" in which to solve the problem. Some common features of good and poor clinical judgment are consequences of efforts to cope with this information 9 processing limitation (Elstein, 1988). Information processing theory offers a meta-theoretical framework that allows the researcher to see what is not directly observable. Instead of the black box approach of decision theory, it seeks to understand the clinical decision making process by recording and analyzing the steps and thoughts of clinicians as they attempt to solve clinical problems. This is most often done using some form of process tracing methodology such as case study simulations and/or verbal protocol analysis. Since information processing theory focuses on the processing of information and not what is used as information, additional theory was sought to focus on the types of information used by the decision maker. Based on theories of skill acquisition (Anderson, 1982; Dreyfus 8 Dreyfus, 1986), clinical judgment can be viewed as a cognitive skill that develops as both practical and theoretical knowledge are applied, refined, and extended in practice situations (Benner, 1983). This framework allows differentiation of the knowledge component into two parts: theoretical and practical. In this combined model of clinical decision making expertise, the characteristics of the decision maker and the task are viewed as the primary determinants of the decision making behavior. Figure 1. shows the association of these 10 information processing concepts (in capital letters) and the study variables (in small letters). The type of measuring instrument used for each study variable is included in italics. For this investigation, the task is the same for all subjects; no task characteristics will be studied. The influence of decision maker characteristics (practical knowledge, theoretical knowledge, and previous clinical experience) on clinical decision making behavior will be examined. DECISION MAKER + TASK DECISION MAKING BEHAVIOR CHARACTERISTICS (held constant) I Theoretical Knowledge Multiple choice exam + I Practical Knowledge Self rating of expertise + I Clinical Experience -------- > I Clinical Decision Making Student or Videodisc Simulation Registered NUrse Performance Figure 1. Association of Study Variables with Information Processing Concepts Information processing theory of problem solving would predict that knowledge and experience would be positively related to decision making behavior. Both skill acquisition 11 theories would predict that the amount of clinical experience and performance scores on a simulated case would be positively related to clinical expertise. Specifically, the more expert clinical decision maker will have higher simulation scores, a greater amount of clinical experience, and larger theoretical and practical knowledge scores. According to Anderson (1985), those with more expertise would have higher practical knowledge. Definition and operationalisation of study variables The decision maker characteristics examined in this study are theoretical knowledge, practical knowledge, and clinical experience. THEORETICAL KNOWLEDGE is the factual knowledge gained by such activities as reading and attending classes and may be formalized as abstract rules and principles. It does not include knowledge gained through actual experience. It is the ”knowing that" or declarative knowledge that Anderson (1982) proposes as the first level of cognitive skill development. This variable will be measured by the cardiovascular subscale of the Basic Knowledge Assessment Tool for Critical Care Nursing (BKAT). Theoretical knowledge is a continuous variable with a range of 0-31 based on the number of correct items. PRACTICAL KNOWLEDGE is defined as knowledge gained through experience. Experience is not just the passage of 12 time, but results when "preconceived notions and expectations are challenged, refined, or disconfirmed by the actual situation" (Benner, 1984 p. 3). Practical knowledge is the "knowing how" or procedural knowledge that Anderson (1982) proposes as the second level of acquiring a cognitive skill. Tanner (1989) describes practical knowledge as clinical expertise. In this study, practical knowledge is operationalized by a self evaluation of clinical expertise in the care of an acute cardiac patient. The Cardiovascular Self-Evaluation Tool (CST) score is an interval variable with a possible range of 0-120. CLINICAL EXPERIENCE is a dichotomous variable that indicates either no experience (senior baccalaureate nursing students) or at least one year of experience (registered nurses working in critical care units). For the registered nurses, an additional continuous variable representing years of full time equivalent experience in critical care units was measured. TASK is held constant by utilizing the same simulation for each subject. No characteristics of task are being studied. CLINICAL DECISION MAKING BEHAVIOR is defined as the use of knowledge (theoretical and practical) in selecting one or more of several alternatives in order to help patients regain, maintain, or promote health. It is operationalized by results obtained from the subjects as they progress 13 through an interactive video clinical simulation in which they identify and solve the patient’s problems. Both the process and outcome of clinical decision making will be examined by performance on this simulation. Process variables of interest are total number of decisions made, number of positively scored decisions, number of negatively scored decisions, and time expended to complete the simulation. An efficiency score will be calculated using the number of positive decisions divided by the number of total decisions. The outcome variable is a proficiency score which reflects the degree to which the subject’s route through the simulation correspond with those judged to be optimal. In addition two subscores (errors of commission and errors of omission) will be calculated. Significance This investigation has the potential to increase understanding of clinical decision making viewed from two perspectives: information processing theory and skill acquisition theory. This study tests the theory predicted relationships between the decision maker characteristics of practical and theoretical knowledge and experience with clinical decision making behavior as measured by a high fidelity simulation. Since the majority of novice-expert studies in nursing have focused on diagnostic reasoning using inductive techniques or small sample sizes, this study 14 will add an important quantitative perspective that looks at both diagnostic and management aspects of clinical decision making. Reliable and valid tools to measure clinical decision making are very limited. The literature revealed no previously published research using interactive video simulation to measure clinical decision making. This study will examine the psychometric properties of an interactive videodisc simulation to measure clinical decision making. This type of tool would be valuable to both nursing research and nursing practice to assess clinical decision making skills. It is only with accurate assessments that clinical nursing expertise in decision making can be maintained and quality patient care assured. In addition, the study results will help determine whether interactive videodisc simulation can differentiate between process and outcome of varying levels of performance. CHAPTER II LITERATURE REVIEW Overall Perspective An overall perspective of the literature on clinical decision making will be followed by results of specific studies related to the variables being investigated in this study. Research on clinical decision making may be classified along four dimensions: 1) by the component of the decision process studied, 2) by the theoretical perspective, 3) by the variables examined which influence both the decision process and outcome, and 4) by the methodology used to study clinical decision making. Each dimension will be reviewed briefly. Clinical decision making can be viewed as a series of decisions encompassing: 1) assessment decisions regarding what to observe in the patient situation, 2) inferential decisions, deriving meaning from data observed (e.g. diagnosis), and 3) management decisions regarding actions to be taken that will be of optimal benefit to the patient. Research studies tend to focus on one of these 3 components of the decision process. The majority of nursing research has centered on the diagnosis and the interaction between 15 16 data gathering strategies and diagnosis, similar to studies on medical decision making. The second dimension for classifying research on clinical decision making is the theoretical perspective and its associated research paradigm. Three theories most commonly used are information processing theory (descriptive), social judgment theory (descriptive 8 prescriptive), and decision theory (prescriptive). An emerging area of study is intuitive reasoning using a phenomenological approach utilizing interviews and grounded theory methods and based on skill acquisition theories. The third classifying dimension is composed of variables that influence both the process and the results of clinical decision making. These were identified by Hansen 8 Thomas (1968) and are as follows: 1) situational or task variables which define the overall complexity of the patient situation including the amount and nature of the information available to the nurse, the number of problems that the patient may be experiencing, and the courses of action available, 2) contextual variables which include the circumstances and setting in which the clinical decisions are made such as institutional policy, and time available to make the judgment, 3) decision maker variables which represent the characteristics the nurse brings to the task such as attitudes 8 values, past experience with similar tasks, clinical knowledge, and level of inferential ability; 17 4) risk/benefit variables that are associated with any action selected by the nurse. Since most research on clinical decision making uses simulated tasks (i.e. patient situations represented by written, filmed, or computerized case studies), the capacity to examine the influence of either the contextual or risk-benefit variables have been limited. However, the introduction of interactive videodisc technology offers a medium for greatly increasing the fidelity of simulations. Future technological advances such as work with virtual reality may make simulations seem as real as the actual experience. The majority of studies on these variables have focused on the nurse as a decision maker; many compared novices with experts. The development of tools to measure the cognitive processes underlying clinical decision making in nursing has been based on process tracing methods. Most studies have used written simulation performance and/or verbal protocol analysis. There has been only modest support for the validity of clinical simulations as a measure of clinical decision making (McLaughlin, Carr, Delucchi, 1981; Farrand, Holzemer, 8 Schleutermann, 1982; Holzemer, Resnik, 8 Slicher, 1987; Holzemer 8 McLaughlin, 1988). Considering the variety of simulation types in use and the limited number of validation studies, the need for further validation studies is apparent. Critics of verbal protocol analysis suggest that instructions to "think aloud" affect 18 performance (Dreyfus 8 Dreyfus, 1986). Henry and colleagues (1989) found no effect of verbalization of cognitive processes on clinical decision making performance of nurses using a computer-assisted clinical simulation. However, the question of the scientific status of verbal reports remains controversial (Nisbett 8 Wilson, 1977; Ericsson 8 Simon, 1980). Several clinical decision making studies based on the phenomenological approach (Benner, 1984; Pyles 8 Stern, 1983; Phillips 8 Rempusheski, 1985; Benner 8 Tanner, 1987; Smith, 1988; Brykczynski, 1989) have focused on intuitive reasoning in contrast to the analytic judgment processes studied by researchers using information processing, social judgment, and decision theory. Literature related to specific study variables and methods ;- a T-!:; . 3‘ --1 . ow ---- 11! ,g- . 2- :,o. nak'n- Most studies on clinical decision making have conceptualized knowledge only as level of education and have not specifically tested for theoretical or practical knowledge. Johnson (1988) in a meta-analysis of 139 studies comparing nursing education and performance, concluded that there is a significant difference between BSN and non-BSN (AD and diploma) prepared nurses on measures of nurse performance related to communication, knowledge, problem solving, and professional role. Clinical decision making 19 studies included in this meta-analysis were Aspinall, 1979; Davis, 1974; Farrand et al., 1982; Grier 8 Schnitzler, 1979; Frederickson 8 Mayer, 1977; and McCloskey, 1983. Additional studies have identified differences in clinical decision making related to level of education. Using five filmed patient situations and 1,576 professional nurses, Verhonick and associates (1968) found a positive relationship between the nurses level of education and number of nursing observations identified. However, years of clinical experience (ranging from under 1 to over 30) was not held constant. This confounding effect of education and experience often occurs in novice-expert studies on clinical decision making. Empirical novice-expert studies have assumed a knowledge difference between novices and experts based on level of education. No differentiation was made between theoretical and practical knowledge. Several studies focused on comparing the cognitive strategies of novice and experts. In a series of studies, Tanner and colleagues explored the cognitive strategies used in the diagnostic reasoning of junior (n=15) and senior (n=13) nursing students and practicing nurses (n=15) using videotaped simulations (Westfall et al., 1986; Tanner, et al., 1987). Although they were unable to find significant differences “between the three groups, these researchers did find that with increased levels of knowledge (and experience) there 20 was a trend toward greater diagnostic accuracy and more systematic data acquisition. Small sample size and diminished statistical power must be considered. Junior students, senior students, and practicing nurses were assumed to respectively possess increasing amounts of knowledge and experience; no measurement of knowledge and experience variables was reported. Matthews 8 Gaul (1979) found that graduate nursing students identified more diagnoses than undergraduate nursing students. Westfall et al.(1986) reported that nurses were more proficient and efficient than students when generating diagnostic hypotheses. The positive relationship between level of education and clinical decision making skill has also been supported in medicine (Neufeld, Norman, Feightner, 8 Barrows, 1981; Norman et al., 1985). Only one study was found that measured knowledge. Henry (1989), using the Basic Knowledge Assessment Tool (BKAT) for critical care nurses, found only limited support for a relationship between knowledge and clinical decision making as measured by four computerized simulations. The BKAT score was significantly correlated with atrial, but not ventricular simulation proficiency scores. Inductive studies have identified the importance of knowledge as a factor in clinical decision making. Corcoran et al.(1988), using a thinking aloud strategy in a telephone 21 triage study, analyzed four nurses' verbal protocols in three cases of varying complexity. They found that three types of knowledge were used, textbook knowledge, practical knowledge, and "rules of thumb" and that the knowledge was context dependent whereas the cognitive processes (hypothesis activation 8 evaluation) were more general in nature. Pardue (1987) surveyed 121 nurses with varied nursing education preparation and reported that all identified knowledge and experience as the most significant factors influencing their clinical decision making ability. ;- 1 _.,;, . .; ‘;-, :59- -, f a,. , . .- :f., ,1 . Experience has been conceptualized as actual years of clinical experience or task familiarity. The majority of studies have used the passage of time framework. Davis (1972, 1974) and Farrand et a1. (1982) found that clinical decision making skill peaks and then declines. Davis found that after five years of experience without continuing education, clinical decision making on five filmed patient situations declined. Farrand et al. found the lowest simulation proficiency scores in nurses with less than 2 years of experience or more than 15 years experience. Aspinall (1976), using a single written case to elicit causes for the patient's condition, found a decline in performance of nurses with more than 10 years of experience. Verhonick et a1. (1968) using five filmed patient situations 22 found that nurses with one to six years of experience made the largest percent of relevant observations. Similar to Davis's study, Del Bueno (1983) used a series of 12 patient situations to which 85 nurses listed pertinent observations and nursing actions. However, Del Bueno treated experience as a dichotomous rather than a continuous variable and found that nurses with at least seven months experience performed better than those with less than seven months experience. No opportunity to examine decline after seven months was permitted in this analysis. Like Del Bueno, many researchers investigating novice- expert differences in clinical decision making have used clinical experience as a dichotomous variable. Although the criteria for selecting novices and experts are often a combination of education and experience, those studies focusing on the effect of differences in experience will be reviewed. Several investigations compared differences between student nurses and experienced nurses resulting in experienced nurses collecting more relevant cues (Itano, 1989), using less extraneous information (Holden 8 Klingner, 1988), activating more complex inferences (Westfall et al. (1986), addressing more problems, (Broderick 8 Ammentorp, 1979), and making more accurate diagnoses (Tanner et al., 1987; Holden 8 Klinger, 1988). Monahan (1991) studied whether or not having a 60 hour clinical experience attached to a junior level baccalaureate 23 nursing theory course effected the clinical judgment accuracy of students. Although the clinical experience group (N a 8) identified a larger number of accurate clinical nursing judgments than the non clinical group (N = 8), the difference was not significant. Several explanations for not finding a statistical difference, other than no effect of clinical experience, can be offered: very low statistical power (approximately .12 even at an alpha level of .10), insensitivity of the clinical judgment instrument, and not controlling for the amount of previous clinical experience. Benner (1984), using a phenomenologic perspective, validated the Dreyfus model of skill acquisition (Dreyfus 8 Dreyfus, 1986). She found nursing decision making processes reflective of the five levels of skill acquisition: novice, advanced beginner, competent, proficient, and expert. Additionally, Benner found that movement through the levels is achieved by experience, i.e. task familiarity. In an inductive study of 21 expert nurses, Benner and Tanner (1987) were able to identify Dreyfus's six key aspects of intuitive judgment. Practical or skilled "know how" was one of these aspects. ;‘ -. .' 3! 9 31; .“1 .° ' '.:+‘ -!’ ' 0"90 '.‘ 1'! l". o ' Three nursing studies have addressed this relationship. Corcoran (1986a, 1986b), using protocol analysis, compared 24 the care planning approaches of six expert and five novice hospice nurses in three case studies of varying complexity. Although Corcoran found no significant relationship between approach to the plan and overall quality of the plan in any of the three case studies, low statistical power must be considered in the interpretation of this finding. Holzemer (1986) compared the route (typical or atypical) taken through one written Patient Management Problem (PMP) with simulation proficiency scores and average number of correct diagnoses of 79 nurse practitioners. The typical route consisted of the usual sequence of history, physical, laboratory tests, and management. The mean proficiency score was significantly higher for the typical group (n=68) than for the atypical group (n=11); however, there were no significant differences between route and mean number of correct diagnoses identified at the end of each PMP section. Henry (1989), using a sample of 140 critical care nurses and four computerized simulations on atrial and ventricular dysrythmias, found several significant relationships between the process and the outcome of clinical decision making. However, these relationships were not consistent across the type of simulation. V 1.3.! E E' 1 !° Studies have examined the construct and criterion related validity of written clinical simulations using a 25 branching approach. Dincher 8 Stidger (1972) studied the relationship between performance on a written simulated clinical situation and evaluation of clinical decision making by the clinical instructor in their investigation of 11 nursing students. A significant high correlation between the simulation performance and instructor evaluation was used as evidence for construct validity for clinical simulation performance. Rank order correlation between the two instruments was significant for simulation efficiency scores. Nonsignificance of proficiency scores was explained by an inflation of proficiency scores due to a scarcity of negatively scored simulation items. Holzemer et al. (1981) used a multitrait-multimethod approach with a national sample of 79 adult nurse practitioners to explore the construct validity of one type of clinical simulation, the written Patient Management Problem (PMP). The multiple traits measured were cognitive knowledge, problem solving skill, and perceptions of practice. The methods included objective measures (multiple-choice examination and PMP simulation) and subjective measures (self-chart audit, colleague evaluation, and self evaluation of clinical practice). Findings provided evidence of only modest support that PMP simulation is a valid measure of the construct clinical problem solving. PMP proficiency score was significantly correlated only with the multiple-choice exam (r=.54, p <.01) and self evaluation of clinical 26 practice (r = .23, p <.05); self chart audit and colleague evaluation were not significant. In another study, Holzemer et al. (1986) failed to find a significant relationship between simulation performance of nurse practitioners and chart audit or observation of clinical performance; thus, offering no support for criterion related validity of the PMP. Stross 8 Bellfly (1979) compared ratings of nurses' self evaluation of their ability to identify heart murmurs on a 5 point scale with the actual test results. Although they concluded the nurses perceptions were accurate, there was insufficient information provided to judge the accuracy of this conclusion. Two studies using physicians and pharmacists investigated the validity of written simulations. McLaughlin, Carr, 8 Delucchi (1980) found evidence of construct validity using the known groups method on two primary care simulations. Gage 8 Fielding (1980) found little evidence to support the relationship between PMP performance and performance in practice as measured by direct observation of pharmacy students interacting with role playing actor, but did find construct validity by comparing psychology students, pharmacy students, and practicing pharmacists. W McCloskey 8 McCain (1988) examined job performance of 320 hospital nurses across units and concluded that the best 27 predictor of critical care skills was the amount of experience the nurse has in practice. Since this was not a practically significant difference in experience, this investigator concurs with ideas presented by Sneed and colleagues (1987). They suggest that although work experience may be useful as a predictor of performance, the quality of the work experience may be more important than the quantity. Five studies used a grounded theory approach which included in-depth interviews of critical care nurses. In Baumann and Bourbonnais (1982) and the two replication studies (Bourbonnias 8 Baumann, 1985; Thomas 8 Sutton, 1985) nurses were presented with a case study (acute myocardial infarction with sudden onset of chest pain) and asked to identify appropriate nursing interventions, rationale for the interventions, and factors that played a role in their clinical decision making. Nurse subjects (total n=120) ranked knowledge and experience as the most important factors in the case study and critical incident reports in all three investigations. Pyles and Stern (1983) conducted in-depth interviews (n=28) to explore detection and prevention of cardiogenic shock in the acute myocardial infarction patient. They identified a process called the "Nursing Gestalt" in which nurses link together basic knowledge, past experiences, identifying patient cues, sensory cues, and the strategies of categorization and 28 differentiation to arrive at a diagnosis. Six experienced critical care nurses (at least two years of clinical experience) were the subjects of a qualitative study by Smith (1988) to investigate the phenomenon of deterioration in the critically ill. Smith found that this phenomenon can be characterized, identified, and acted on by experienced critical care nurses. Three studies using information processing approaches focused on triage decisions of emergency department and clinic nurses and use of supplemental data by cardiovascular nurses. Rausch 8 Rund (1981) studied triage decisions made by 15 RNs on 385 patients in an emergency department. A comparison of the nurses' predictions made during triage interviews with the actual outcomes ranged from 60% on diagnostic impressions to 90% on whether the patient would be admitted or discharged. Corcoran, Narayan, 8 Moreland (1988) conducted a study of nurses' decision making during telephone triage using three hypothetical cases of varying complexity. Analysis of the nurses' verbal protocols revealed three types of knowledge were used: factual, practical, and rules of thumb. Corcoran-Perry 8 Graves (1990) investigated supplemental information seeking of 46 cardiovascular nurses from 3 hospitals and discovered that the nurses used patient data, agency data, and domain knowledge as supplemental data (data not in memory). Cowan, M.J. (1990) and Kinney, M.R. (1990) offered 29 respective reviews of research in cardiovascular nursing 8 research in education for critical care nursing. Both concluded that there is a need for studies with larger sample sizes which use valid 8 reliable instruments and control for confounding variables. Vancott, Tittle, Moody, 8 Wilson (1991) analyzed 130 critical care nursing practice research articles published in five referred journals between 1979 and 1988. They reported that problems in sampling and analysis significantly declined in the latter half of the decade while errors in design and methodology declined but not significantly. All three reviews concur that the research is improving in quality and increasing in quantity; however, sample sizes remain small. Summary In their seminal experiments on clinical inference in nursing, Hammond and colleagues (1966a) concluded that cognitive tasks in nursing are probabilistic, complex, and varied. In an extensive review of the literature, Tanner (1983) failed to find any significant advancements in the understanding of clinical judgment in nursing beyond Hammond’s work. In 1984, Benner, utilizing a phenomenologic approach, provided rich descriptions (exemplars) of expert clinical decision making. Her work has inspired this researcher to incorporate the Dreyfus model of skill acquisition (Dreyfus 8 Dreyfus, 1986) in this investigation of clinical decision making in order to better understand 30 the role of knowledge and experience in the development of clinical decision making expertise. Although studies in nursing have supported a positive relationship between level of knowledge and clinical decision making, most have conceptualized knowledge as level of education. This study focuses on task specific knowledge and skills (both theoretical and practical knowledge) in an effort to validate the positive relationship between knowledge and clinical decision making skill. Clinical experience, measured in years, has shown a curvilinear relationship with clinical decision making ability. Both inexperienced and those with many years of experience demonstrate low clinical decision making skills while those in the middle demonstrate a high skill level. As in novice to expert studies, this study uses experience as a dichotomous variable based on task familiarity. This investigation focuses on practical knowledge, theoretical knowledge, and clinical experience with a goal of validating the positive relationships described in the literature between these variables and clinical decision making skill. In addition, this study explores whether an interactive videodisc simulation can differentiate between the clinical decision making skills (process and outcome) of subjects with varying levels of knowledge and experience. CHAPTER III DESIGN AND METHODOLOGY Selection of the research design was based on: 1) the type of research questions being investigated, and 2) a desire to increase the statistical precision of data analysis procedures. A descriptive correlational design was used to examine the relationships between theoretical knowledge, practical knowledge, clinical experience, and clinical decision making processes and performance. In order to reduce the size of the sample required to produce adequate power for data analysis, the researcher compared a group of inexperienced clinicians (senior nursing students) with experienced clinicians (practicing nurses) on the variables of interest. The effect of clinical experience was thus demonstrated by differences in how these two groups performed on knowledge tests and clinical decision making measurements. The relationship of theoretical and practical knowledge to clinical decision making skills was tested by combining data from the two groups. Process tracing methodology using an interactive videodisc simulation was used to assess clinical decision making behavior. Process tracing was used because it allows 31 32 direct access to what information was obtained to make a decision, what decision was made, and the order in which the information was accessed and the decisions were made. This information, automatically recorded by the computer and placed in log files for each subject, was used to compute a variety of scores reflecting both decision making process and outcome skills as well as to make inferences about what decision strategies had been used in arriving at a choice. Research questions and hypotheses 1. What is the relationship between theoretical knowledge (BKAT score), practical knowledge (CST score), 8 simulation performance (based on all subjects)? There will be positive relationships among BKAT, CST, and simulation proficiency score. 2. Will inexperienced clinicians (student nurses) and experienced clinicians (critical care nurses) differ on the Basic Knowledge Assessment Tool(BKAT) score, Cardiovascular Self-Evaluation Tool(CST) score, 8 simulation performance (measured by proficiency score)? Nurses with at least one year of critical care experience will score higher than students. 3. Are there any recognizable and repeated patterns or paths taken through the simulation? 4. If yes, are these patterns associated with varying levels of knowledge, amount of clinical experience, or simulation performance score? 5. Is there a relationship between simulation process and outcome? Time to complete the simulation will be negatively correlated with the simulation proficiency score. Efficiency scores will be positively correlated to proficiency scores. 33 Data collection Methodology Four instruments were used to collect data: three written questionnaires (demographic profile, basic knowledge test, self evaluation of clinical expertise) and one computerized interactive videodisc simulation. The ngggzapnig_zzgfilg (see Appendix A) was developed by the researcher to collect data on the sample characteristics for descriptive and comparative purposes. Information on potentially relevant variables, e.g. previous computer use, student assignment to patient(s) with dysrthymias, and years of RN clinical experience in critical care, were also included. The profile was reviewed for face validity by a panel of two researchers, one critical care staff nurse, one critical care nurse administrator, and one student. (N=5). Suggested changes in phrasing and terminology to improve clarity were incorporated. The WWII—41 is designed to measure the basic knowledge needed for safe critical care nursing practice (Toth 8 Richey, 1984; Toth, 1984, 1986). It is a 100 item multiple choice, fill in the blank test with seven subscales: cardiovascular, monitoring lines, pulmonary, neurology, endocrine, renal, nutrition, and miscellaneous. BKAT scores are reported as number of answers correct (0-100) and requires approximately 45 minutes to complete. Kinney (1990) in her review of the literature on research in education for critical care 34 nursing concluded that the BKAT was the only instrument identified for assessing knowledge in critical care nursing. The authors of the BKAT have reported reliability data on the four versions. The latest version, BKAT-4, was tested in 1990 on a purposive national sample of 84 critical care nurses with a Cronbach's alpha of .86. Cronbach's alpha for version three was .73 (n=84) in a national random sample of AACN members (Toth, 1986). Earlier versions of the exam had Cronbach's alpha coefficients ranging from .83 to .86 in two groups of nurses (n=100, n=92) and a group of students (n=38) (Toth, 1984, 1986; Toth 8 Richey, 1984). The authors state that content validity has been established through literature review and by a panel of experts in critical care nursing practice, critical care nursing education, and critical care medicine and that construct validity has been established using known group differences. Henry (1989) modified the BKAT cardiovascular subscale by changing the fill in the bank items to multiple choice for ease in scoring and obtained a Cronbach's alpha of .80 in a group of critical care nurses (n=142) for the 100 item test. Only the 31 item cardiovascular subscale was used in this study in order to reduce the amount of time required for subject participation (see Appendix B). Toth 8 Ritchey (1984) have reported a Cronbach's alpha of .81 for this cardiovascular subscale. Reliability will be examined in this study. 35 The W is a 40 item ordinal scale consisting of task statements related to the knowledge and skills required to care for a cardiovascular patient. It was used in this study to provide a self evaluation measure of practical knowledge. It is a modified version of the CST developed by Henry (1989) for her study on clinical decision making. Henry (1989) developed this tool by selecting nursing process based items related to the care of a patient with a dysrhythmia from the list of 200 critical care competencies from the American Association of Critical-Care Nurses CCRN Validation Study (1984, 1988). Henry (1989) created the scale ratings based on the skill acquisition models of Anderson (1982, 1985) and Dreyfus 8 Dreyfus (1986). The subjects are asked to rate their knowledge and skill related to the designated tasks on a scale of 1 - 4. The descriptors for the scale ratings are: 1 no knowledge or skills related to the task, 2 a theoretical knowledge about task, but limited practical skill related to task, 3 = knowledge and skills necessary for task in uncomplicated patient situations and 4 = knowledge and skills necessary for task in complex patient situations. Before analyzing the data, the rating scale was recoded from 1'- 4 to 0 - 3 for ease in interpreting results. Subject 36 ratings of each item were summed for a total score (range of 0 - 120). Henry (1989) reported evidence of content validity and construct validity using known group comparisons as well as reliability (Cronbach's alpha of .98 with n=142). Two additional items, #15 and #31, were added by the researcher to the original 38 item instrument in order to address the aging concepts that were incorporated in the simulation (see Appendix C). Both reliability and construct validity will be assessed in this investigation. The WW1: is a computerized level three interactive videodisc case study simulation. The program entitled "Nursing Care of Elderly Patients with Acute Cardiac Disorders" was recently developed by a special project staff from the American Journal of Nursing with Mary Anne Rizzolo, EdD, RN as project director and Mary Jo Larkin Hall, MA, RN, CCRN as the subject matter expert. At the time of this study, no other copies of this simulation were being used in the mid Michigan area. Two separate case studies were offered in this program; only one of these case studies was used in this study. The individual proceeding through this simulation manages the care of the patient by making decisions about whether to assess or intervene and selecting choices from assessment and intervention menu screens as she/he progress through the case study. The menu of choices on these 37 screens are always the same, but the user must select only those which are appropriate at that point in the patient's care. Specific written, audio, or audiovisual feedback is given for each choice selected. Feedback is in the form of actual clinical information requested or the effect of an action/s selected. Although the program allowed a great deal of flexibility in the sequence of choices that could be made by the user, there was some program sequence control. The user was forced to make certain helpful choices before the program would continue. This provided a common base of information for events that followed. Feedback to the user would indicate that their current choice was inappropriate at this point in time and were requested to make another selection. The subjects were warned that this statement would be appearing, that it was built into the program as an instructional feature that could not be removed, and that it should not necessarily be perceived as negative. The user makes his/her choices by touching the appropriate area on the monitor screen. No keyboarding is used. A detailed description of this case study can be found in Appendix D. Since the simulation was designed for instruction rather than assessment or evaluation, it required some modifications both in structure and how it was used. Each selection made by the user is given a score of +3 to -3. The automatic display of these scores was removed, although the scoring capability was maintained. Approval for this 38 modification was acquired from the simulation producers. As developed, this simulation offers two modes of use: normal and challenge. In the normal mode, users receive feedback and rationale from an expert nurse about appropriate assessments and interventions. In the challenge, or self-assessment mode, no such feedback is given. Only the challenge mode was made available for the subjects. Subjects were instructed not to use the library feature and all complied. A method to track the path (all choices) made by each user was made possible through a printout of the computer log files indicating every touch area used by the subject. The simulation provided data for calculating proficiency, efficiency, omission, and commission error scores. The scoring system was established by the producers of the program (The American Journal of Nursing Company) using a panel of cardiovascular nurse experts (Rizzolo, personal communication). Each choice that a user could make was assigned a value ranging from +3 to -3 depending on how that choice would positively or negatively effect the patient at that point in time. A +3 or + 2 indicates an extremely important assessment or intervention, +1 is an appropriate, but not essential to patient care at that time, a 0 is given to those choices that have little or no effect on the patient, a -1 indicates a decision that will not harm the patient, but may delay more appropriate assessment or 39 intervention, and a -2 and -3 are actions that may cause harm to the patient. Values are also assigned when the user neglects to choose essential assessments or interventions. A final score (the sum of the positive items minus the sum of the negative items) is calculated by the computer and may be accessed at the end of the simulation. The maximum number of points possible was 71, but was reduced to 64 for this study when the 12 lead ECG and defibrillation sections were removed by the researcher from the scoring totals. Interpreting the 12 lead ECG was an optional item in the program. The scoring was such that anyone who selected it received 3 points regardless of responses given. Because this did not accurately reflect the subjects correct or incorrect responses, it was not used in the total score. The defibrillation section caused major difficulties because of the touch screen interface with the graphic depiction of a defibrillator. Subjects could lose many points because of responding too quickly. Because this represented a computer user problem rather than measuring a nursing skill, this section worth four points was eliminated from the total score. The scoring system was the basis for calculating several scores to reflect different dimensions of performance on the simulation. Each subject received a proficiency score, an error of commission score, an error of omission score, and an efficiency score (McGuire, 1976). 4O Prgfiigigngy is the extent to which the subject's decisions agreed with those of the expert decisions. The score is calculated by computing the algebraic sum of the weighted items selected by the subject, dividing that sum by the maximum number of points possible (in this case 64). The resulting decimal fraction is then multiplied by 100 and the proficiency score is reported as a percentage. Effiigiengy is the proportion of decisions that were helpful in the resolution of the problem. It is computed by dividing the number of positively related items that the subject selected by the total number (not values) of the subject's choices. Errgr§_gf_gmi§§1gn is the extent to which the subject failed to make decisions experts regard as important to the resolution of the problem. It is calculated by subtracting the sum of the positively weighted items from the maximum score possible, dividing the result by the maximum score possible, and multiplying by 100 to get a percentage score. Errgr§_9f_ggmm1§§ign is the extent to which the subject made decisions experts regard as harmful to the resolution of the problem. It is computed by dividing the sum of the negatively weighted items by the maximum possible score and multiplying by 100. Forms of reliability assessment such as internal consistency and stability are inappropriate for simulation tests (Dincher 8 Stidger, 1976). Studies of the reliability of clinical simulations have examined the magnitude of three 41 major sources of error variance: variation introduced by the scoring procedure used; interrater variation in scoring simulations; and intercase variation in quality of performance (Swanson, 1984). Since only one case was used in this study, only the first two error variance sources are applicable. Because the computerized scoring system was designed for feedback and not evaluation, the researcher selected a panel of four expert clinicians in cardiovascular nursing to score the simulation as an assessment tool. Since the diversity of opinion among experts precludes scoring by a single individual, use of consensus judgment of a panel of experts seems to be the necessary and appropriate method for key construction (Mazzuca and Cohen, 1982). Selection criteria for this panel included current employment in an adult critical care unit as an RN, at least five years clinical experience in critical care, and recommendations by peers as an expert clinician. Information on these experts can be found in Appendix E. Selection of subjects The two populations of interest for this research study are registered nurses with at least one year of clinical experience in an adult critical care unit and senior students enrolled in a baccalaureate degree nursing program. The subjects for this study were volunteers from the above mentioned populations. Thirty-five registered nurses 42 (RNs) were recruited from three adult critical care units (N=114) at a large tertiary hospital in the mid-Michigan area. Thirty-five nursing students were recruited from a senior class of 72 enrolled at a large mid-Michigan university college of nursing. A power analysis was conducted to determine the probability that the planned statistical analyses would detect statistically significant relationships and differences. Nursing students were used as part of the sample in order to maximize the anticipated effect size. Differences between nursing students (unexperienced clinicians) and RNs (experienced clinicians) on the variables of interest should be greater than using an all RN sample; thus, reducing the sample size required. Results reveal that a sample size of 70 will be adequate (power of 80) to detect a large effect size with an alpha of .05 for all tests planned (Cohen, 1988). Procedure for data collection Data collection was completed during a four day period in March, 1991 for the students and during the month of August, 1991 for the registered nurses. All subjects completed 3 written instruments (Demographic Profile, BKAT, 8 CST) prior to completing the computerized simulation. The majority of subjects completed all instruments during one two-hour session. All subjects were given directions for completing the interactive video simulation via a written 43 sheet prior to completing the simulation (see Appendix F). The researcher closely monitored the first few minutes of the simulation to make sure there were no technical difficulties or procedural questions. WW Approval from the Michigan State University Committee for Human Review of Subjects (UCHRIS) was granted prior to data collection. Additional approval for utilizing nurse subjects was obtained from Sparrow Hospital’s nursing research committee. Anonymity of subjects was maintained by using code numbers instead of names. A list of names and code numbers was maintained by the researcher only for subjects who could not participate in one two hour block. This list was destroyed as soon as the subjects had completed both sessions. Each subject desiring a copy of study results self-addressed an envelope which was kept in a file separate from the subject’s data. Wartime Students were notified of the opportunity to participate in this study by the researcher who made a short presentation at the end of a regularly scheduled class. Written handouts explaining how to sign up were distributed to the class (See Appendix G). Students were assured that their course grade would not be affected by whether or not they participated in the study. A small fee of $10, as compensation for approximately two hours of their time, was 44 offered as encouragement to participate. In addition a lottery system using seven nursing NCLEX review books as prizes was utilized. These incentives to participate were deemed especially necessary since students would need to return to school earlier than usual from spring break in order to participate. The students who returned the sign up sheets were given a reminder phone call one to two days prior to their first session time. Two sessions were needed, one for completing the written instruments and one for the simulation. Often they were sequential, but not always. All subjects did complete the written instruments prior to the simulation and all data collection took place within four consecutive days. Data collection was done in groups of four to six students and supervised by the researcher or a trained assistant. Campus classrooms and a computer lab with six stations was available for data collection. Consent forms were signed at the first session with a copy given to each student (see Appendix H). Data collection was condensed into four consecutive days in order to limit the opportunity for sharing among subjects. Student subjects were verbally requested to refrain from discussing the content of the testing instruments with any of their classmates until data collection was completed. No evidence of information sharing was demonstrated during data collection. 45 MW Individual meetings with the hospital's vice-president for nursing and managers of each of the three critical care units were initiated by the researcher to explain the study and elicit support in recruiting nurse subjects. An explanatory letter of invitation was distributed to each of the 114 registered nurses employed on the three units (see Appendix I for sample letter). Interested nurses completed the tear off form at the bottom of the letter (name, nursing unit, phone number, best time to call) and returned it to the hospital nursing office. The researcher contacted these nurses and set up two-hour appointments for data collection. As stipulated in the letter, returning the tear-off form indicated their consent to participate. If the nurse returned the entire letter rather than the tear off form, a copy of the letter was returned to the subject at the time of data collection. All data collection took place in a private office at the hospital and was monitored by the researcher. Since the simulation requires special equipment (computer, videodisc, touch screen monitor) and only one unit was available in the hospital, only one subject could be tested at one time. However, this was deemed preferable to the alternative of requiring the nurses to go off-site to the university's computer lab. As with the student subjects, nurse subjects were requested to refrain from discussing the content of the testing instruments with any 46 of their colleagues until data collection was completed. Again, no indication of data sharing was demonstrated during the one month period of data collection. A lottery system with monetary prizes was used to encourage participation. The five $100 winners were notified after all 35 subjects had participated in the study. In addition, the subjects earned points in the hospital's clinical career ladder program for participating in the research. WW All subjects completed the written instruments first (profile, bkat, cst) followed by the computerized simulation (each takes approximately one hour). The researcher monitored this procedure, answered procedural questions, and provided some debriefing after the experience. All of the RN subjects (N=35) and all but one of the student subjects (N=34) requested results of the study. Data Analysis and Management A microcomputer was used to build and analyze the data sets using SPSS/PC+. Correlations (Pearson product moment) were used to examine the relationships among simulation proficiency score, BKAT score, and CST score. Multiple regression was used to identify the factors that contribute most to the performance of the nurses on the simulation. Cronbach’s alpha was used to test reliability of the CST and 47 Kuder Richardson 20 was used to test reliability of the BKAT. T-tests using the Bonferroni adjustment of alpha were used to examine construct validity of the BKAT, CST, and simulation. Multivariate analyses of covariance (MANCOVA) was used to examine the effect of knowledge and experience on clinical decision making proficiency with age as a covariate. In addition, printouts of each subject’s decision path through the simulation were available for analyzing. CHAPTER IV RESULTS This chapter is organized into four sections. The demographic characteristics of the study sample are described in the first section. The second section reports the reliability and validity of the instruments from the study sample, and the study hypotheses are tested in the third section. The last section describes the simulation patterns of decision making associated with varying levels of knowledge and clinical experience. Sample subjects Thirty-five senior students from a large mid-Michigan university college of nursing and thirty-five registered nurses (RNs) employed on three critical care units at a large mid-Michigan hospital participated in the research study. Demographic characteristics common to both the student and RN group are presented first followed by a comparison of the two groups. Demographic characteristics specific to each group will then be reported. The sample subjects (N=70) ranged in age from 22 years to 50 years with a M of 30.5 and SD of 8.0. Sixty-two (89%) of the sample subjects were female, 61 (87%) had previous 48 49 hands-on experience with computers, and 44 (63%) subscribe to at least one nursing journal. Table 1 presents a comparison of the demographic characteristics collected from both groups. Table 1. Comparison of Student and Registered Nurse Demographic Characteristics STUDENTS RNs TOTAL (N = 35) (N = 35) (N = 70) n ( % ) n ( % ) n ( % ) Gender Female 32 (91.4) 30 (85.7) 62 (88.6) Male 3 ( 8.6) 5 (14.3) 8 (11.4) Previous computer experience Yes 29 (82.9) 32 (91.4) 61 (87.1) No 6 (17.1) 3 ( 8.6) 9 (12.9) Nursing journal subscription Yes 20 (57.1) 24 (68.6) 44 (62.9) No 15 (42.9) 11 (31.4) 26 (37.1) M (SD) M (SD) M (SD) Min-Max Min-Max Min-Max Age in years 24.5 (5.0) 36.5 (5.7) 30.5 (8.0) 22.0-46.0 22.0-50.0 22.0-50.0 Chi-square analysis of the dichotomous variables of gender [X3 (1 df, n=70) = .46, p = .56] and subscription to nursing journals, (X? (1 df, n=70) =.98, p = .32] revealed no significant difference between the student and RN group. Chi-square analysis of the dichotomous variable, previous 50 hands-on computer experience, revealed that 50% of the cells had an expected frequency of < 5; therefore, Fisher’s exact test was used (p = ~4.E+ 265) revealing no significant difference between the student and RN group. A pooled variance 2 tailed t-test (t = 9.32, df 68, p< .0003) revealed a significant difference in mean age between the two groups. The registered nurse group (M s 36.5) was expectedly older than the student group (M = 24.5). The age difference will be taken into consideration during later testing of research hypotheses. Demographics specific to each group are now addressed. All of the students were enrolled in the last term of a four year baccalaureate nursing program. All had successfully completed an adult medical-surgical course in which theory and clinical experience related to care of the acute cardiac patient was included. Thirty-one (89%) of the students had additional experience working as an extern or nursing assistant in a hospital, and eight of these had been assigned to critical care units as part of that experience. Twenty-four (34%) of the students stated they had been assigned as a student to care for a patient with a dysrhythmia while 11 (16%) had not been assigned. All eight students who had additional experience in a critical care unit stated they had also cared for a dysrhythmic patient as a student. The 35 registered nurses volunteering to participate in 51 this study represented 30% of the total pool of 114 registered nurses employed on three hospital critical care units. Approximately the same number of nurses volunteered from each unit: emergency department (13), intensive care unit (10), and cardiac intensive care unit (12). Table 2 presents the demographic characteristics of the registered nurse group (N = 35) collected from categorical variables and Table 3 presents demographics from continuous variables. The average age was 36.5 years with a mean of 8.9 full time equivalent (FTE) years of clinical experience and a mean of 5.9 FTE years of critical care experience. The majority of the RN group were female graduates of associate degree nursing (ADN) programs over five years ago, worked full time 12 hour day shift, had been employed on their current work unit for at least two years, had taken a critical care course since graduation, were ACLS certified, had attended at least one continuing education program in the last year, had read at least two journal articles in the last month, subscribed to at least one nursing journal, and had over four full time equivalent years experience in critical care. Thirty-two of the nurses had staff positions while three were assistant department managers. However, the assistant department managers reported the following percentages of time in patient care: 100%, 98%, and 35%. (Note: The 100% manager was a full time employee who reported doing her administrative functions on an overtime basis.) 52 Table 2. Demographic Characteristics of RN Group from Categorical Variables (N = 35) n (percent) Critical Care Unit Emergency Department 13 (37.1) Intensive Care Unit 10 (28.6) Cardiac Intensive Care Unit 12 (34.3) Position Staff nurse 32 (91.4) Assistant Department Manager 3 ( 8.6) Shift Days 19 (54.3) Nights 8 (22.9) Mixed evenings 8 days or nights 6 (17.1) Missing data 2 ( 5.7) Length of Shift 12 hour 24 (34.3) 8 hour 1 ( 1.4) Mixed 12 8 8 hour 9 (12.9) Less than 8 hour 1 ( 1.4) Percentage Time Employed Full time 25 (35.7) Part time 10 (14.3) < 10 hours/week 2 (20.0) 10 - 19 hours/week 4 (40.0) 20 - 36 hours/week 4 (40.0) Nursing Education Diploma 4 (11.4) ADN 21 (60.0) BSN 10 (28.6) ACLS Certification Current 23 (65.7) Not current 5 (14.3) Never certified 7 (20.0) CCRN Certification Current 6 (17.1) Not current 1 ( 2.9) Never certified 28 (80.0) Critical Care Course Yes 26 (74.3) No 9 (25.7) 53 Table 2 (cont'd). n(percent) Attendance at Continuing Education Program in Last 12 Months Yes 29 (82.9) No 6 (17.1) Currently Enrolled in College Program No 23 (65.7) Yes 12 (34.3) Nursing Baccalaureate 5 (41.7) Nursing Masters 3 (25.0) Non-nursing Baccalaureate 4 (33.3) Journal Articles Read in Last Month None 4 (11.4) 1-2 15 (42.9) 3-4 8 (22.9) 5 or more 8 (22.9) Table 3. Demographic Characteristics of RN Group from Continuous Variables (N = 35) M SD Min./Max. Age (years) 36.5 5.7 22.0/50.0 Clinical nursing experience (FTE years) 8.9 5.0 1.25/18.25 Critical care nursing experience (FTE years) 5.9 4.0 1.0/15.0 Years employed on current unit 4.9 4.5 .25/17.0 Years since graduation 9.2 5.3 1.0/18.0 The researcher collected data regarding the amount of previous full time experience (in years and months) and the percentage and amount of previous part time experience (in years and months) so that full time equivalent years could 54 be calculated. This was felt to be a more valid measure of clinical experience than just asking number of years. A paired t-test on the means of the critical care experience in number of years (M a 6.8, SD = 4.3) and critical care experience in FTE years (M = 5.9, SD = 4.0) resulted in t = 2.36 with 34 df, p = .024. The mean FTE years was significantly lower than the mean regular years. The researcher can be 95% confident that the mean difference will be between .002 and 1.848 years or 2.4 months and 1.8 years. This difference could be even greater if clinical experience is requested in only years and not years and months. Thus, researchers who wish to gather data related to amount of clinical experience could increase the validity and sensitivity of measurement of clinical experience by using FTE years of clinical experience. Reliability and validity of Study Instruments Reliability refers to consistency of measurement - the extent to which variation in a set of test scores represents systematic differences among individuals rather than sources of error variation (Stanley, 1971). Internal consistency measures were used to assess reliability in the BKAT and CST instruments and interrater reliability related to the scoring of the simulation was used for the interactive video simulation. ”Validity ... refers to the appropriateness, 55 meaningfulness, and usefulness of the specific inferences made from test scores" (APA, AERA, 8 National Council on Measurement in Education, 1985, p.9). Cronbach (1988) argues that categorizing validity into three types, content, criterion-related (concurrent and predictive), and construct, is outmoded and that one should focus on the ”validity argument". Since construct validity is the more general and encompasses the others, it will be the focus of the validity assessment of the three research instruments in this study. The reliability and validity of each study instrument will now be addressed. The Cardiovascular Subscale of the Basic Knowledge Assessment Tool (BKAT) is a 31 item multiple choice test. Subjects recorded their answers on scan sheets which were then scored by a mainframe computer using the Grader III program. The total possible points ranged from 0 to 31. Table 4 shows a descriptive summary of scores on this test. Table 4. Descriptive Summary of BKAT Scores All Subjects Students Nurses (N = 70) (N = 35) (N = 35) Min 10.00 10.00 13.00 Max 30.00 25.00 30.00 Median 21.50 15.00 27.00 Mode 15.00 15.00 28.00 Mean 21.16 16.40 25.91 5.96 3.66 3.49 SD 56 The Kuder Richardson formula (KR20) for item homogeneity was used to calculate a reliability coefficient of 0.86 (n=70). Knapp (1991) recommends interval estimation of coefficient alpha in order to make inferences to the population from the sample. The 95% confidence interval for this alpha is (0.81, 0.91) and is thus statistically significant at the .05 level. The authors of the instrument reported a cardiovascular subscale Cronbach's alpha coefficient of 0.81. Nunnally (1978) reports an alpha of 0.70 to be adequate for this type of instrument. Thus, the instrument as used in this study reflects more than adequate internal consistency. Evidence of construct validity was shown by the comparison of known groups. A 1 tailed pooled variance t- test comparing the means of the RN and student groups resulted in t(68) = 11.13, p < .0003. As expected, the RN group (M = 25.9, SD = 3.49) performed significantly better than the student group (M = 16.4, SD = 3.66). The modified Cardiovascular Self-evaluation Test (CST) is a 40 item rating scale. Subjects recorded their rating for each item on a scan sheet that was then scored by a computer. The original rating scale of 1 through 4 was recoded to 0 through 3 to better reflect a summated summary score for each subject; thus, scores could range from 0 to 120. Table 5 presents a summary of the descriptive statistics for this measurement tool. 57 Table 5. Descriptive Summary of CST Scores All Subjects Students Nurses (N = 70) (N = 35) (N = 35) Min 36.00 36.00 73.00 Max 118.00 100.00 118.00 Median 87.00 69.00 109.00 Mode 116.00 50.00 116.00 Mean 87.79 70.20 105.37 SD 23.05 16.92 12.46 Cronbach's coefficient alpha was 0.98 which is the same as the author reported (Henry, 1989) on the 38 item test. The 95% confidence interval for this alpha is (0.97, 0.99) and is thus statistically significant at the .05 level. This reflects very high internal consistency and perhaps excessive redundancy. However, it was deemed essential by the researcher to include multiple items to assess all four steps of the nursing process as they relate to specific tasks in caring for the cardiovascular patient. Corrected item-total correlations ranged from .19 to .89 with the two new items being .64 and .66. Henry (1989) reported item- total correlations of .45 to .88. Evidence for construct validity of this instrument was gained by using the known groups method. Since the level of measurement for the CST was ordinal, a Mann-Whitney U was performed to compare the mean ranks between the student group (M rank = 19.9) and the RN group (M rank = 51.1). As expected the RN group ranked significantly higher than the 58 student group (U = 65.5, 2 tailed p < .00005) on this self evaluation instrument. Figure 2 shows the CST medians for the student and RN groups. 0 40 80 120 Md : Md : Students RNs 0 = No Knowledge or Skills 40 = Theoretical Knowledge, but limited practical skills 80 = Knowledge 8 Skills in uncomplicated patient situations 120 = Knowledge 8 Skills in complex patient situations Figure 2. Comparison of Student 8 RN CST Median Scores The interactive videodisc simulation used in this study had not been previously tested for it’s reliability or validity as a research assessment tool. It's validity as an instructional program for nursing students had been demonstrated using a pretest posttest design and content validity was established using cardiovascular nurse experts and nursing textbooks (personal communication, Mary Anne Rizzolo). Reliability was assessed in relation to the two major sources of error variance in using clinical simulations as measuring tools: variation introduced by the scoring procedure used and interrater variation in scoring the simulation. The scoring scheme for this simulation involved 59 assigning a value, ranging from a +3 to a -3, to each of the 224 clinical decision points. This scoring system was part of the computerized interactive video simulation program produced by the American Journal of Nursing (AJN) and was automatically recorded for each subject; thus, interrater reliability of scoring each decision point was not an issue. The score for each simulation decision point can be found in Appendix J. The researcher recruited a panel of four local expert clinicians in cardiovascular nursing to score the simulation as an assessment tool for clinical decision making. Rather than using the original +3 to -3 rating scale developed by AJN, these four experts used a simplified rating system to score each of the 224 decision points: + 1 = decisions that would positively impact the patient, - 1 = decisions that would negatively impact the patient, 8 0 = decisions that would have neither a positive or negative effect on the patient. Research has shown that changes in the number of categories into which options are classified and moderate changes in the weights assigned to those categories make little difference in the resulting scores (Bligh, 1980; Norcini et a1, 1983). The computerized scoring system was then recoded from +3 to -3 to this simplified version of +1 to —1. All positive scores (+1, +2, and +3) were assigned a +1, zeros remained the same, and all negative scores (-1, -2, and -3) were assigned a -1. The panel and computer scoring system 60 were then compared. Table 6 shows that there were very few differences between the panel and the computerized scoring. They differed on only 20 out of 224 decision points; percentage of total agreement was 91.1%. A comparison of row and column totals shows that in comparison to the computer program, the experts rated more decisions points as positive (46 versus 31), more as negative (137 versus 136) and less as zero (41 versus 57). Table 6. Comparison of Local Expert and AJN Panel Scoring of Simulation AJN PANEL -1 0 +1 Row Total -1 134 3 137 LOCAL 0 2 39 41 PANEL +1 15 31 46 Column Total 136 57 31 242 Cramer's V for the contingency table in Table 6 is .83, p < .00005. The simulation contains eight major decision points related to whether the nurse should assess or intervene. It is noteworthy that the experts rated these eight major decision points with +1 if the best of the two options was selected while the computer scored the best option with 0. 61 This represented more of a scoring system difference than a substantive difference in the appropriateness of the clinical decision between the local expert panel and the American Journal of Nursing expert panel. If these eight points are excluded from the total, the total percentage of agreement is 94.4%. Topf (1986) reports that there is some consensus among behavioral scientists that a 70% percentage agreement is necessary, 80% is adequate, and 90% is good. Cohen’s Kappa, an interrater reliability coefficient, was employed to adjust for agreement due to chance. The results showed a high degree of interrater agreement (Kappa = .84 for all 224 decision points, and Kappa = .90 for the 216 decision points). Landis 8 Koch (1977) suggest that Kappa statistics of .61 to .80 represent substantial agreement and .81 to 1.00 almost perfect agreement. The researcher concluded that the original AJN panel of experts who scored the program were representative of the larger population and that their scores served as valid criteria for determining the scores of the subjects in this study. A comparison of the local panel and AJN panel of experts score for each decision point can be found in Appendix K. A proficiency score was calculated by computing the algebraic sum of the subject’s decision points, dividing that sum by the maximum number of points possible (64). The resulting decimal fraction is then multiplied by 100 and the proficiency score is reported as a percentage. This score 62 represents the extent to which the subject's decisions agreed with those of the computerized expert decisions. A descriptive summary of these scores is presented in Table 7. Table 7. Descriptive Summary of Simulation Proficiency Scores All Subjects Students Nurses (N = 70) (N = 35) (N = 35) Min -l7.l9 -l7.19 29.69 Max 87.50 60.94 87.50 Median 46.09 32.81 65.63 Mode 39.06 17.19 65.63 Mean 46.25 30.18 62.32 SD 23.31 18.00 14.49 Evidence for construct validity was shown again by use of known group comparisons. As expected, the mean simulation proficiency score for the student group (M = 30.18) was significantly lower than the mean proficiency score of the RN group (M = 62.32) based on a two-tailed pooled variance t of 7.96, df 68, p < .0005. In addition to the proficiency score, the following simulation performance scores were calculated for each subject: commission error score, omission error score, and efficiency score. A comparison of the student and RN scores are found in Table 8. 63 Table 8. Comparison of the Student and RN Simulation Scores All Subjects Students RNs Scores (%) (N - 70) (N = 35) (N = 35) Efficiency Range 36 - 79 36 - 66 46 - 79 Mean 55.39 50.53 60.25 so 8.49 6.79 7.18 99mm158198_Errer Range 9 - 111 36 - 111 9 - 63 Mean 47.88 63.93 31.83 so 22.99 19.34 12.99 9m15919o_Error Range 0 - 13 0 - 13 2 - 9 Mean 5.34 5.89 4.78 so 2.63 3.01 2.07 Failure to achieve 100% proficiency is, by definition, attributable to a combination of errors of commission and omission. The commission error score is calculated by dividing the sum of the negative weights by the total possible score (64) and multiplying by 100. The omission error score is calculated by subtracting the sum of the positive weights from the maximum possible score (64) and multiplying by 100. The efficiency score is reported as the percentage of the subject's decisions that are positive and indicates how well the subject discriminates between decisions that are helpful and those that are detrimental to the patient. It is obtained by dividing the total number of positive Adecisions by the total number of decisions made. 64 The data in Table 8 should be interpreted in light of these score definitions and the biases introduced into the scoring system because the simulation was designed as an instructional program. The subjects were required to make certain selections before they could progress in the simulation. Because the subjects were given positive points for these required decisions, the range of possible scores was restricted. Instead of zero, the minimum number of positive points possible was 55; the obtained scores ranged from 56 to 64. This restriction of range can also be seen in the omission error scores (0% to 9%); the structure of the simulation did not allow for many omission errors. In contrast, the simulation did allow for many commission errors; scores ranged from 9% to 111%. The multiple branching structure of the simulation allowed many opportunities for negative options to be selected. The sum of negative points earned by each subject ranged from -6 to -71. Since the total number of positive points possible was only 64, commission error percentages over 100 occurred. Two-tailed separate variance t-tests were used to compare the mean differences between the students and RNs on the commission error and omission error scores and a two- tailed pooled variance t-test was used for the efficiency scores. Results are depicted in Table 9. Using the Bonferroni adjustment for multiple comparisons, the significance level for each test was reduced to .02 in order 65 to maintain an overall alpha of .05. We can be 95% confident that the mean efficiency score of the RN group is higher than the student group and that the mean commission error score of the RN group is lower than the student group. The mean omission score differences were not significant. Table 9. Mean Comparisons of Efficiency, Commission Error, and Omission Error Scores Student RN t(df) p M (SD) M (SD) Efficiency 50.5(6.79) 60.2(7.18) 5.82(68) <.0005 Commission 63.9(19.33) 31.8(12.99) 8.16(59.5) <.0005 Omission 5.9(3.01) 4.8(2.07) l.81(60.34) .076 Additional information about the simulation experience was collected from the subjects after they had completed the simulation (see form in Appendix L). Four statements about the simulation experience were followed with a Likert type seven point scale from 1 = strongly disagree to 7 = strongly agree. The responses were recoded into three categories for use in Chi-square analyses (1,2,3 = disagree, 4 = neither disagree or agree, and 5,6,7 = agree). A comparison of the student and RN responses for the first statement: "I found the program to be a realistic representation of the nursing care decisions required when caring for cardiac patients." are presented in Table 10. Chi-square test of independence 66 reveals that responses of the students and RNs were not significantly different (X2 = .350, 2 df, p = .840). Because the significance of this result is questionable (67% of the cells had an expected frequency of less than 5), a Mann-Whitney test was used to compare mean ranks. The student mean rank of 38.29 was not significantly different from the RN mean rank of 32.72 (U = 515.0, 2-tailed p = .228). The majority of the subjects (87%) found the simulation to be realistic. Table 10. Frequency Responses of Students and RNs on Simulation Realism Disagree Neutral Agree Row Total Students 1 3 31 35 RNs 2 3 30 35 Column Total 3 6 61 70 (4.3%) (8.6%) (87.1%) A comparison of the student and RN responses for the second statement: "Most of the decisions were easy for me to make.” are presented in Table 11. The chi-square test of independence reveals that responses of the students and RNs were significantly different.(X? = 12.09, 2 df, p = .002). The majority of the RNs found the simulation decisions easy while the majority of students did not find them easy. 67 Table 11. Frequency Responses of Students and RNs on Ease of Simulation Decision Making Disagree Neutral Agree Row Total Students 15 7 13 35 RNs 4 4 27 35 Column Total 19 11 40 70 (27.1%) (15.7%) (57.1%) A comparison of the student and RN responses for the third statement: "I felt confident about most of my decisions." are presented in Table 12. Thirty-four of the 35 RNs were confident in their simulation decisions while only 17 of the 35 students were confident. Chi-square analysis reveals that responses of the students and RNs were significantly different.(X? = 21.17, 2 df, p a .00003). However, the result is questionable since 33% of the cells had a minimum expected frequency of less than 5. The Mann- Whitney test was used to compare mean ranks. The student mean rank of 21.97 was significantly less than the RN mean rank of 49.03 (U = 139.0, 2-tailed p < .00005). 68 Table 12. Frequency Responses of Students and RNs on Confidence in Simulation Decisions Disagree Neutral Agree Row Total Students 11 7 17 35 RNs 0 l 34 35 Column Total 19 11 40 70 (27.1%) (15.7%) (57.1%) A comparison of the student and RN responses for the fourth statement: "I found the program confusing." are presented in Table 13. The chi-square test of independence reveals that responses of the students and RNs were not significantly different (X?== .567, 2 df, p = .753). Only 24 (34%) of the subjects found the program confusing. The following comments were made by students who found the program confusing: "took a while to understand program instructions, I had a hard time following figmgtimgg, I felt it difficult to obtain the information I wanted when the computer didn't want to give it to me, at times took me time to realize I didn't have to do all of the assessment areas". RNs made the following comments related to program confusion: "difficulty operating the simulation, unsure how to get to various steps, wanted a doctor's okay before giving medications (lidocaine), didn’t know exactly what computer sequence was requesting, e.g. what emergency measures included, a little confusing on what emergency 69 protocol was, computer did not take into consideration that you could do two things at once, video skipped around 8 not sure what they wanted next, and not familiar with the different screen options". Table 13. Frequency Responses of Students and RNs on Confusion of Simulation Disagree Neutral Agree Row Total Students 18 6 11 35 RNs 18 4 13 35 Column Total 36 10 24 70 (51.4%) (14.3%) (34.3%) The computer recorded the beginning and ending time (hour:minutes:seconds) for each subject using the simulation. Using this information, the researcher calculated the amount of time each subject required to complete the simulation. Simulation completion time for all subjects ranged from 31 to 57 minutes with Md = 38, Mo = 34, M = 38.6, and SD = 5.0. Time to complete the simulation has a moderate negative correlation with the simulation proficiency scores (r = -.44, 2-tailed p <.0005). As the subjects took more time, their performance on the simulation decreased. Subjects who were able to make the best clinical decisions in the simulation took less time. A comparison of 70 the means with a two-tailed separate variance t-test (t = 2.75, df 59.74, p = .008) provides evidence that the students (M a 40.2, SD = 5.7) required a significantly longer period of time to complete the simulation than the RNs (M a 37.0, SD = 3.8). Although statistically significant, a difference of only 3 minutes has very little practical significance. The 95% confidence interval for this mean difference is .9 to 5.5 minutes. It is interesting to note that as a group the students were much more varied in the amount of time for completion than the RNs. The greater variability of the student group compared to the RN group was also true on the simulation proficiency scores and CST scores. Hypotheses Testing The presentation of results in this section will begin with the testing of relationships among the study variables and will be followed by the testing of differences between the clinically inexperienced student group and the experienced Registered Nurse group. W The researcher used correlational analyses to examine the predicted linear relationships between theoretical knowledge (BKAT), and practical knowledge (CST) on clinical decision making skill (SIMULATION). Pearson Product Moment correlation coefficients were obtained using scores from the 71 BKAT, CST, and SIMULATION instruments. As shown in the correlation matrix in Table 14, all the variables are significantly correlated, p < .0005. As theory predicts, the BKAT scores reflecting theoretical knowledge were positively related to clinical decision making SIMULATION proficiency scores (r=.63) and the CST scores reflecting practical knowledge were positively related to clinical decision making SIMULATION proficiency scores (r=.47). In addition, the BKAT and CST scores are positively related (r=.73). Scatterplots confirm the linearity of the relationships (Appendix M). If the variables are examined individually, the BKAT is more highly correlated than the CST to the Simulation Proficiency Score. That is, the BKAT explains 40% (r@=.397) of the variability in the SIMULATION proficiency score while the CST (r@=.221) explains only 22%. Additional analyses were done to investigate how these variables work together to explain variation in clinical decision making ability. Table 14. Correlation Matrix of BKAT, CST, SIMULATION Scores (N I 70) SIMULATION BKAT CST SIMULATION 1.00 BKAT .63* 1.00 CST .47* .73* 1.00 * 1 tailed, p < .0005 72 These correlations were analyzed further using multiple regression with Simulation Proficiency Scores as the dependent variable. Aaronson (1989) recommends that when using a theoretical model of relationships among variables, the variable furthest from the dependent variable should be entered first into a multiple regression. Based on Anderson's theory of cognitive skill acquisition, the researcher hypothesized that theoretical knowledge combined with experience predicts practical knowledge and theoretical knowledge combined with practical knowledge predicts clinical expertise in decision making. Theoretical knowledge (BKAT) meets this criteria of being furthest from the dependent variable and was entered as the first predictor in a regression equation using SIMULATION proficiency as the dependent variable. This resulted in an R2 of .394, Adjusted R2 of .385, and a significant BKAT slope coefficient of 1.57 (t = 6.64, p < .00005). Practical knowledge and skills (CST) was added as a second predictor. This two predictor model was significant, F (2,67) = 21.80, p < .00005. While the BKAT slope coefficient of 1.50 remained significant, the CST slope coefficient of .025 was not significant (t=.28, p=.78). The resulting regression equation revealed that as a set, BKAT and CST scores explain 39% (R2 .394, Adjusted R2== .376) of the overall variance in SIMULATION proficiency scores. CST did not make an additional contribution to the explanation of simulation 73 proficiency score variance over and above BKAT. There are many reasons why the parameter estimate of CST may not be significant in this regression model other than CST not being a contributing factor to SIMULATION Proficiency Scores. Restricted variance in CST scores (SD = 23.1, M = 87.8) and a curvilinear relationship between CST and SIMULATION proficiency scores (scatterplot shows a positive linear relationship) were ruled out. Multicollinearity could not be ruled out. The two predictor variables, BKAT and CST, are highly correlated (r = .73). The overlap in variance explained by these two variables could prevent CST from entering the regression equation because of too little unique variance. This is further reinforced by realizing that if BKAT and CST were both providing unique contributions to explaining the dependent variable, R2 would be .61 (.39 BKAT + .22 CST). Since R2 is only .39, BKAT and CST must share some common variance in simulation proficiency which is being credited to BKAT. Since only 39% of the variance was explained by this regression model, it is apparent that relevant variables are missing from the equation. The researcher added clinical EXPERIENCE a predictor variable. This variable entered the equation as a dummy variable with students = 0, and RNs = 1. Although this three predictor model (clinical EXPERIENCE, BKAT, and CST) explains 51% (R2 = .513, Adjusted R2 = .491) ‘of the variance in SIMULATION proficiency scores; only 74 clinical EXPERIENCE has a significant slope coefficient (b = 19.26, t= 4.02, p=.0002). The slope coefficient for CST (b= -.196, t= -1.54, p=.128) and BKAT (b= -1.84, t= 1.74, p=.087) are nonsignificant. Both the CST and BKAT slope coefficients become negative and BKAT becomes nonsignificant in this regression model, a common symptom of multicollinearity. It is interesting to note that if clinical EXPERIENCE is used as the only predictor of SIMULATION proficiency scores, it explains 69% of the i variance. I In order to confirm the existence of multicollinearity, three regression equations were produced by regressing each of the three independent variables on all the other independent variables. The results are shown in Table 15. Table 15. Regression Models Using Independent Variables PREDICTORS DEPENDENT VARIABLE Rz BKAT & EXPERIENCE CST .62 CST & EXPERIENCE BKAT .67 CST 8 BKAT EXPERIENCE .72 High multicollinearity is recognized by an R2 that approaches 1.0. Note the high R2 for the last model in Table 15. Seventy-two percent of the variance in clinical EXPERIENCE is being explained by CST and BKAT. This 75 relationship is likely to produce regression models with unstable predictors as demonstrated in this study. Examination of the residuals from this three predictor regression equation was done to evaluate some of the assumptions required for regression analysis. A histogram of the residuals revealed an approximately normal distribution. A scatterplot of the residuals and estimated values from this regression appear to be randomly scattered with no curve or funnel pattern. The plot suggests that the constant variance and linearity assumptions are both reasonable for these data. Differences_netTeen_§rcnn§ Multivariate analyses of covariance (MANCOVA) was used to examine the differences between the performances of the inexperienced STUDENT group and the experienced RN group on the SIMULATION, BKAT, and CST. Because the mean age of the student and RN group was significantly different, age was included in the analysis as a covariate. Age is significantly correlated with the three dependent variables, SIMULATION (r = .58), BKAT (r = .60), and CST (r = .60) with p < .0005 for all. The Hotellings statistic was used to test the equality of the three dependent means (SIMULATION, BKAT, CST) with the two independent samples (students and RNs). The result of Hotellings = 1.35, approximate F '(3,65) = 29.17, p < .0005 indicates that the null hypothesis 76 of no differences between the students and RNs on the three dependent variable measures can be rejected when age is held constant. Univariate F tests were then performed on each dependent variable with the results shown in Table 16. Table 16. Analysis of Covariance* for Effect of Experience on Simulation, BKAT, and CST Scores VARIAELE .MS MS ERBQB F** D SIMULATION 5836.63 285.37 20.45 < .0005 BKAT 690.16 12.97 53.20 < .0005 CST 8596.31 223.51 38.46 < .0005 * age as the covariate ** df = 1,67 for all tests With age held constant, the mean scores of the student and RN groups were significantly different for all three measures (SIMULATION, BKAT and CST) with the RNs performing better than the students. The means, standard deviations, and 95% confidence intervals can be seen in Table 17. 77 Table 17. 95% Confidence Intervals for Students and RNs on SIMULATION, BKAT, and CST Factor MlSD) N 95% CI SIMULATION Students 30.18(19.00) 35 23.65, 36.70 RNs 62.32(14.50) 35 57.34, 67.30 BKAT Students 16.40(3.66) 35 15.14, 17.66 RNs 25.91(3.49) 35 24.71, 27.11 CST Students 70.20(16.92) 35 64.39, 76.01 RNs 105.37(12.46) 35 101.09,109.65 The assumptions for MANOVA were examined and met. Univariate homogeneity of variance tests for each of the dependent variables (SIMULATION, BKAT, CST) and covariate (AGE) supported the equality Of the variances. Bartlett-Box F(1,l3872) was 2.43, p = .119 for SIMULATION; 3.09, p = .079 for CST, .070, p = .792 for BKAT; and .473, p = .492 for AGE. A multivariate test for homogeneity of the variance- covariance matrices of the two groups supported equality (Boxs M = 16.05, F (10,22106) = 1.50, p = approx. 0.131). Stem and leaf displays and normal probability plots of the dependent variables showed approximate normal distributions. A three-way analysis of covariance was performed on the simulation proficiency scores in order to examine the independent variables (BKAT, CST, and clinical experience) for interaction effects. Age was again used as a covariate. CST was recoded into two groups using a score of 80 as the 78 cutoff. Eighty would be the score of a subject who rated each of the 40 items on the CST with a 2 (2 = Knowledge 8 skills in uncomplicated patient situations). BKAT was recoded into two groups using a score of 21 (70% of the maximum score of 30) as the cutoff. Clinical experience was divided into two groups, with inexperienced = students and experienced = RNs. The results of this analysis, as seen in Table 18, shows that one of the interactions was significant, clinical experience and CST. Table 18. Analysis of Covariance: Effect of Clinical Experience, CST, 8 BKAT on Simulation Proficiency Source 88 DF MS F o Covariate AGE 12530.60 1 12530.60 48.45 <.0005 Main Effects EXPERIENCE 2756.33 1 2756.33 10.66 .002 BKAT 61.07 1 61.07 .24 .629 CST 558.57 1 558.57 2.16 .147 2-way Interactions EXP. 8 BKAT 737.72 1 737.72 2.85 .096 EXP. 8 CST 1381.56 1 1381.56 5.34 .024 BKAT 8 CST 149.22 1 149.22 .58 .450 Explained 21451.26 7 3064.47 11.85 <.0005 Residual 16036.04 62 258.65 Total 37487.31 69 543.29 A contingency table of cell means for the two interacting factors, clinical experience and CST, is found 79 in Table 19. For the students, the mean simulation proficiency score decreases from 34 to 21 when CST changes from low to high. For the RNs, the mean simulation proficiency score increases from 30 to 63 when CST changes from low to high. Table 19. Cell Means for Clinical Experience and CST Inexperienced Experienced Students Registered Nurses Low CST 34.44 29.69 High CST 20.88 63.28 WW Different problem-solving patterns can be identified using the simulation scores (McGuire, Solomon, 8 Bashook, 1976). The various patterns of scores reflect four different approaches in assessing and meeting patient needs in this simulation: 1) competent, 2) constricted, 3) undiscriminating, and 4) random. These four decision making patterns were identified by plotting the proficiency scores and commission error scores as shown in Figure 3. 80 1 1 1 1 1 1 .1 1 1 1 1 1 1 1 87.54 R COHPETEHT RAHDOH RR APPROACH APPROACH R H a 45 H I 4 R! R N0 RRB P R! r R 0 RSSR f R S SS i 52.51 R S R c i SRRR e R S S n SSSSS c 35 54—5 y COHSTRICIED R S; UHDISCRiHiHATiHO APPROACH S S S APPROACH S H I 0 S H I 21 c S O 17.51 § * r S e s 01 S S S S -17.5~ S ~ érzl'ul'so'ri'ct'iol' 0 16 32 48 64 80 96 112 Commission Error Scores S: Students R: RHs S: Multiple occurrence Figure 3. Decision Making Patterns of Students and RNs The subjects who use a competent approach are both thorough and discriminating. They correspond closely to the criterion group (i.e. they select most of the choices that the criterion group regards as clearly indicated and avoid most of the choices classified as harmful). They have moderate to high proficiency and efficiency scores with few errors of either omission or commission. Those who use an undiscriminating or "shotgun" approach make multiple 81 undifferentiated choices and decisions. They have very low efficiency scores and moderate to low proficiency scores usually combined with many errors of commission and few errors of omission. Subjects using a constricted approach have high efficiency scores and moderate to low proficiency scores with few errors of commission and many errors of omission. Those using a random approach appear to be selecting Choices and making decisions indiscriminately. They make many helpful and harmful decisions and neglect many indicated choices. The data in Figure 3 indicate that the majority of the 35 RNs (91%) used a competent approach while a majority of the 35 students (54%) used an undiscriminating approach. It was not surprising to find only a few subjects classified as random or constricted since the structure of the simulation prevented many errors of omission. Three students and one RN used the random approach while no one used the constricted approach. It is apparent that different patterns of decision making were used by the clinically experienced RNs than by the inexperienced students as they progressed through the simulation. Would these same patterns emerge for varying levels of theoretical knowledge (BKAT scores) and practical knowledge and skills (CST scores)? Figure 4 is a plot of the proficiency and commission error scores for the high and low BKAT scorers. 82 1 1 1 1 1 1 1 1 1 1 1 1 1 87.5- CGPETEHT RAW APPROACH APPROACH U U I 30 U I 1 U U L I 15 L I 3 U 70- wu P W r U o ULLU f U L LL i 52.5 U S U c i LLLU e U L L n LULI.S c 35 L<—L y MSTRIC‘I’ED U LL MiSCRiHIHA‘I’iHO APPROACH S U L APPROACH S U I O U m b c L I 0 L L I 17 o 17.54 L L r L e s 04 L L L U -17.5 L - i'zl'd'so'riTai'wl' 0 16 32 48 64 80 96 112 Conission Errors U:Upper Half BKAT Scorers L:Louer Half BKAT Scorers S:Hultiple occurrence (Upper 8 Lower) Figure 4. Decision Making Patterns of High 8 Low BKAT Scorers The majority of the subjects (30 out of 35) who scored high on the BKAT used the competent problem solving approach while only four used the shotgun approach and one used the random approach. About half of the subjects who scored low on the BKAT used the competent approach while the other half used the shotgun approach. Figure 5 is a plot of the proficiency and commission error scores for the high and low CST scorers. 83 1 1 1 1 1 1 1 1 1 1 1 1 1 87.5 U COOKED" m % uu APPROACH APPROACH u u - 29 u - 1 u u L . 16 L - 3 u 70‘ LUS 9 us r U o ULLU f u L LL 1 525 000 1 C i LUUU e U LL n LLLLS c 35 Ld—L y COHSTRICTED 0 LL UHDiSCRiHiHAiiHO APPROACH s L s APPROACH s u-o L u-s c LIO L LI16 o 17.5. L r I. C 8 o. L u u L -17.54 L F ' 8 T 21 T7 46 ' so 1 71 41 88 ' 101 1 o 16 32 4s 64 so 96 112 Couission Errors Owner Half CS! Scorers L:Louer Half CS? Scorers S:Hultiple occurrence (Upper 8 Lower) Figure 5. Decision Making Patterns of High 8 Low CST Scorers The pattern is very much like that of the high and low BKAT scorers. The majority of the subjects (29 out of 35) who scored high on the CST used the competent approach while only 5 used the undiscriminating approach and 1 used the random approach. Again, about half of the subjects who scored low on the CST used the competent approach and half used the undiscriminating approach. In summary, the competent approach was used most frequently by RNs and high scorers on the BKAT and CST. The 84 undiscriminating and random approach were used most frequently by students and low BKAT and CST scorers. CHAPTER V DISCUSSION This chapter begins with discussion of the study findings followed by overall significance and limitations. Implications for nursing education, nursing practice, and future research will then be addressed. Findings The study findings will be presented as they relate to each of the research questions. An additional section on miscellaneous findings will be included. 1. What is the relationship between theoretical knowledge (BKAT score), practical knowledge (CST score), 8 clinical decision making (simulation proficiency score)? Correlational analyses supported the researcher's hypothesis of positive relationships between all three study variables: theoretical knowledge, practical knowledge, and clinical decision making. All were statistically significant (1 tailed, p <.0005). The BKAT scores (r = .40) were more highly correlated than the CST scores (r = .22) to the simulation proficiency scores. In addition, the BKAT and CST scores were very highly correlated (r = .73). According to Dreyfus' skill acquisition theory (Dreyfus 8 Dreyfus, 1986), theoretical knowledge and practical 85 l V 4 86 knowledge both contribute to expertise in clinical decision making as revealed in this study. Although they do not suggest relative amounts of their contribution, Dreyfus 8 Dreyfus do conceive of theoretical "knowing that" and practical "knowing how" knowledge as two different types of knowledge. If the BKAT and CST were indeed measuring these two different constructs, the correlation between them is disappointingly high. Anderson’s two stage theory of cognitive skill acquisition would allow for this moderately high correlation since the second stage of practical knowledge would be based on the first stage of theoretical knowledge (Anderson, 1985). Multiple regression was used to investigate how these variables (BKAT 8 CST) work together to explain variation in clinical decision making ability. This two predictor model explained only 39% of the variance. BKAT was the only significant predictor with CST not providing any unique explanation of variation in simulation performance. When clinical experience was added as a dichotomous predictor (student or RN), R2 increased by 12% for a total R2 of .51. However, because of multicollinearity, only clinical experience was significant in this model. While checking for multicollinearity, CST was regressed on BKAT and Clinical experience for an R2 of .62; i.e. BKAT and Clinical experience explain 62% of the variation in practical knowledge. Although substantial, skill acquisition theory 87 would predict a higher percentage of explanation since practical knowledge is considered a meld of theoretical knowledge and clinical experience. An interaction effect of CST and clinical experience, discovered using ANCOVA, provides a possible explanation of the difficulties encountered in this regression analysis. This interaction will be discussed in the next section in response to the second research question. Considering that only 51% of the variance in clinical decision making was explained by the three predictor model of theoretical knowledge, practical knowledge, and clinical experience, specification error may be present. Additional decision maker characteristics that were not studied and that could affect decision making include attitude, self esteem, self confidence, stress level, and self efficacy. It is interesting to note that many times during the simulation, the researcher Observed many of the registered nurses tapping their fingers or feet and appearing very anxious to intervene. In fact, some of them would say what nursing actions they wanted to take before being given a chance to respond. No such activity was noted by the students. Although specific attitudinal Characteristics were not measured in this study, self confidence and self efficacy may have been measured in part by the self evaluation of practical knowledge (CST). Further discussion of findings on practical knowledge will be presented under 88 research question two. Another explanation of the low predictive power of the regression equation may be that the task was not really held constant. The contextual cues seen by one subject may have been missed by others; thus allowing the characteristics of task to enter into the equation. The registered nurses did find the simulation decisions to be significantly easier (Mann-Whitney U - 139.0, 2-tailed p <.00005) than the students. The registered nurses were also more confident.(XF a 12.09, 2 df, p = .002) in their decisions than the students. 2. Will inexperienced clinicians (student nurses) and experienced clinicians (registered nurses) differ on the Basic Knowledge Assessment Tool (BKAT) score, Cardiovascular Self-Evaluation Tool (CST) score, 8 simulation performance (measured by proficiency score)? Multivariate analyses of covariance was used to examine the differences between the performances of the student and RN group on the Simulation, BKAT, and CST. Since the mean age of the RNs was significantly greater than the students, age was used as a covariate. The RN group performed better than the student group on all three measures with the differences being both practically and statistically different. This finding supported the researcher's hypothesis that nurses with at least one year of critical care experience will score higher than student nurses. It also furnishes support for construct validity of all three study instruments. Although the students and RNs differed on each of the 89 variables (simulation performance, BKAT, 8 CST), MANCOVA provides no way to check for interaction effects. A three- way analysis of covariance was performed and revealed a significant interaction effect of clinical experience and CST. For students, the mean simulation proficiency score decreases when CST scores increase; while for RNs, the mean simulation proficiency score increases when CST scores increase. This difference between students and RNs may be explained by the fact that the CST was a self evaluation instrument. Abbott and colleagues (1988), in a study of student self evaluation of clinical performance, reported that students often rate themselves lower than peers or instructors. Phillips (1979) found "weaker" students do not see themselves realistically. A combination of these two findings could explain why clinically skilled students may underrate themselves and lesser skilled students may overrate themselves on the CST self report questionnaire. A negative correlation between CST and simulation performance (r = -28, p .10) for the student group (N = 35) in this study supports this hypothesis. In contrast, self evaluation by registered nurses has been found to correlate positively with performance on simulations. Holzemer et al (1981) found self evaluation of clinical skills by RNs to be positively correlated with PMP simulation performance (r = .23, p < .05) as well as with knowledge exam scores (r = 22, p <.05) and colleague evaluations (r = .39, p <.01). Using 90 the CST, Henry (1989) found self evaluation of clinical skills by RNs to be correlated with only one of four simulations (r = .32, p = .008). Findings related to research question three and four will be discussed next. 3. Are there any recognizable and repeated patterns or paths taken through the simulation? 4. If yes, are these patterns associated with varying levels of knowledge, amount of clinical experience, or simulation proficiency score? Four different decision making patterns were identified by plotting the proficiency simulation scores by the commission error scores: 1) competent (thorough and discriminating), 2) undiscriminating ("shotgun” approach), 3) constricted, and 4) random. Fifty-four percent of the students used the undiscriminating approach and 91% of the RNs used the competent approach. This coincides with an expectation that the RNs would perform better on the simulation. The decision making pattern of the RNs closely reflected the criterion group; i.e. they made many choices in the simulation that positively affected the patient while neglecting items that would harm the patient or delay needed assessment or intervention. The competent pattern reflects subjects who can set priorities. According to Benner’s five stages of clinical expertise, they would be classified as stage 3, competent Clinicians (Benner, 1984). Although 37% of the students utilized the competent approach, more of 91 than (54%) used the undiscriminating approach. The undiscriminating decision making pattern resembles the behavior that a novice or advanced beginner would exhibit while making clinical decisions. Benner (1984) describes these first two stages of clinical expertise as ones in which the person applies learned rules without regard to priorities. Only a few subjects were classified as random in their decision making approach and none as constricted. This may have been an artifact of the simulation structure rather than non use Of these two patterns. The simulation was designed so that errors of omission were very restricted and thus, patterns based on these errors were inhibited. These decision making patterns were also associated with varying levels of theoretical knowledge (BKAT scores) and practical knowledge (CST scores). Subjects who scored high (above the median of 21) on the BKAT and high (above the median of 87) on the CST used the competent approach while those who scored low used the undiscriminating approach. The last research question will now be addressed. 5. Is there a relationship between simulation process and outcome? In addition to decision making patterns, two other types of information were obtained relative to the clinical decision making process: time to complete the simulation and an efficiency score. The researcher's hypothesis, that 92 time to complete the simulation is negatively correlated with the simulation proficiency score, was supported (r = - 44, p < .0005). Research on novice-expert differences in problem solving has provided much support for the notion of experts being quick and accurate in their judgments (Glaser 8 Chi, 1988). This proved to be the case in this simulation. The ”expert" nurse was able to move more quickly through the simulation than the "novice" student (t = 2.75, df 59.74, p= .008). The efficiency score reflects how well the subject discriminates between decisions that are helpful and those that are detrimental or have no effect on the patient. As expected the students had significantly lower efficiency scores (M - 50.5, SD = 6.79) compared to the RNs (M = 60.2, SD = 7.18). As novices, with little clinical experience, the students were unable to distinguish which choices were best in the simulation situation. For example, they would select all or most of the assessment items before deciding to intervene. This is not surprising since comprehensive assessments are a highly valued part of the nursing process currently taught in the baccalaureate program attended by the students. 6. Miscellaneous findings. Because of the structure of the multiple choice exam (BKAT) and the simulation choices, it was possible to 93 directly compare the subjects responses to two similar items. Ninety percent of the subjects correctly identified the administration of oxygen as an appropriate intervention for angina on the BKAT, but only 73% selected administration of oxygen as an intervention for angina in the simulation. Several explanations for this difference are plausible. In the simulation, the subject was required to identify the presence of angina as well as the appropriate intervention; while on the BKAT, angina was already identified as part of the question stem. Thus, the two types of tests (multiple choice exam and interactive videodisc simulation) were requiring difference levels of knowledge to answer what appeared to be a similar question. Since all subjects were presented with relevant cues for identification of angina, these cues were either not recognized as angina or not prioritized correctly. Since multiple audiovisual and written cues were available in the simulation, attending to irrelevant cues could have masked the problem. On the second item, 80% of the subjects identified ventricular fibrillation from an electrocardiogram (ECG) strip on the BKAT; while 90% of the subjects identified ventricular fibrillation from the ECG strip on the simulation. It is possible that the simulation provided subjects with additional cues to the identification of this dysrhythmia, e.g. visual picture of the patient not responding to the nurse and the nurse palpating the carotid 94 artery and saying "no pulse". These two item comparisons provide an example of how situational or contextual cues can either enhance or hinder accurate clinical decision making in videodisc simulations. Although these item comparisons demonstrate that the written multiple choice exam and the interactive videodisc simulation were assessing different knowledge and skills, further study would be needed to capture the uniqueness of each tool. overall Significance The significant contributions of this investigation pertain to two areas: 1) increased understanding of clinical decision making as a cognitive skill, and 2) support for the reliability and validity of the three research instruments: BKAT, CST, and computerized interactive videodisc simulation. This study has provided support for the positive relationship between two types of knowledge (theoretical and practical) and clinical decision making as measured by a simulation proficiency score. Significant correlations of the theoretical based knowledge examination (BKAT) and self evaluation of practical knowledge (CST) with simulation performance supported findings of Holzemer et al. (1981). Although Henry (1989) found a significant correlation with BKAT scores and simulation performance, she was unable to find a significant relationship between CST scores and 95 simulation performance. The explanation for Henry’s nonsignificant finding may relate to the type of simulation used for measuring decision making. Henry’s computerized simulations focused on one of the nurse's more medically dependent roles (the recognition and intervention for specific dysrythmias) while this study and Holzemer's study used case study simulations that included both dependent and independent nursing functions. The effect of clinical experience on clinical decision making was revealed by comparing inexperienced students with experienced registered nurses. Registered nurses performed significantly better on all three study measures: theoretical exam, practical self evaluation, and clinical simulation. These results support findings in previous novice-expert studies in nursing (Benner, 1984; Benner 8 Wrubel, 1982; Tanner et al., 1987; Holden 8 Klinger, 1988). In addition, these study findings support the idea that clinical decision making expertise is acquired through instruction and experience; a thesis of cognitive skill acquisition as proposed by Dreyfus 8 Dreyfus (1986) and Anderson (1982) and applied by Benner (1984). The relationship between clinical decision making process variables (efficiency scores and time to complete the simulation) and outcome (proficiency) scores were strongly supported (r I .87, p < .0005) and (r = - 44, p <.0005) respectively. A process and outcome relationship was further supported by the finding of different decision 96 making patterns for varying levels of simulation proficiency, theoretical knowledge, and practical knowledge. More generally, this study has provided evidence to support predictions based on information processing theory. Three characteristics of the decision maker (theoretical knowledge, practical knowledge, and clinical experience) were shown to be significant determinants of decision making behavior when applied to a specific task. This investigation provided evidence of reliability and validity of all three research measurement tools. Consistent with previous research by the developers of the BKAT (Toth, 1984, 1986) and Henry (1989), internal consistency reliability for the BKAT was adequate (KR20 = .86). Cronbach’s alpha was .98 for the CST which was consistent with its developer (Henry, 1989). Support for the validity of the BKAT and CST was demonstrated by its ability to discriminate between groups expected to be different, i.e. the students and the critical care nurses. An unexpected finding, and one not previously reported in the literature, was the interaction effect Of clinical experience and self evaluation (CST) on simulation performance. In the student group, CST and clinical simulation scores were negatively correlated and in the RN group, CST and clinical simulation scores were positively correlated. Special attention was given to the scoring of the 97 interactive videodisc simulation since this instrument had never been used as a research tool. A local panel of four expert cardiovascular nurse clinicians scored the 224 simulation decision points according to a 3 point scale (positive, zero, and negative) and these ratings were compared with the computerized scoring developed by the simulation producers. The total percentage agreement was 91% and Cohen's Kappa for interrater reliability was .84. The researcher concluded that the developers panel of experts was representative of the larger population and that their scores served as valid criteria for determining the scores of other subjects. It can be inferred that the simulation possesses construct validity since it was designed to assess ability to assess and manage care of an acute cardiac patient and since simulation scores showed that RNs possessed this ability much more than did students. Limitations Threats to internal and external validity will be described as they relate to the correlational design of this study. History, maturation, and testing were threats to the internal validity. History must be considered since it was possible for subjects to have been introduced to the BKAT or interactive videodisc simulation prior to participating in this study. Since the simulation was not available in the mid Michigan area and the BKAT was not being used by the area hospitals or educational institutions, the chances of 98 prior exposure were small. However, subjects were verbally questioned regarding prior exposure to the exam or simulation with only one RN indicating that she "thought she had taken the full 100 item BKAT several years ago". Since she stated that the BKAT items were not familiar, she was maintained as a subject. Risk of maturation was present since all subjects did not complete all study instruments in one sitting. For some of the subjects there was a small window of time for increased knowledge between completing the written instruments (BKAT and CST) and completing the computerized videodisc simulation. The length of time between data collection periods for students was very short, ranging from a few hours to three days. Students were verbally reminded not to prepare for the simulation and not to share any information about the study with classmates or critical care nurses in the community until after the study was complete. Since data collection occurred during registration week of Spring term, the students received no formal nursing education that would differentially affect their response in this study. All but one registered nurse completed the data collection in one sitting. The nurses were verbally reminded by the researcher of the extreme importance of not sharing contents of the study instruments with their colleagues until all testing was completed. Since the nurses were recruited from three units in the same hospital 99 and data collection occurred over the period of a month, maturation for the nurse group was indeed a threat. Casewise plots on the residuals for regression analysis using the BKAT and CST as predictors of simulation performance scores showed no pattern. Since the cases were sequenced by order of testing, these results are indicative of no interaction between cases. Testing was the third internal validity threat. In order to minimize the effect of testing, random ordering of the three tests for each subject would be ideal. However, because of the special equipment need for the interactive videodisc simulation, this was not possible. All subjects did complete the BKAT and CST prior to completing the simulation; so if cuing effects occurred, all subjects were exposed. Since sample selection was volunteer and not random, generalizability of the findings is limited to groups of critical care nurses and baccalaureate nursing students with the same characteristics as the sample. In addition, the findings must be interpreted based on the limitations of using a simulation to measure clinical decision making skills. Although interactive videodisc simulation offers a medium that can closely approximate reality, it is not yet known to what extent the cognitive processes used in simulations are the same as those used in actual practice. 100 Implications for nursing Education and Practice This study has provided support for the validity of this interactive videodisc simulation as an assessment tool for measuring clinical decision making in care of the acutely ill cardiac patient. Because this type of instrument can measure both process and outcome, it could be used in several different ways: as a self assessment for nursing students or novice critical care nurses, as a test of clinical decision making competence within the context of a program curriculum, as an evaluation of beginning skills on a critical care unit, or as a pre/post instrument for a educational cardiac care module or experience. Clinical preceptors and clinical instructors may find this a welcome addition to the difficult task of evaluating clinical decision making skills. However, caution must be observed in using simulations for evaluation. Since research supports the task specificity of clinical decision making, a variety of simulations will be needed to assess competency. In addition, criterion validity of interactive videodisc simulations needs to be established. It is hoped that the positive results of this study will encourage the production, study, and use of many more interactive videodisc simulations. The National Council of State Board Licensing Examination for Registered Nurses is currently investigating the use of computerized interactive videodisc simulation as a means to test decision making on licensure 101 examinations. This investigation has illustrated some of the difficulties in using a simulation designed for teaching as an assessment tool. This researcher would recommend the development and use of separate simulations for instructional and evaluative purposes. The type of feedback needed for testing in simulations, e.g. the real consequences of choices made without evaluative remarks, may not be appropriate in the teaching mode. For example, in the case of Mr. Talbert, the consequences of inappropriate assessments and/or interventions would have led to major complications and/or death. These consequences could have been depicted rather than a statement indicating that the selected alternative was inappropriate and why. Since practical knowledge and clinical experience are important in the development of clinical decision making expertise, the use of interactive videodisc simulations for instruction would be beneficial in providing a variety of safe patient situations in which learners could practice. These simulation experiences could be used in preparation for or concurrently with on site clinical experiences for nursing students or as orientation and preparation for registered nurses transferring to a nursing specialty area that is unfamiliar to them. 102 Implications for Future Research Specific follow up research needs to address the same variables of theoretical knowledge, practical knowledge, and clinical experience using a large all RN sample. This will allow the researcher to test more than two levels of clinical experience. At least the first three stages of clinical expertise as identified by Benner (1984) could be used in Classifying subjects. Until technology advances to the level of virtual reality, it may not be possible to assess the stages of proficiency and expert using simulations. Modification of the CST to measure practical skills as they relate to Benner's stages rather than exclusive use of the nursing process may be beneficial. Lastly, a greater emphasis on collecting and analyzing more process oriented data would be recommended to, not only help validate the total performance score, but to better understand the construct of clinical decision making. Continued study of the role of computerized interactive videodisc simulations in both the assessment and teaching of clinical decision making is essential. Videodisc technology makes research on use of contextual cues in clinical decision making more attractive. The validation of videodisc simulation performance as a measurement of clinical decision making with several criterion measures needs further exploration. Because research has shown clinical expertise to be domain specific, a priority for 103 further study is the investigation of clinical decision making over several simulations. A taxonomy to specify the multiple types of decision making events to be simulated would be beneficial in focusing research efforts. Despite over twenty five years of research on clinical decision making in nursing, there remains a great need for further investigations that will help develop and validate a theory of expertise in clinical decision making. BIBLIOGRAPHY AACN Certification Corporation (1984). Critiga1_gare nursin9_nractice_ta§k_statemeot§1 Newport Beach. CA: Author. AACN Certification Corporation (1988). Einal_r§pgth e9oteot_aod_99ostru9t_xalidatien_of_the_§QBN CeItif12aIi9n_Exaninatieo_1282:12§41,Newoort Beach. CA: Author. Abbott, S. D., Carswell, R., McGuire, M., 8 Best, M. (1988). Self-evaluation and its relationship to clinical evaluation. 19urnal_9f_uur§189_fidueatien .21(5). 219- 224. Anderson, J.R. (1982). Acquisition of cognitive skill. 2529891991221_Bexiex. 82. 369-406- Anderson. J-R- (1985)- C99nitixe_osxehclesx_and_its implications . New York: Freeman. Aaronson, L. (1989). A cautionary note on the use of stepwise regression. Ngr§13g_3g§gg;§n, 18(5), 309-311. Baumann, A. 8 Bourbonnais, F. (1982). Nursing decision making in critical care areas. Jggznal 9f Agyanged Nursingl 1: 435'445- Baumann, A. 8 Deber, R. (1989). The limits of decision analysis for rapid decision making in ICU nursing. Benner, P. (1984). o v'c t ° w ' ° . Palo Alto, CA: Addison-Wesley. Benner, P. 8 Wrubel, J. (1982). Skilled clinical knowledge: The value of perceptual awareness. Ngr§g_§ggggtgr, 1(3), 11-17. 104 105 Benner, P. 8 Tanner, C. (1987). Clinical judgment: How expert nurses use intuition. Amerigan_1gnrnal_gf Nursing 81. 23- 31- Bligh, T.J. (1980) Written simulation scoring: a comparison of nine systems. Presented at the annual meeting of the American Educational Research Association, Boston. Bourbonnais, F. 8 Baumann, A. (1985). Crisis decision making in coronary care: A replication study. Nursing_ ERREIE. 15(4). 4'13- Cohen. J- (1988)- Statiati2al_oewer_analx§is_f9r_the behayigral_§§i§ngg§. Hillsdale, NJ: Lawrence Erlbaum Associates. Corcoran, S. (1986a). Planning by expert and novice nurses in cases of varying complexity. Researsh_in_Nur§ing_8 HealIh. 2. 155-162- Corcoran, S. (1986b). Task complexity and nursing expertise as factors in decision making. Nursing_R§§eazgh, 35, 107- 112. Corcoran, S., Narayan, S., 8 Moreland, H. (1988). "Thinking aloud" is a strategy to improve clinical decision making- Heart_8_Lnog. 11(5). 463-468- Cowan, M. J. (1990). Cardiovascular Nursing Research. In J. J. Fitzpatrick 8 R. L. Taunton (Eds. ). v ew Of_Nur§ing_Reeear2h Vol 8 (pp 3- 33). New York: Springer Publishing Co. Cronbach, L.J. (1988). Five perspectives on validity argument. In H. Wainer 8 H.I. Braun (Eds.), Test yalidity (pp. 3-17). Hillsdale, NJ: Laurence Erlbaum. Davis, B. G. (1972). Clinical expertise as a function of educational preparation. Ng;§_ng_3g§ga_gh, 21, 530- 534. Davis, B.G. (1974). Effects of levels of nursing education on patient care: A replication. Ngrsing_3g§gargh, 2;, 150-155. Dincher, J.R. 8 Stidger, S.L. (1976). Evaluation of a written simulation format for clinical nursing judgment: A Pilot study. Nureiog_Be§earsh. 25. 280- 285. Dreyfus, H.L. 8 Dreyfus, S.E. (1986). Ming_gygr_maghing. New York: The Free Press. 106 Elstein, A. S. (1988). Cognitive Processes in Clinical Inference and Decision Making. In D. C. Turk 8 P. Salovey (Eds ) Beaa9nin91_Infereneel_8_ludgmeot_io Clinical_£exehelesx (pp- 17- ~50). New York: Free Press/Macmillan. Elstein, A.S. 8 Bordage, G. (1979). Psychology of clinical reasoning. In G. Stone, F. Cohen, 8 N Alder (Eds.), Health_p§yghglggy (pp. 333-368). San Francisco: Jossey-Bass. Elstein, A.S., Holmes, M.M., Ravitch, M.M., Rovner, D.R., Holzman, G. B., 8 Rothert, M. L. (1983). Medical decisions in perspective: Applied research in cognitive psychology- 2eraoe9tixee_in_8191992_8_nedicine. 29. 486- 500. Elstein, A.S., Rovner, D.R., Holzman, G.B., Ravitch, M.M., Rothert, M.L., 8 Holmes, M.M. (1982). Psychological approaches to decision making in health care. In R.B. Deber 8 G. G. Thompson (Eds. ). oi es He a e° De2is19o_maki_9_8_E_aluatieo_2f_Effe_tixene§e. TorontO. Canada: University of Toronto Department of Health Administration. Elstein, A.S., Shulman, L.S., Sprafka, S.A. (1978). Mggigal v ° ' s ' . Cambridge, MA: Harvard University Press. Ericsson, K.A., Simon, H.A. (1980). Verbal reports as data. 2529891991281_Bexiex. 81. 215-251- Farrand, L., Holzemer, W.L., 8 Schleutermann, J. (1982). A study of construct validity: Simulations as a measure of nurse practitioners' problem solving skills. HBIfiiDQ_B§§§§IQDI ll: 37‘ 42- Glaser, R. 8 Chi, M.T.H. (1988). Overview. In M.T.H. Chi, R- Glaser. & N-J- Farr (Eds-). The_Nature_9f_Exoertie_. Hillsdale, NJ: Lawrence Erlbaum Associates, Publishers. Grier, M.R. (1984). Information processing in nursing practice. In H. H. Werley 8 J.J. Fitzpatrick (Eds. ), (Vol.2) (pp. 265- 287). New York: Springer Publishing. Hammond, K., Kelly, K., Schneider, R., 8 Vancini, J. (1966a). Clinical inference in nursing: Analyzing tasks representative of nursing problems. Nursing BQEQQIED. lip 134-138- 107 Hammond, K., Kelly, K., Schneider, R., 8 Vancini, J.(1966b). Clinical inference in nursing: Information units used. Nursing_Be§eersh. 15. 236-247. Hammond, K., Kelly, K., Castellan, H., Schneider, R., 8 Vancini, J. (1966c). Clinical inference in nursing: Use of information-seeking strategies by nurses. Nursing_3e§ear2h. 1Q. 38-45- Henderson, V. (1982). The nursing process-is the title right? l2urnal.9f_Adxansed_Hur§ing. 1. 103-116. Henry, S.E., LeBreck, D.B., 8 Holzemer, W.L. (1989). The effect of verbalization of cognitive processes on clinical decision making. Bgfiggzgn_in_ngzging_§ 3111331. .12.: 187-193 . Henry. 8-3- (1989). Ine_effe21_Qf_lexel.21_natient_assitx_ d a i ° c s ui1n_xar2ing_lexels_9f_knogledge_and_ex2erienge- Unpublished doctoral dissertation. University 0 California, San Francisco, CA. Holden, G.W. 8 Klingner, A.M. (1988). Learning from experience: Differences in how novice vs. expert nurses diagnose why an infant is crying. Jggxga1_9§ Nursing_Edngatign. 21. 23-29- Holzemer, W., Resnik, 8., Slichter, M. (1986). Criterion- related validity of a clinical simulation. Journal_gfi Enr§1n9_Edusatign. 2;. 286-290- Holzemer, W. 8 McLaughlin, F. (1988). Validity of clinical simulations. flestern_19urnal_2f_nnzsing_3e§earsn. 19. 73-83. Holzemer, W.L., Schleutermann, J.A., Farrand, L., 8 Miller, A.G. (1981). A validation study: Simulations as a measure of nurse practitioner problem—solving skills. HursinQ_Be§eazsh. 19(3). 139-144- Itano, J.R. (1989). A comparison of the clinical judgment process in experienced registered nurses and student nurses. l9nrnal_2f_uur§1ng_zgugetign. 25. 120-125- Johnson, J.H. (1988). Differences in the performances of baccalaureate, associate degree, and diploma nurses: A meta-analysis- Beseargn_in_nur§ing_8_nealth. 11. 183- 197. 108 Kassirer, J.P. 8 Gorry, G.A. (1978). Clinical problem solving: A behavioral analysis. Anna1§_gf_1ntg;nal Hedisine. 82. 245-255. Kinney, M.R. (1990). Education for Critical Care Nursing. In J.J. Fitzpatrick 8 K.L. Taunton (Eds.). Annual Bex1ew_gf_Nur§1ng_Be§earsh. Vol-8 (pp 161-176). New York: Springer Publishing Co. Knapp, T.R. (1991). Coefficient alpha: Conceptualizations and anomalies. Beseargh_1n_Nursing_8_Health. 11. 457- 480. Kuhn. T-S- (1970)- Ihe_s1rn2ture_9f_§21entifis_rexglug1gn§- Chicago: University of Chicago Press. Landis, J.R. 8 Koch, G.G. (1977). The measurement of observer agreement for categorical data. fiigmgtrigg, 11, 159-174. Mazzuca, S. A. and Cohen, S.J. (1982) Scoring patient management problems, external validation of expert consensus. Exalua112n_and_tne_flealgn__rgfe§§1_n§.é. 210- 217. McGuire, C. H., Solomon, L. M., and Bashook, P. G. (1976). Q2nstru2112n_and_nse.gf_urits_n_§1m_latign_- USA: The Psychological Corporation. McCloskey, J.C. 8 McCain, B. (1988). Variables related to nurse performance. Image, 12(4), 203-207. McLauglin, F. E., Carr, J. W., 8 Delucchi, K. L. (1980). Selected psychometric properties of two clinical simulation tests. J9urnal_gf_Ned1gal_Eduget1gn.1.. 375-376. McLauglin, F.E., Carr, J.H., 8 Delucchi, K.L. (1981). Measurement properties of clinical simulation tests: Hypertension and chronic obstructive pulmonary disease. Nursing_Besesrgh.71Q. 5-9. Norman, G.R., Tugwell, P., Feightner, J.H., Muzzin L.J., 8 Jacoby, L.L. (1985). Knowledge and clinical problem- solving ability- Neg1211_112211123. 12. 344-356- Norcini, J.J., Swanson, D.B., Grosso, L.J., Webster, G.D. (1983). A comparison of several methods of scoring patient management problems. c ’ s t e n I Washington, DC. 109 Nisbett, R.E. 8 Wilson, T.D. (1977). Telling more than we can know: Verbal reports on mental processes. Esxsholog1sal_3sxiey. 11. 231-257- Neufeld, V.R., Norman, G.R., Feightner, J.W., 8 Barrows, H.S. (1981). Clinical Problem-solving by medical students: A longitudinal and cross-sectional analysis. Hed1sal_Edusation. 11. 315-322- Newell, A., Shaw, J.C., 8 Simon, H. (1958). Elements of a theory of human problem solving. Rezonologioal_aexieu. 65, 151-166. Newell. A-. & simon. H- (1972)- Human_nroblem_solxingo Englewood Cliffs, NJ: Prentice-Hall. Nunnally, J. (1978). 25y;hgmgtrig_1ngg:y. NY: McGraw-Hill. Page, G. G. 8 Fielding, D. W. (1980). Performance on PMPs and performance in practice: are they related? 19g;nal_gf Medigal_zéugas12n. £5. 529- 537. Patel, V.L. 8 Groen, G.J. (1986). Knowledge based solution strategies in medical reasoning. Cognitiyg_§gigg§g, 12, 91-116. Phillips, M. K. (1979). Faculty plans for evaluation of students in the clinical setting. In Ih§_ghallgng§_gf g11n1991_gyalnatign (Publication No. 16- -1764, 35-42), New York: National League for Nursing. Polanyi, M. (1958). £§1§93§1_kngylgggg. London: Routledge 8 Kegan Paul. Predk0. J-E- (1974). uul11:meQ11_s1mulation_1esnn1gue1__a_ me1h2d_1o_s1udx_1he_nur§1ng_diagn2s11s_nroses§. Unpublished master’s thesis, University of Rochester, Rochester, NY. Pyles, S. H. 8 Stern, P. N. (1983L Discovery of nursing gestalt in critical care nursing: The importance of the gray gorilla syndrome. a ' s Ssholaz§h1n. 11(2). 51- 57- Radwin, L.E. (1990). Research on diagnostic reasoning in nursing- Nur§1n9_niagno§1§. 1(2). 70-77- Rausch, T. 8 Rund, D. (1981). Nurses’ Clinical Judgments. , 12(12), 24-26. 110 Sneed, J., Vivian, V., 8 D'Costa, A. (1987). Work experience as a predictor of performance: a validation study- WW. 12(1). 42- 57. Stanley, J.C. (1972) Reliability. In R.L. Thorndike (ed.) . Washington, D.C.: American Council of Education. Stross, J.K. 8 Bellfly. L.C. (1979). Continuing Education for coronary care nurses. Heart_§_Lgng, g, 318-321. Sanford, S.J. 8 Disch, J.M. (1989). AAQNL§_§;§nggrg§_fgr_ WW- Norwalk. CT: Appleton 8 Lange. Smith, S. (1988). An analysis of the phenomenon of deterioration in the critically ill. Imggg, 2g (1), 12- 15. Tanner, C.A. (1983). Research on clinical judgment. In W.L. Holzemer (Edd . MW (pp 2-32). Thorofare, NJ: Charles B. Slack. Tanner, C.A. (1987). Teaching clinical judgment. In J.J. Fitzpatrick 8 R.L. Taunton (Eds.), Annga1_3gyigg_gf ' (pp 153-173). New York: Springer Publishing Co. Tanner, C.A., Padrick, K.P., Westfall, U.E., 8 Putzier, D.J. (1987). Diagnostic reasoning strategies of nurses and nursing students. Nursing Research, 36, 358-363. Thompson, D.R., 8 Sutton, T.W. (1985). Nursing decision making in a coronary care unit. Internatigngl_lggrngl 9f ugrging Studies, 3;, 259-266. Topf, M. (1986). Three estimates of interrater reliability for nominal data. Hugging_3g§§g;gh, 35(4), 253-255. Toth, J.C. (1984). Evaluation the use of the Basic Knowledge Assessment Tool (BKAT) in critical care nursing with baccalaureate nursing students. Image;_1ng_ggg;gal_ Warship. 11. 67-71. Toth, J.C. (1986). The Basic Knowledge Assessment Tool (BKAT) - validity and reliability: A national study of critical care nursing knowledge. Wes;g;n_gggrggl_gf MW. 1. 181-196- 111 Toth, J.C. 8 Ritchey, K.A. (1984). News from nursing research: The Basic Knowledge Assessment Tool (BKAT) for critical care nursing. figa;;_§_ngg,I11, 272-279. Vancott, M., Tittle, M., Moody, L., 8 Wilson, M. (1991). Analysis of a decade of critical care nursing practice research: 1979 to 1988. near§_§_Lgng, 29(4), 394-397. Verhonick, P.J., Nichols, G.A., Glor, 8., 8 McCarthey, R.T. (1986). I came, I saw, I responded: Nursing observation and action survey. Nursing_3§§g§zgn, 11. 38-44 0 Westfall, U.E., Tanner, C.A., Putzier, D.J., 8 Padrick, K.P. (1986). Activating clinical inferences: A component of diagnostic reasoning in nursing. Beggargh_in_flg;§1ng_8 321113. 2. 259'277- APPENDICES .APPTEHIEX.A PARTICIPANT PROFILES CODE NUMBER: PARTICIPANT PROPILE FOR THE RN* *reduced copy 1. TODAY'S DATE: 2. DATE OF BIRTH: month day year 3. SEX: Female Male 4. FIRST LEVEL OF NURSING EDUCATION: Diploma ADN BSN 5. HIGHEST LEVEL OF NURSING EDUCATION: Diploma ADN BSN MSN Year of graduation 6. DATE OF EMPLOYMENT IN PRESENT PATIENT CARE UNIT: 7. NAME or UNIT(e.g. coo, ICU, or ER): 8. CURRENT CLINICAL NURSE LEVEL: Level I Level II Level III If levels not applicable, SPECIFY POSITION TITLE: 9. PERCENTAGE OF TIME IN DIRECT CARE: % 10. CURRENT SHIFT: DAYS EVENINGS or NIGHTS 8-HOUR 12 HOUR or OTHER specify 11. CURRENTLY WORKING FULL TIME OR PART TIME if PT, # of Hours per week 12. CLINICAL NURSING EXPERIENCE (IN YEARS & MONTHS): Most of this experience was: Full Time or Part Time if PT, # of hours per week Most of experience was on: One Type of Unit or A Variety of Units 13. CRITICAL CARE EXPERIENCE (IN YEARS & MONTHS): Most of this experience was: Full Time or Part Time if PT, # of Hours per week Specify name of unit/s: Patients on these units were: Adults Children or Both If adults, most were 65 or older: Yes No 112 14. 15. 16. 17. 18. Topic/s 19. 20. 21. 22. 23. 24. 25. EXPERIENCE CARING FOR CARDIAC PATIENTS (IN YEARS & MONTHS): Most of this experience was: Full Time or Part Time if PT, # of Hours per week Patients on these units were: Adults Children or Both If adults, most were 65 or older: Yes No Most of this experience was: Acute Care Step-down/intermed. Rehabilitation Other ACLS CERTIFICATION: Current Not Current Never Certified CCRN CERTIFICATION: Current Not Current Never Certified COMPLETION OF CRITICAL CARE OR CCRN REVIEW COURSE: At present hospital: yes no Another site: yes no If yes, when ““ “' If yes,when '—————— ATTENDED COfiTINUING EDUCATION PROGRAM/S IN LAST 12 MONTHS: BB 0 If yes, program/s sponsored by: Employer Other or Both CURRENTLY ENROLLED IN A COLLEGE DEGREE PROGRAM: Yes NO If yes, A Nursing Program or Non-nursing Program A Baccalaureate Masters or Other HIGHEST NON-NURSING DEGREE: None AD BS/BA Masters Other Year of graduation SUBSCRIBE TO NURSING JOURNAL/S: Yes NO If yes, specify which ones: gURRENT MEgBER OF A NURSING PROFESSIONAL ORGANIZATION: 68 O If yes, specify organization/s: Have you attended organizational meetings in the last year: Yes No PREVIOUS HANDS-ON EXPERIENCE WITH COMPUTERS: Yes NO If yes, current use is about: Daily Weekly Monthly ¥AVE YOU EXER BEEN EMPLOYED AS A NURSING ASSISTANT OR LPN: BB 0 If yes, Length of time: Was it in a critical care unit: Yes No Were most of the patients 65 or older: Yes No agggNG THE LAST MONTH, HOW MANY NURSING JOURNAL ARTICLES HAVE YOU None One Two Three Four Five or more * * * END OF PROFILE * * * 113 CODE NUMBER: PARTICIPANT PROFILE FOR STUDENTS* *r l. 2. 3. 4. If 6. 7. 8. 9. 10 11 12 13 educed copy TODAY'S DATE: DATE OF BIRTH: month day year SEX: Female Male FIRST LEVEL OF NURSING EDUCATION: Senior BSN Student Diploma ADN BSN HIGHEST LEVEL OF NURSING EDUCATION: Senior BSN Student Diploma ADN BSN MSN Year of graduation Senior BSN Student, skip to number 20. DATE OF EMPLOYMENT IN PRESENT PATIENT CARE UNIT: NAME OF UNIT: CURRENT CLINICAL NURSE LEVEL: If levels not applicable, SPECIFY POSITION TITLE: PERCENTAGE OF TIME IN DIRECT CARE: 8 . CURRENT SHIFT: DAYS EVENINGS or NIGHTS B-HOUR 10-HOUR 12 HOUR or OTHER specify . CURRENTLY WORKING FULL TIME OR PART TIME if part time . CLINICAL NURSING EXPERIENCE (IN YEARS & MONTHS): Most of this experience was: Full Time or Part Time if part time Most of experience was on: One Type of Unit or A Variety of Units . CRITICAL CARE EXPERIENCE (IN YEARS 8 MONTHS): Most of this experience was: Full Time or Part Time if part % Specify name of unit/s: Patients on these units were: Adults Children or Both If adults, most were 65 or older: Yes No 114 14. 15. 16. 17. 18. 19. 20. 21. 22. 23. 24. 2S. CARDIAC CARE EXPERIENCE (IN YEARS & MONTHS): Most of this experience was: Full Time or Part Time if part % Patients on these units were: Adults Children_ or Both If adults, most were 65 or older: Yel“ Most of this experience was: aggte Care Rehabilitation or er “" -———' ACLS CERTIFICATION: Current Not Current Never Certified CCRN CERTIFICATION: Current Not Current Never Certified COMPLETION OF CRITICAL CARE OR CCRN REVIEW COURSE: At present hospital: yes no Another site: yes no If yes, when If yes, when ATTENDED CogTINUING EDUCATION PROGRAM/S IN LAST 12 MONTHS: O. 0 If yes, program/s sponsored by: Employer Other or Both Specify topic/s CURRENTLY ENROLLED IN A COLLEGE DEGREE PROGRAM: Yes No If yes, A Nursing Program or Non-nursing Program A Baccalaureate Masters or Other HIGHEST NON-NURSING DEGREE: None__ AD__ BS/BA Masters Other Year of graduation ——sperrry—— SUBSCRIBE TO NURSING JOURNAL/S: Yes No CURRENT MEMBER OF A NURSING PROFESSIONAL ORGANIZA ION: Yes No If yes, specify organizations s: PREVIOUS HANDS-ON EXPERIENCE WITH COMPUTERS: Yes No If yes, current use is about: Daily Weekly Monthly ¥AVE YOU EVER BEEN EMPLOYED AS A NURSING ASSISTANT OR LPN: OS If yes, Length of time: Was it in a critical care unit: Yes No Were most of the patients 65 or older: Yes No IF CURRENTLY A NURSING STUDENT, gaze you begn assigned to patient/s with dysrthymias: o If yes, were any of these patients 65 or older: Yes No * w * END or PROFILE * * * 115 APPENDIX B BASIC KNOWLEDGE ASSESSMENT TOOL (BKAT) IN CRITICAL CARE NURSING Version Four CARDIOVASCULAR SUBSCALE ONLY' ”reduced copy 217.19.119.93 Select the m answer for each question. Mark the answer on the computerized scan sheet using a #2 pencil. 1. Initial measures for the treatment of angina pectoris include all of the following E39211 a) rest b) morphine cl oxygen d) nitroglycerine 2. The classical ECG change in myocardial infarction (MI) is a: a) normal 0 wave b) ST segment elevation c) prolonged O—T duration dl prolonged P-R interval 3. Elevated cardiac iso-enzymes generally occur in all of the followingM: al congestive heart failure bI pericarditis c) closed chest injury d) cardiac surgery 4. The major therapeutic goal in the treatment of cardiogenic shock is to: a) increase afterload b) lower the BUN c) increase cardiac output d) decrease extracellular fluid volume 5. Mr. Hart is two days post MI. During his first time getting out of bed his pulse increases from 86/min to 96/min. Based on this response, the nurse should: a) ask him to slow his pace b) allow him to continue c) have him lie down immediately dl check his vital signs 6. In dealing with a depressed patient during the first days post Ml, the most appropriate nursing action would be: a) encourage the patient to ventilate his concerns bl restrict visits from the family members c) provide privacy by leaving the patient alone d) provide a quiet environment for the patient 116 10. 11. 12. 13. The wave in the cardiac cycle that represents atrial depolarization is the: a) P b) 0 c) R (1) T A 088 complex wider than 0.12 seconds most likely indicates: al normal ventricular conduction bl bundle branch block cl second degree heart block dl myocardial infarction How many seconds is the normal P—R interval? a) 0.04 - 0.10 bl 0.12 - 0.20 C) 0.22 - 0.26 d) 0.28 - 0.32 The following rhythm strip represents: [place rhythm strip here] al idioventricular rhythm b) junctional rhythm c) complete heart block dl second degree heart block, Type II The ventricular rate in question 10 is approximately how many beats per minute? al 56 bl 7O cl 90 dl 38 The dysrhythmia in the following strip is: [place rhythm strip here] a) sinus tachycardia bl atrial flutter cl atrial fibrillation dl ventricular flutter A strong ventricular stimulus is potentially dangerous in which period of the cardiac cycle? a) U wave b) P wave cl T wave d) ORS complex 117 14. 15. 16. 17. 18. 19. 20. The rhythm strip below shows: [place rhythm strip here] a) ventricular fibrillation b) atrial fibrillation cl ventricular tachycardia dl atrial tachycardia The maig purpose of enclosing a pacemaker generator in a rubber glove or similar apparatus is to prevent: al the pacemaker from getting dirty bl moisture from corroding the pacemaker cl accidental change in settings dl electrical interference with the pacemaker In the following rhythm strip, the pacemaker is exhibiting: [place rhythm strip here] a) failure to sense b) failure to capture c) normal function dl demand function The initial drug treatment for ventricular tachycardia is: a) lsuprel 1.0 mg in 250 ml 05W drip bl Epinephrine 1:10.000 1.0 mg IV bolus cl Atropine 0.6 mg IV bolus d) Lidocaine 50-100 mg IV bolus The rhythm strip below shows: [place rhythm strip here] a) ventricular tachycardia bl atrial tachycardia cl atrial fibrillation dl ventricular fibrillation The cardiac rhythm of atrial flutter is: al a benign condition in most people bl normal following myocardial infarction cl hazardous, as the ventricular rate may suddenly increase dl hazardous, as it may progress to complete heart block Upon recognizing ventricular fibrillation, the nurse should first: al perform a precordial thump bl establish unresponsiveness / cl give Lidocaine IV push d) check the ECG leads 118 21. 22. 23. 24. 25. 26. 27. 28. One of the first drugs to be administered in the treatment of complete heart block would be: a) Atropine bl Lidocaine cl Ouinidine dl Digoxin Your patient has atrial flutter with a ventricular response of 150 beats per minute. Therapy for this rhythm includes: a] Digoxin, Verapamil, cardioversion bl Lidocaine. sodium bicarb, cardioversion cl Lidocaine, potassium chloride, pacemaker dl Isordil, Nitropaste, Pronestyl The correct enery setting for defibrillation is how many watt/seconds: al 25—30 bl 50 cl 100 dl 200-300 Signs of cardiac tamponade may include all of the following m: al distended neck veins bl pulsus paradoxus cl decreased systolic pressure d) bradycardia A patient becomes apneic and pulseless. The monitor shows asystole. The drug that would most likely be used initially is: a) Calcium Gluconate bl Atropine cl Epinephrine dl Lidocaine Special care should be exercised when administering IV Dopamine because: a) infiltration leads to tissue necrosis bl high doses cause a bradycardia cl precipitation can occur when used in a dextrose solution d) low doses decrease renal perfusion Precautions in using IV nitroprusside include all of the following EXQEPT: al protection of the solution from light bl careful monitoring for a sudden increase in heart rate cl alertness to the development of hypertensive crisis d) use of a fresh mixture at appropriate intervals All of the following may be manifestations of digitalis toxicity w: a) rapid A-V conduction bl premature ventricular contractions cl nausea dl yellow vision 119 29. 30. 31. The most common symptom of a toxic blood level of Lidocaine is: a) elevated blood pressure bl confusion cl abnormal clotting time d) metal taste If the physician did not use Atropine for a bradycardia. which of the following could be used to increase the heart rate: al Inderal b) Ouabain cl lsuprel d) Verapamil When administering Lidocaine to a patient, the proper functioning of which of the following body systems would be most useful to know to determine the correct dosage? a) hepatic bl gastrointestinal cl respiratory dl endocrine ° ° - ' THIS ISTHEEND OFTHETEST r r r : BKAT-4 Copyright, 1990 Kathleen A. Ritchey, RN, MSN Veterans Administration Medical Center Washington, DC 20422 Jean C. Toth, RN, DNSc The Catholic University of America Washington, DC 20064 Used with permission 120 IURPEEHDIXICZ CARDIOVASCULAR SELF-EVALUATION TOOL (CST) DIRECTIONS: The following 40 items describe specific tasks related to the care of an adult cardiovascular patient. Please use the Scantron answer sheet and a #2 pencil to rate your knowledge and skills for each item using the following scale [1 is lowest on the scale and 4 is highest]: 1 - No knowledge or skills related to the task 2 - Theoretical knowledge, but limited practical skills related to the task 3 - Knowledge 8 skills related to the task in uncomplicated patient situations 4 - Knowledge 8 skills related to the task in complex patient situations 1. Collect subjective 8 objective data to determine the gravity of the patient’s condition in order to maintain a current database. 2. Interview patient or significant other 8 review hospital records in order to obtain present 8 past medical history 8 signs/symptoms of cardiovascular problems. 3. Inspect cardiovascular system including the skin, neck veins, extremities and precordium. 4. Palpate precordial area, arteries 8 veins. 5. Auscultate for heart sounds, systemic blood pressure, 8 venous blood flow. 6. Assure completion of appropriate cardiovascular laboratory studies, radiological exams, 8 diagnostic tests. 7. Obtain cardiac rhythm strip 8 measure intervals. 8. Obtain hemodynamic parameters. 9. Document in patient record 8 convey to other health team members pertinent cardiovascular physical assessment findings within a time frame consistent with the gravity of the patient's condition. 10. Gather psychological, social, 8 spiritual data from patient 8 11. 12. significant aothers in relation to present cardiovascular condition. Perform a 12-lead EKG. Assemble 8 interpret data obtained from patient records 8 cardiovascular assessment to identify patient problems/needs. 121 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. 23. 24. 25. 26. 27. 28. Use pertinent cardiovascular physical assessment findings to identify patient problems/needs. Use pertinent serum 8 urine lab studies, radiological examinations, 8 diagnostic tests to identify patient's cardiovascular problems/needs. Use pertinent physiological, psychological, 8 social concepts of aging to assist in the identification of an elderly patient's cardiovascular problems/needs. Evaluate the EKG to identify patient problems/needs. Evaluate the hemodynamic parameters to identify problems/needs. Collaborate with patient, significant others, 8 other health care team members to identify cardiovascular problems/needs. Establish the priority of the cardiovascular problems/needs according to the actual/potential threat to patient, and reassess as the database changes. Record in patient record 8 communicate the identified cardiovascular problems in a timely manner consistent with the gravity of the patient's condition. Devise a plan of care, identify appropriate goals 8 determine nursing interventions for a patient with dysrhythmia. Specify cardiovascular interventions that communicate acceptance of the patient's 8/or significant other's beliefs, culture, religion 8 socioeconomic background. Develop 8 organize the cardiovascular plan of care to reflect the priority of identified problems/needs in collaboration with the patient, significant others, 8 other health care team members. Identify areas for education of the patient 8 significant others based on specific cardiovascular problems/needs. Revise the plan of care to reflect the patient's current status 8 specific cardiovascular problems/needs. Communicate the cardiovascular plan to those involved in the patient's care. Record the cardiovascular plan of nursing care in the patient's hospital record. Implement care for patient with dysrhythmias in an organized 8 humanistic manner. 122 29. 30. 31. 32. 33. 34. 35. 36. 37. 38. 39. 40. Provide care for patient with dysrhythmia in such a way as to prevent complications 8 life-threatening situations. Implement the plan of nursing care in collaboration with the patient, significant others, 8 other health care team members. Utilize physiological, social, 8 psychological concepts of aging in providing nursing care for an elderly patient with a dysrthymia. Coordinate care delivered by health care team members. Document interventions in the permanent record. Evaluate results of nursing care continuously. Collect data from all pertinent sources for evaluation within an appropriate time interval after intervention. Compare the patient's response to expected results 8 attempt to determine the cause of any significant differences. Determine the relevance of the nursing interventions to the identified problems/needs. Collaborate with the patient, significant others, 8 other health care team members in the evaluation process. Review 8 revise the plan of care based on evaluation results. Document evaluation findings in the patient's hospital record. * * * THIS IS THE END OF THE SELF-EVALUATION * * * Adapted from Cardiovascular Self-Assessment Tool Used with Permission Suzanne Bakken Henry University of California San Francisco 123 .APPTUHIEX D INTERACTIVE VIDEO CASE STUDY DESCRIPTION The case study begins as Mr. Talbert, a 73 year old man, arrives in the Emergency Room complaining of chest pain. The user must decide on whether to assess or intervene and then on what data to collect and/or what intervention(s) are required at this point in time. Following nursing intervention(s), a 12 lead ECG is taken, and the user may choose to interpret this information. Interpretation of the ECG was an optional activity that was not used in the final scoring of the simulation. Next, the nurse administers a protocol dose of nitroglycerin and a few minutes later Mr. Talbert develops a headache and dizziness. The user must again make decisions related to essential assessment and intervention. Mr. Talbert's lab results become available and the user must indicate those that are abnormal. This segment shows the nurse and physician discussing the possibility of thrombolytic therapy. The user must decide what assessment data to collect and what intervention(s) are required in order to see if Mr. Talbert fits the criteria for this treatment. Following the initiation of thrombolytic therapy, a medical student arrives to draw the patient's blood for arterial blood gases. The user must decide what his/her responsibility is in this situation. A few minutes later, Mr. Talbert complains of a ”skipping feeling" in his chest. Again the user must decide what essential assessment data to collect, and what intervention(s) are required. Several minutes pass and Mr. Talbert says he does not feel well and loses consciousness. The learner must identify the rhythm on the ECG monitor and select appropriate intervention(s). When Mr. Talbert requires defibrillation, the user must set the defibrillator appropriately. This defibrillator section was not used in the final scoring because users had difficulty getting the computer to respond appropriately. The case study concludes with a summary of the patient's treatment following his discharge from the Critical Care Unit, including scenes from his triple by-pass surgery. Important considerations for education and discharge planning are provided and the simulation is over. 124 APPENDIX E EXPERT PANEL MEMBERS (N=4) Denise Grimes, RN, MSN (1988 graduate) Clinical Nurse Specialist in SICU/CICU Ingham Medical Center Lansing, Michigan Sixteen years experience in critical care (CICU) ACLS Certified Member of Sigma Theta Tau 8 American Association of Critical Care Nurses Kathy Ribbons, RN, BSN (1989 graduate) Nurse Manager, Emergency Department Ingham Medical Center Lansing, Michigan Eleven years experience in critical care (Surgical ICU, Cardiac ICU, and ER) ACLS and CEN Certified Member of Sigma Theta Tau 8 Emergency Nurses Association Alan O'Brien, RN, BSN (1980 graduate) Assistant Department Manager, CCU Sparrow Hospital Lansing, Michigan Eleven years experience in critical care (CCU) ACLS and CCRN certified Member Sigma Theta Tau 8 American Nurses Association Marilyn Meinhardi, RN (1966 Diploma graduate) Assistant Department Manger, CCU Sparrow Hospital Lansing, Michigan Twenty-five years critical care experience (CCU-ICU) Member American Association of Critical Care Nurses 125 APPENDIX F DIRECTIONS FOR THE INTERACTIVE VIDEO SIMULATION CODE NUMBER TODAY'S DATE DIRECTIONS FOR THE INTERACTIVE VIDEO SIMULATION. *reduced copy 1. i th h reen Hith this system, choices are entered by touching an area on the screen. It is recommended that you sit directly in front of the screen and touch the center of your choice for best results. The screen emits one of two sounds when touched. A high-pitched sound indicates that the computer has recorded your touch. A low pitched sound indicates that an inappropriate area of the screen has been touched, or that the computer is processing data and is no yet ready to accept your input. If this occurs, just touch again, and try to do so in the center of the touch area. You will also noti::dthat when the computer records your touch, a color change occurs in the area you have touc . 2. .As sment n rv i S r After each video segment describing a situation requiring nursing care, you will be asked to make choices from Assessment and/or Intervention screens. The choices on these screens are always the same, but you must select only those which are appropriate at that point in the patient's care, and you must make your selections in the order of their priority. After you make an assessment or intervention choice, the outline around the touch box will disappear, as a reminder that you have already made that selection. 3. Other Fgatures Availgle CONTINUE BACK whenever the CONTINUE arrow appears in the right corner, you must touch this arrow to continue to the next screen or video segment in the program The BACK arrow only appears when there are two or more screens of information in succession. It allows you to return to the previous screen if you wish. 6. go QQT use the Take ngtrol Feature: This means not using the library, replay, resume, bookmark, main menu, exit or start over. when you are given a choice of (a) Starting over or (b) Finding out why your choice was incorrect and continuing, always choose (b) and continue with the program. 5. End of gage Stggy 1 The case study will end with a screen full of text and the words “End of Case Study 1" at bottom of screen. This is the end of the research simulation. LEAVE THE COMPUTER ON. Please complete the reverse side of this sheet and let Joan know that you are finished. 1J26 .APPTHHIIX G; STUDENT RECRUITMENT FLYER * * Opportunity to Participate in Research Project * * CLINICAL DECISION MAKING USING INTERACTIVE VIDEO SIMULATION* (*reduced copy) WHO: Senior baccalaureate nursing students WHAT: Approximately 2 hours of your time Written questionnaires - 1 hour Computerized simulation - 1 hour [no computer skills needed] WHEN: First week of Spring Term [during registration 8 first 3 days of class] Tuesday, March 26 through Friday, March 29 WHERE: Life Sciences Building on campus Different rooms depending on date WHY: I To increase our understanding of how to evaluate clinical judgment skills I Earn ten dollars as a token of appreciation for your time I Chance to win N-CLEX review book (7 available) I Chance to practice clinical decision making skills using the latest in simulation technology HOW TO SIGN UP: Select two time periods from those listed below. Plan to meet me in the room indicated for those two times. Complete and return the form below or call Joan Predko to schedule your times: 355-6525 (office) or 655-1916 (home). Evening or Saturday sessions can be arranged. TUESDAY, MARCH 26 IEDIIESDAY, MARCH 27 Room A205 Life Sciences Room 127C Life Sciences, S-wing Afternoon Morning After 1:00pm - 2:30p 9:00n - 10:30am 1:00pm - 2:30pm 3:00pm - 4:30pm 11:00am - 12:30pm 3:00pm - 4:30pm 5:00pm - 6:30pm 5:00pm - 6:30pm THURSDAY, MARCH 28 FRIDAY, MARCH 29 Room A216 Life Sciences Room 1080 Life Sciences, B-wing Horni Afternoon Horni Aftgggggn 11:55 - 12:30?» 1:00p! - 2:30pm 9:00am 40:30am 1:00m - 2:30;:- 3:00pm - 4:30pm 11:00am - 12:30pm 3:00pm - 4:30pm 5:00pm - 6:30pm 5 00pm - 6:30p Session 1: Session 2: date 8 time date 8 time Return this form by Friday, March 15 to: Joan Predko, A228 Life Sciences or main office A230 THANK YOU for volunteering! Be sure to indicate the two time periods you have selected on your portion of the form. See you next term!! 127 .APPTTUIIX El STUDENT CONSENT FORM [peel off code #] CONSENT FORM You are being asked to participate in a project in which we hope to learn more about how nurses and nursing students make clinical judgments. This study is being conducted as part of a doctoral dissertation at Michigan State University. You were selected as a possible participant because you are a senior baccalaureate nursing student. PLEASE BE AWARE THAT THIS WILL NOT AFFECT YOUR COURSE GRADES IN ANY WAY. If you agree to participate, you will be asked to 1) complete 3 pencil- and-paper questionnaires (a portion of the Basic Knowledge Assessment Tool for Critical Care Nursing, a Cardiovascular Self-evaluation Tool, and a demographic profile sheet) and 2) respond to a computerized interactive videodisc simulation. Anticipated total time required is 2 hours. Testing will be outside of class time. There are no know risks involved in testing. Ten dollars per subject is being offered as a token of appreciation for the time given for this activity. In addition, the data from this study may assist nursing educators in better understanding the evaluation of clinical judgment skills. All individual information that is obtained in connection with this study will be kept confidential, and after data collection is completed, will be anonymous. No participant will be identified by name in any oral or written report. Group results will be published in the form of a dissertation and in journal articles. If you desire a copy of the study results, please self-address the envelope provided. You may refuse to participate or withdraw from this study at any time without affecting your future relations with Michigan State University. Should you have any questions now or later with regard to this study, please feel free to contact Joan E. Predko, doctoral candidate at Michigan State University, (517 355-6525). You will be given a copy of this form to keep. On the basis of the above statement, I agree to participate in this project. Participant's Signature Date Witness's Signature Date 128 APPENDIX I EXPLANATORY MEMO 8 CONSENT FOR RNS MEMORANDUM‘ *reduced copy July 22, 1991 TO: Nursing Associates, Sparrow Hospital FROM: Joan Predko, RN, PhD Candidate Assistant Professor Michigan State University College of Nursing RE: Participation in a Clinical Nursing Research Project I am conducting a research project to study the CLINICAL DECISION MAKING SKILLS OF REGISTERED NURSES who provide direct care for adult patients with acute cardiac disorders. In their Standards for Nursing Care of the Critically Ill (1989), The American Association of Critical Care Nurses has explicitly identified the use of nursing process (i.e. clinical decision making) as the basis for delivery of nursing care. And yet, only a few small studies have focused on decision making skills of critical care nurses. The primary purpose of this study is to examine the ability of different types of educational tools, especially interactive video simulation, to discriminate between the clinical decision making skills of inexperienced and experienced cardiac care nurses. All registered nurses currently working in ICU, CCU, or ER and who have had at least one year of clinical experience in a critical care unit are being asked to participate; however, participation is strictly voluntary. If you choose to become a part of this study, you will be asked to complete three written forms (45 minutes) and one interactive videodisc computer simulation (45 minutes). Knowledge and/or experience with computers is not required or expected. To maintain anonymity, only code numbers will be used in this study. Since no names will be used, individuals cannot be identified even by the researcher. A system to allow participants to obtain group study results and protect their identity has been developed. As an incentive, each of the 35 study participants will have an opportunity to win 8100. (Five $100 awardees will be randomly selected; chances are 1 out of 7.) In addition, participation in this study will earn points in the clinical ladder program under Professional Development V, Item C. If you are interested in participating, please return the completed form below to the Nursing Office as soon as possible so that I can schedule an appointment with you. Participation will take place in the Nursing Office during July 8 August. Subjects will be selected on a first come, first serve basis. Returning the form below indicates your consent to participate. Withdrawal from the study at any time is possible without negative consequences. If you have any questions, please contact me at 517/355-6525 during the day or 517/655-1916 in the evening. Thank you for your cooperation in this effort. NAME NURSING UNIT PHONE NUMBER BEST TIME TO CALL fietugp to Nggsing Officg a in Tue da u at 1991 129 APPENDIX J COMPUTER PROGRAM SCORING OF SIMULATION JOURNAL O EVENT I: ADMISSION TO ER Q ASSESS or -3 INTERVENE (not approp.) SC ‘*ASSESSMENT MENU** Physical Exam _;i_ if before VS, ECG, Pt CC (not approp. now) _;1_ if after VS, ECG, Pt CC ("do something”) +3 Vital Signs E Interview Patient +3 Current complaint & related history Complete past medical history -6 if before VS, ECG, Pt CC __;1__ if after VS, ECG, Pt CC Lab Results -g if before VS, ECG, PT CC 0 if after ECG _11_ Cardiac Monitor +1 NSR Interpretation -1 Any other response (up to 3) _11_ 12 lead ECG DONE (ordered) -2 if no VS, ECG, Pt CC 0 INTERVENE NOW or -3 DO NOT INTERVENE a RESUME CASE STUDY (not approp.) **INTERVENTION MENU** 0 Notify MD Oxygen therapy (notified) +§ A L/mdn ‘3 1 or 8 or 10 L/min Positioning Pluid Management 0 (currently semifowlers) 0 (currently 20 cc/hr) '2 flat -; 75 cc '2 flat, legs elevated ’2 150 cc 9 full fowlers '3 300 cc -2 dangling Medications -2 Emergency Interventions 0 Nitroglycerine 0 Aluminum hydroxide DONE -2 Lidocaine '2 if no oxygen '2 Atropine READ 12 LEAD ECG: 9 YES or e +3 NO Correct interpretation 0 Incorrect reponses up to 3, then must choose correct * Program error, should be +1 EVENT II: PATIENT REACTION AFTER RECEIVING NITROCLYCERINE (Has been given oxygen, blood drawn, 12 lead done, nitro given) 0 ASSESS or h -§ INTERVENE (not approp.) 130 MASSESSMENT MENU** '2 Physical Exam Interview Patient 0 Current complaint A related history ‘2 Complete past medical history +3 Vital Signs -2 Lab Results (not available yet) ECG DONE '2 if no VS +1 Cardiac Monitor +1 NSR Interpretation -1 Any other response (up to 3) *2 12 lead ECG (not approp.) 0 INTERVENE NOW or -3 RESUME CASE STUDY **INTERVENTION MENU*' Notify K) Oxygen therapy '2 if lst selection 0 (at 6 L/min) *_2__ if other than first * Program error, should be +1 -; 1 or 8 or 10 L/min Positioning Fluid Management -3 (currently semifowlers) 0 (currently 20 cc/hr) +3 flat, legs elevated Q 75 cc + flat 0 150 cc -g full fowlers 0 300 cc '2 dangling Medications -g Emergency Interventions '3 if before positioning (not approp.) '3 for each medication if selected after position DONE -2 if no position change EVENT III: INTERPRETATION OF LAB RESULTS Screen one: 0 All normal -1 For each wrong answer up to 10 Screen two: +1 All normal -1 For each wrong answer up to 4 EVENT IV: TEROMBOLYTIC THERAPY CONSIDERED 0 ASSESS or -g INTERVENE (not approp.) 131 **ASSESSMENT MENU** Physical Exam +3 Thorax a Lungs +1 correct interpretation -1 any other up to 3 Cardiovascular +1 correct interpretation -1 any other up to 2 ..+_3_ 0 Neurological 0 EENT 0 Musculoskeletal +3 Extremdties +3 Abdomen +3 Skin .22.. DONE (must select all but EENT, Musc., Neuro) O Vital Signs ECG 0 Cardiac Monitor 0 NSR Interpretation -1 Any other up to 3 0 12 lead ECG 0 INTERVENE NOW or 0 RESUME CASE STUDY **INTERVENTION MENU** 0 Notify MD (notified) Positioning g (currently semifowlers) -1 flat, legs elevated _;1__ flat 9 full fowlers -1 dangling Medications -1 Nitro '3 any other up to 4 EVENT V: ABG ORDER .311.— -1 Question order Any other answer up to 1 EVENT VI: HEART SKIPPING/JUMPING INTERVENE (not approp.) 0 ASSESS OR *— _:_3._ 132 Interview Patient 0 Current complaint & related hx +3 Complete past medical hx __Q___ Lab Results '2 DONE (Must select Physical & a Past medical history Oxygen therapy 0 (at 6 L/min) -g 1 or 8 or 10 L/min Fluid Management 0 (currently 20cc/hr) -; 75cc -2 150cc -2 300cc -2 Emergency Interventions (not approp.) 0 DONE (none are necessary) "assassins! MENU“ -2 Physical Exam +3 Vital Signs ECG +3 Cardiac Monitor +1 PVC Interpretation -1 Any other up to 3 -2 12 lead ECG (not approp.) 0 INTERVENE NOW or -3 Interview Patient 0 Current complaint 5 related history '3 Complete past medical history '2 Lab Results DONE (must do VS, ECG) __'.Z.__ RESUME CASE STUDY (not approp.) **INTERVENTION MENU** +1 Notify MD (-1 if not selected) 0 Positioning (currently semifowlers) '2 flat, legs elevated -g flat __9_ full fowlers -2 dangling Medications +3 Lidocaine -2 Any other up to 4 EVENT VII: PATIENT UNRESPONSIVE INTERPRET ECG: +1 Correct -1 -3 ASSESS or 0 (not approp.) **INTERVENTION MENU** Notify H) '2 if lst choice +1 if after Emerg. -1 Positioning if before Emerg. _;;_ Positioning if after Emerg. -2 (currently semifowlers) -; flat, legs elevated +1 flat -2 full fowlers -g dangling -2 Medications End of Simulation INTERVENE 0 Oxygen therapy (at 6 L/min) -; 1 or 8 or 10 L/min 0 Fluid Management (currently 20 cc/hr) -1 75cc '2 150cc -3 300cc -2 Emergency Interventions (not approp.) (must do medications) Incorrect up to 1 -2 Oxygen therapy -2 Fluid Management Emergency Interventions -2 CPR -2 Precordial thump -g Prep for cardioversion +2 Prep for defibrillation +2 ON *_0_ Set 200 joules 412_ Charge .15?“ -2 DONE (must select emerg.) * Program error, should be +2 133 APPENDIX K EXPERT PANEL SCORING OF SIMULATION Note: Differences from AJN Scoring indicated by asterisk with AJN value in parentheses EVENT I: ADMISSION TO ER * +1 ASSESS or * 0 INTERVENE (0) (-1) (allow oxygen therapy only before assessment) --ASSESSMENT MENU-- Physical Exam Interview Patient _;j__if before VS, ECG, Pt CC +1 Current complaint 5 related history (not approp. now) _;1_ if after VS, ECG, Pt CC Complete past medical history ("do something") -1 if before VS, ECG, Pt CC -1 if after VS, ECG, Pt CC +1 Vital Signs Lab Results -1 if before VS, ECG, PT CC __9_ if after ECG +1 Cardiac Monitor +1 NSR Interpretation -1 Any other response (up to 3) +1 12 lead ECG DONE (ordered) -1 if no VS, ECG, Pt CC *+1 INTERVENE NOW or -1 DO NOT INTERVENE & RESUME CASE STUDY (0) (not approp.) --INTERVENTION MENU-- * +1 Notify MD Oxygen therapy (0) (notified) +1 4 L/min -1 1 or 8 or 10 L/min Positioning Fluid Management 0 (currently semifowlers) Q (currently 20 cc/hr) -1 flat -1 75 cc -1 flat, legs elevated -1 150 cc 0 full fowlers -1 300 cc -1 dangling Medications -1 Emergency Interventions (0) * +1 Nitroglycerine 9 Aluminum hydroxide DONE -1 Lidocaine -1 if no oxygen or nitro -1 Atropine Optional (THIS SECTION DELECTED FROM SIMULATION SCORES IN STUDY) READ 12 LEAD ECG: 0 YES or 0 NO +1 Correct interpretation Incorrect reponses up to 3, then must choose correct .-1 (0) EVENT II: PATIENT REACTION AFTER RECEIVING NITROCLYCERINE (Has been given oxygen, blood drawn, 12 lead done, nitro given) * +1 ASSESS or -1 INTERVENE (0) (not approp.) 134 (Event II cont’d) "assassunu' menu-- -1 Physical Exam Interview Patient 0 Current complaint 5 related history -1 Complete past medical history +1 Vital Signs -1 Lab Results (not available yet) ECG DONE -1 if no VS and cardiac monitor +1 Cardiac Monitor *(only VS) _11__ NSR Interpretation _:L Any other response (up to 3) -1 12 lead ECG (not approp.) * +1 INTERVENE NOW or -1 RESUME CASE STUDY Notify MD --INTERVENTION MENU-- Oxygen therapy -1 if lst selection 0 (at 4 L/min) +1 if other than first m——_ -1 1 or 8 or 10 L/min Positioning Fluid Management -1 (currently semifowlers) 0 (currently 20 cc/hr) +1 flat, legs elevated 0 75 cc +1 flat (0) * +1 150 cc -1 full fowlers (0) * +1 300 cc -1 dangling Medications -1 Emergency Interventions -1 if before positioning (not approp.) -1 for each medication if selected after position DONE (up to A) -1 if no position change or MD *(only position) EVENT III: INTERPRETATION OF LAB RESULTS 0 All normal -1 For each wrong answer up to 10 Screen two: +1 All normal -1 For each wrong answer up to A —~ EVENT IV: THROMEOLYTIC THERAPY CONSIDERED * +1 ASSESS (0) or -1 INTERVENE (not approp.) 13S (Event IV cont'd) "ASSESSMENT MENU-- Physical Exam Interview Patient +1 Thorax A Lungs 0 Current complaint 5 related hx +1 correct interpretation +1 Complete past medical hx '1 any other up to g +1 Cardiovascular +1 correct interpretation '1 any other up to Z 0 Neurological (0) * +1 EENT 0 Musculoskeletal +1 Extremities +1 Abdomen .;:1__ Skin '1 DONE (must select all but Musc. e Neuro) * & EENT *11__ Vital Signs 0 Lab Results (0) ECG '1 DONE 0 Cardiac Monitor (Must select Physical & 0 NSR Interpretation & Past medical history '1 Any other up to 3 0 12 lead ECG 0 INTERVENE NOW or O RESUME CASE STUDY ~ "INTERVENTION MENU" * +1 Notify H) Oxygen therapy (0) (notified) 0 (at 6 L/min) __'_L_ 1 or 8 or 10 L/min Positioning Fluid Management 0 (currently semifowlers) Q (currently 20cc/hr) '1 flat, legs elevated '1 75cc '1 flat '1 150cc 0 full fowlers '1 300cc '1 dangling Medications '1 Emergency Interventions '1 Nitro (not approp.) '1 any other up to 6 0 DONE (none are necessary) EVENT V: AEG ORDER +1 Question order Any other answer up to 1 _:_1_ EVENT VI: HEART SKIPPING/JUMEING * +1 ASSESS 0R '1 INTERVENE (0) (not approp.) 136 (Event VI cont'd) "ASSESSMENT MENU-- '1 Physical Exam +1 Vital Signs ECG +1 Cardiac Monitor +1 PVC Interpretation '1 Any other up to 3 '1 12 lead ECG (not approp.) * +1 INTERVENE NOW or '1 Interview Patient 9 Current complaint 5 related history '1 Complete past medical history '1 Lab Results DONE _:_1_ (must do VS, ECG) RESUME CASE STUDY (0) (not approp.) "INTERVENTION MENU-- +1 Notify ll) '1 if never selected 0 Positioning (currently semifowlers) '1 flat, legs elevated '1 flat 0 full fowlers '1 dangling Medications +1 Lidocaine '1 Any other up to A EVENT VII: PATIENT UNRESPONSIVE INTERPRET ECG: +1 Correct '1 ASSESS or * +1 INTERVENE (not approp.) (0) "INTERVENTION MENU-- Notify M) if before Emerg. if after Emerg. FF '1 Positioning if before Emerg. 0 Positioning if after Emerg. '1 (currently semifowlers) '1 flat, legs elevated +1 flat '1 full fowlers '1 dangling __;1 Medications End of Simulation 0 Oxygen therapy (at k L/min) '1 1 or 8 or 10 L/min Q Fluid Management (currently 20 cc/hr) '1 75cc '1 150cc '1 300cc '1 Emergency Interventions (not approp.) pour _:.1_ (must do medications) '1 Incorrect up to 1 Oxygen therapy _.'_.1_ '1 Fluid Management Emergency Interventions '1 CPR '_Q__ Precordial thump '1 Prep for cardioversion +1 Prep for defibrillation --------- >_.tL01' THIS SECTION __9_ Set 200 joules NOT USED _:1_ Charge """"" >_:.LSync '1 DONE (must select emerg. 8MB) * only emerg. ('1) 137 IKPPTEUDIXIIJ SIMULATION FEEDBACK D1rect1gns: Listed below are statements about your simulation experience. Please circle the number which indicates the degree to which you agree or disagree with each statement. Written comments are encouraged. 1. I found the program to be a realistic representation of the nursing care decisions required when caring for cardiac patients. Strongly DISAGREE 1 2 3 4 5 6 7 Strongly AGREE Comments: 2. Most the decisions were easy for me to make. Strongly DISAGREE l 2 3 4 S 6 7 Strongly AGREE Comments: 3. I felt confident about most of my decisions. Strongly DISAGREE l 2 3 4 S 6 7 Strongly AGREE Comments: 4. I found the program confusing. Strongly DISAGREE 1 2 3 4 5 6 7 Strongly AGREE Comments: 5. The best part of the program was: 6. The worst part of the program was: 7. I participated in this research project because: THANK YOU FOR YOUR ASSISTANCE! 138 APPENDIX M SCATTERPIDTS PLOT or cs1 u11u SIMULATION l l L l l l 1 11 3 12 1 21 110‘ 1 1 1 1 11 11 13 1 2 1 1 2 11 1 11 11 c 32.5« 2 11 1 11 s 1 111 1 1 1 1 1 1 1 1 1 11 1 551 1 1 1 1 2 1 11 -1% ' 1% ' 4% ' 7% ' 0 so 60 9o SIMULATION PLOT OF BKAT UITH SIWLATICN l l l L l l l 32‘ 1 1 1 11 1 1 12 12 11 11 1 1 1 1 1 a 24- 1 1 1 11 F x 1 12 1 A 1 1 1 1 1 1 2 2 1 1 11 1 161 1 1 1 12 121 2 1 1 1 1 1 8+ 1- -1% ' 1% ' 4% ' 7% o 30 60 9o SIMULATION 9101 or cs1 HIT" BKAT l I l l l l 2 2 4 3 1 1 110‘ 1 1 1 1 1 1 ~ 1 1 1 1 1 2 1 2 1 1 11 1 1 1 1 1 c 82.5- 1 2 1 2 1 s 2 1 1 1 1 1 1 1 1 1 1 1 2 1 55. 1 1 1 3 1 1 1 '13.7% '19.2% r24.7% '30.2 11 16.5 22 27.5 BKAT 139 \ "1 , I _ m 1H s IIIBIIIIIIIII2IIIIII|IIIIIIIIIIIIII8IIIIIIIIIIIIIIIIIIIIII