COUNSELOR EDUCATORS’ EXPERIENCE IMPLEMENTING EVIDENCE-BASED PRACTICE CONCEPTS INTO REHABILITATION COUNSELING EDUCATION PROGRAM By Bilal Urkmez A DISSERTATION Michigan State University in partial fulfillment of the requirements Submitted to for the degree of Rehabilitation Counselor Education – Doctor of Philosophy 2018 ABSTRACT COUNSELOR EDUCATORS’ EXPERIENCE IMPLEMENTING EVIDENCE-BASED PRACTICE CONCEPTS INTO REHABILITATION COUNSELING EDUCATION PROGRAM By Bilal Urkmez Evidence-based practice (EBP) has become popular across helping disciplines (Straus, Glasziou, Richardson, & Haynes., 2010) and is now emphasized within the rehabilitation counseling discipline as a newly integrated model of practice (Leahy & Arokiasamy, 2010; Yaeda et al., 2015). One of the most important reasons for the growth of EBP in the rehabilitation counseling discipline is its value for enhancing the effectiveness of rehabilitation service delivery practice (Chan et al. 2010; Chan, Tarvydas, Blalock, Strauser, & Atkins, 2010). While EBP has permeated and affected a wide array of healthcare, rehabilitation counselors face challenges related to training in EBP knowledge and skills. This in-depth qualitative research study was conducted using grounded theory analysis to answer the research questions of interest in the study, (a) what are rehabilitation counselor educators’ perceptions and experience of teaching EBP concepts? (b) how do rehabilitation counselor educators implement EBP concepts into their curricula? (c) what are the most challenging aspects of implementing EBP concepts in rehabilitation counseling education programs? (d) what resources do rehabilitation counselor educators need to effectively cover EBP in their curricula? Interviews were conducted with 12 rehabilitation counselor educators. The results of this study indicated the following: First, rehabilitation counselor educators hold positive attitudes toward EBP, and educators emphasized that they teach and integrate EBP concepts to their rehabilitation counseling education programs. Second, the participants face some challenges when teaching EBP knowledge and skills to their students. Third, the majority of the participants use different type of educational strategies to effectively cover EBP concepts and overcome teaching and implementing barriers. Lastly, the study results indicated rehabilitation counselor educators needed additional resources, such as in-professional development, a connection between education and practice, and clear accreditation standards to effectively cover EBP concepts in their training program. Further, Implementing Evidence-Based Practice (IEBP) framework was discussed. Lastly, Implications for rehabilitation counselor educators and practitioners, as well as recommendations for future research, are discussed. Keywords: evidence-based practice, implementation of EBP, counselor educators, teaching EBP, rehabilitation counseling, rehabilitation counseling education program. Copyright by BILAL URKMEZ 2018 ACKNOWLEDGEMENTS I would like to take this opportunity to thank the faculty of the Michigan State University (MSU) Rehabilitation Counselor Education Program for providing me the opportunity to be a part of such an outstanding legacy. I would like to thank my advisor, Dr. Michael J. Leahy, for his time, patience, guidance, and his incredibly thorough editing of my work over the years. Thank you for your guidance and advice that made this work better in every way. I want to thank Dr. John Kosciulek, Dr. Elizabeth Heilman, Dr. Gloria Lee, and Dr. Connie Sung for being a part of my dissertation committee and for their guidance and support through this process. During my years at Michigan State University, I had so much support from many people and I had good friends too. My friends have pushed me to keep moving forward, and also have provided me comfort when there were times I needed to move backwards to move forwards again. I would like to thank to my family, my lovely wife Ayse, and my son Burak. I have taken a lot from our family time and thank you for understanding. To Burak, thank him for completing my heart. I hope you grow up realizing that anything is possible with enough hard work; so, shoot for the stars my little boy. My deepest gratitude to Ayse, with all her support, bearing with my absence from my parenting duties especially the last few weeks of my extensive writing period. Finally, I would like to thank to my dad, my mom, my sisters, and my brothers for cheering me up when I needed it. Dad, I know how much you would proud of me, and just imagining your proud face gives me the strength I need for the next steps in my career. v TABLE OF CONTENS LIST OF TABLES .....................................................................................................................ix LIST OF FIGURES .................................................................................................................... x KEY TO ABBREVIATIONS ....................................................................................................xi CHAPTER 1 ............................................................................................................................... 1 INTRODUCTION ...................................................................................................................... 1 Statement and Significance of the Problem .............................................................................. 5 Purpose of the Study ............................................................................................................... 7 Research Questions ................................................................................................................. 8 Summary ................................................................................................................................. 8 Definition of Terms ................................................................................................................. 9 CHAPTER 2 ............................................................................................................................. 11 LITERATURE REVIEW .......................................................................................................... 11 General Overview of Evidence-Based Practice ...................................................................... 11 Brief History of Evidence-Based Practice .............................................................................. 12 Steps for Evidence-Based Practice ........................................................................................ 12 Hierarchical Levels of Evidence ............................................................................................ 15 EBP in Rehabilitation Counseling Practice ............................................................................ 16 Employment ...................................................................................................................... 16 Accountability ................................................................................................................... 19 Regulatory Organizations .................................................................................................. 21 Barriers to EBP Implementation in Rehabilitation Counseling Practice ................................. 22 Evidence-Based Practices and Environmental Factors ........................................................... 25 EBP Interventions in Rehabilitation Counseling Practice ....................................................... 26 Rehabilitation Counselors’ Perceptions and Insufficient Preparation in EBP ......................... 29 EBP Training in Rehabilitation Counseling Education Program ............................................ 32 Major Barriers to EBP Training in Rehabilitation Counseling ............................................... 35 Summary ............................................................................................................................... 36 CHAPTER 3 ............................................................................................................................. 37 METHODOLOGY ................................................................................................................... 37 Introduction........................................................................................................................... 37 Research Design .................................................................................................................... 38 Sampling ........................................................................................................................... 40 Data Collection Procedure and Ethical Consideration ............................................................ 41 Informed Consent .............................................................................................................. 42 Confidentiality .................................................................................................................. 43 The Interview Guide .......................................................................................................... 43 Reliability, Validity and Trustworthiness............................................................................... 45 vi Clarity of Instrumentation.................................................................................................. 45 Member Checks ................................................................................................................ 46 Triangulation ..................................................................................................................... 46 Data Saturation .................................................................................................................. 47 Debriefing ......................................................................................................................... 47 Data Analysis Procedures ...................................................................................................... 48 CHAPTER FOUR ..................................................................................................................... 52 RESULTS ................................................................................................................................. 52 Participants ........................................................................................................................... 53 Participants Profiles .............................................................................................................. 55 Themes ................................................................................................................................. 57 Personal Perceptions about EBP ............................................................................................ 58 Positive Perception about EBP .......................................................................................... 59 Perceived Importance of Teaching EBP ............................................................................. 61 Integrating EBP throughout the Curriculum ....................................................................... 63 Perceived Barriers toward the Implementation of EBP .......................................................... 64 Students’ Academic Development ..................................................................................... 65 Students’ Attitudes toward Research ................................................................................. 68 Educators’ Level of Knowledge ........................................................................................ 71 Lack of Empirical Support ................................................................................................. 73 Common Terminology ...................................................................................................... 77 Lack of Time ..................................................................................................................... 78 Necessary Resources for EBP Training ................................................................................. 79 Professional Development ................................................................................................. 79 Keeping Updated ............................................................................................................... 81 Creating Connections ........................................................................................................ 83 Reshaped Master Curriculum ............................................................................................ 84 Regulatory Organizations .................................................................................................. 87 Focusing on PhD Education .............................................................................................. 88 Educational Strategies for Teaching EBP .............................................................................. 89 Case Study Approaches ..................................................................................................... 90 Using Research Methods Class .......................................................................................... 92 Creating Connections with Practice ................................................................................... 94 Analyzing Empirical Research Articles ............................................................................. 96 Overview of Implementing Evidence-Based Practice Framework .......................................... 98 Narrative Explanation of the Framework ............................................................................. 101 Summary ............................................................................................................................. 102 CHAPTER FIVE .................................................................................................................... 104 DISCUSSION ......................................................................................................................... 104 Introduction......................................................................................................................... 104 Personal Perceptions about EBP .......................................................................................... 105 Perceived Barriers toward the Implementation of EBP ........................................................ 107 Necessary Resources for EBP Training ............................................................................... 113 Educational Strategies for Teaching EBP ............................................................................ 115 vii Strengths and Limitations of the Study ................................................................................ 118 Implications of the Study..................................................................................................... 120 Implications for Rehabilitation Counseling Education Program ....................................... 120 Implication for Practitioners ............................................................................................ 124 Implication for Research and Theory ............................................................................... 125 Implication for Future research ........................................................................................ 126 Conclusion…………………………………………………………………………………....129 APPENDICES ........................................................................................................................ 131 Appendix A: Informed Consent Document .......................................................................... 132 Appendix B: Interview Guide .............................................................................................. 135 Appendix C: Participant Demographic Information ............................................................. 136 REFERENCES ....................................................................................................................... 138 viii LIST OF TABLES Table 1. Hierarchical Levels of Evidence…………………………………………………….15 Table 2. Participants Demographic Characteristics…………………………………………..54 Table 3. An Outline of the Main Themes and Sub-Themes ………………………………….58 ix LIST OF FIGURES The Figure 1. Visual Model of 5 steps in Evidence Based Practice Based Upon the Sackett et al. (1996) Model …………………………………………………….…...….…13 The Figure 2. Implementing Evidence-Based Practice Framework (IEBP) in Counselor Education …………………………………………………………………….……………100 x KEY TO ABBREVIATIONS CACREP The Council for Accreditation of Counseling and Related Educational Programs RC Rehabilitation Counselor RCE Rehabilitation Counselor Educator RCEP Rehabilitation Counseling Education Program EBP Evidence-Based Practice VR Vocational Rehabilitation xi CHAPTER 1 INTRODUCTION Evidence-based practice (EBP) has become popular across helping disciplines (Straus et al., 2010; Tanner, 2008; Howard, Allen-Meares, & Ruffolo, 2007) and it is now emphasized within the rehabilitation counseling discipline as a newly integrated model of practice (Chan et al., 2009; Leahy & Arokiasamy, 2010; Tansey et al., 2014; Yaeda et al., 2015). One of the most important reasons for the growth of EBP in the rehabilitation counseling discipline is its value for enhancing the effectiveness of rehabilitation service delivery practice (Chan et al. 2010; Chan, Tarvydas, Blalock, Strauser, & Atkins, 2009). EBP emphasized that rehabilitation counselors (RC) should have an interest in delivering the best possible services to their clients, based whenever possible on the best clinical practices available from the research evidence (Chan, Tarvydas, Blalock, Strauser, & Atkins, 2009). Rehabilitation counselors provide a large portion of vocational and psychological services to a diverse client population with unique presenting problems and in varied settings and EBP helps the counselors to provide effective rehabilitation services (Leahy et al., 2014). These effective services help people with disabilities to maximize their opportunity for employment, economic self-sufficiency, and independence; inclusion in society is a paramount goal of rehabilitation counseling (Chan et al. 2016). Further, the use of EBP enables the counselor to fulfill their ethical obligation to their clients by better protecting consumers from harm (non- maleficence), improving efficiency in the utilization of scarce resources (justice), and allowing consumers to exercise knowledgeable self-determination and informed choice (autonomy) (Chan et al. 2016; Tarvydas, Addy & Fleming, 2010; Chan et al. 2011; Chan et al. 2010; Chan, Tarvydas, Blalock, Strauser, & Atkins, 2009; Leahy & Arokiasamy, 2010;Tansey, 1 Bezyak, Chan, Leahy, & Lui, 2014). Due to the current emphasis in EBP for enhancing the effectiveness of rehabilitation services and ethical considerations, RCs have devoted energy to understanding how EBP can be helpful to people with disabilities and how best to integrate EBP in their practice setting, given its benefits and challenges in providing direct services to clients. Rehabilitation counselors need to know that specific vocational rehabilitation (VR) services provide successful employment outcomes and help people with disabilities live independently. Although the general effectiveness of vocational rehabilitation counseling has been empirically demonstrated (Pruett, Rosenthal, Swett, Lee & Chan, 2008), there is a serious lack of specific EBP that accurately defines which specific VR services produce which employment outcomes for individuals with disabilities (Leahy & Arokiasamy, 2010). State- Federal VR agencies need to know that service provision patterns have a high probability of predicting successful outcomes for VR consumers (Chan, Taryvdas, Blalock, Strauser, and Atkins, 2009). The powerful question posed by Paul (1967): “What treatment, by whom, is most effective for this individual with that specific problem, and under which set of circumstances?” (p. 111) remains for the most part unanswered within the public rehabilitation program (Leahy et al., 2009). Answers to this question will provide VR counselors with information regarding which services and interventions have a high probability of predicting successful outcomes for VR customers (Fleming, Del Valle, Muwoong, & Leahy, 2013). In addition to determining which approach works best for whom, how, and under what conditions, the state-federal vocational rehabilitation program has been challenged to demonstrate the effectiveness of VR services (Rubin, Chan & Thomas, 2003). For example, the Work Incentives Improvement Act of 1999 and the Workforce Investment Act of 1998 require the state-federal VR program to demonstrate service efficacy to maintain and expand program 2 funding and services (Kosciulek, 2004). The state-federal vocational rehabilitation program was placed by the U.S. Government Accountability Office (GAO, 2005) on a list of high-risk programs for not keeping up with scientific advances and social changes (GAO, 2005). Further, the Rehabilitation Services Administration (RSA) has begun to encourage state vocational rehabilitation agencies to integrate the best scientific evidence with clinical expertise and client perspectives. In addition to the GAO and RSA, the National Institute on Disability, Independent Living, and Rehabilitation Research (NIDILRR) emphasized developing EBP in all research projects publicly funded through NIDILRR (Brannon, 2010). The NIDILRR has also emphasized development of research studies to meet standards for inclusion in evidence-based systematic reviews (Schlosser, 2006). There is now a heightened emphasis on the meaning of research findings and translating and disseminating EBP, so they affect and inform practice and policy (Leahy & Arokiasamy, 2010). Without a doubt, VR agencies and RCs are under increasing pressure to demonstrate that they are using empirically supported intervention to improve the effectiveness of rehabilitation service delivery (Chan et al.2009; Del Valle et al., 2014; Sherman et al., 2014). In order to decrease these pressures and improve the effectiveness of rehabilitation services to provide the best services for people with disabilities, RCs must become more knowledgeable about EBP concepts. As Tansey et al. (2014) have stated, rehabilitation professionals must become more knowledgeable about EBP and effective VR service delivery practices to maximize the impact of services on the lives of persons with disabilities. Specifically, rehabilitation counselors need to know the EBP decision-making model, which includes: (a) developing answerable clinical questions to clinical issues, (b) knowing how and where to locate the best available evidence, (c) critically appraising the evidence, (d) making meaningful translations to effectively apply the 3 evidence to a diverse clientele, and (e) evaluating outcomes of interventions applied (Straus, Glasziou, Richardson, & Haynes, 2011; Kosciulek, 2010; Chan et al., 2010). While EBP has permeated, and affected a wide array of healthcare, RCs face challenges related to training in EBP knowledge and skills. Several qualitative studies found that RCs generally hold positive attitudes toward the use of EBP to improve their service and counseling outcomes (Fitzgerald, Leahy, Kang, Chan, & Bezyak, 2017; Pfaller et al., 2016; Tansey, Bezyak, Chan, Leahy, & Lui, 2014; Grahman et al., 2006; Yeada, Iwanaga, Fujikawa, Chan, &, Beztak, 2015; Bezyak, Kubota & Rosenthal, 2010). However, these studies also identified several major barriers to implementing EBP including a lack of knowledge and insufficient academic preparation. Rehabilitation counselors also reported that they had limited time for continuing education or advanced training in EBP (Corrigan, Steiner, McCracken, Blaser, & Barr, 2008). Moreover, a lack of library and internet resources, limited training in academic search strategies, and low agency support have also been reported as obstacles to implementing EBP (Bezyak, Kubota, & Rosenthal, 2010; Chronister, Chan, da Silva Cardoso, Lynch, & Rosenthal, 2008; Tansey, Bezyak, Chan, Leahy, & Lui, 2014). There is insufficient first-hand information regarding the underlying causes of a lack of knowledge and insufficient academic preparation in EBP for rehabilitation counseling students. In addition to first-hand information regarding the reasons of these major barriers, these studies also focused on RCs’ attitudes toward EBP. There are no data show that from rehabilitation counselor educators’(RCE) perception and experience of teaching in EBP knowledge and skills in rehabilitation counseling education program (RCEP). In order to identify reasons for these major barriers, additional data is needed to understand how EBP training is offered currently in RCEP, what major barriers to implement EBP concepts in RCEP, what resources RCE need to 4 effectively cover EBP in their curricula? Additional data also needed to help RCE modify their curriculum to include a strong emphasis on evidence-based methodologies, knowledge translation, and research utilization. This study was provided insight through research in the area of EBP training experiences of rehabilitation counselor educators. The current study was necessary for multiple reasons. The existing literature primarily uses quantitative information to show professionals training needs of rehabilitation counselor practitioners. These studies demonstrated several major barriers to employing EBP including, a lack of knowledge, and insufficient training and empirically validated psychological and VR interventions (Fitzgerald et al., 2016; Pfaller et al., 2016; Tansey et al., 2014; Bezyak, Kubota & Rosenthal, 2010). The available literature suggests that existing EBP training is not satisfactory, failing to provide instruction that includes evidence-based methodologies, knowledge translation, research utilization, and training in academic search strategies. In addition, there is little descriptive research on rehabilitation counselor educators’ perceptions of EBP and teaching EBP knowledge skills. Thus, a qualitative study was necessary to understand the nature of teaching EBP experiences in rehabilitation counseling curriculum. Findings from this study can be applied to the further development of rehabilitation counselor education programs. Statement and Significance of the Problem Rehabilitation counselors play a vital role in assisting people with chronic illness and disability to achieve their independent living and employment goals, leading to better health and quality of life (Dean, Pepper, Schmidt, & Stern, 2014; Dutta et al., 2008; Leahy et al., 2014; Martin, West-Evans, &Connelly, 2010). Given the importance of EBP in rehabilitation counseling practice (Chan et al., 2011; Leahy & Arokiasamy, 2010), training in EBP that can effectively bridge science and practice to improve rehabilitation counseling students’ EBP 5 knowledge and skills seems to be critically needed. Explicit training for EBP in rehabilitation counselor education can supply master’s level rehabilitation counseling students with the knowledge and skills necessary for giving effective support while providing rehabilitation services to clients. However, there are training challenges in EBP for rehabilitation counselors. Recently, Fitzgerald et al. (2016) explored certified rehabilitation counselors’ perceptions of the effectiveness of EBP and their preparedness for implementing EBP within rehabilitation counseling settings. The study’s findings highlighted the importance of preparation and training to use EBP. Although some of the participants reported that they received instruction in EBP during their more recent graduate training, many of the participants disclosed that they did not receive sufficient training for EBP in their master’s level curriculum and were not provided adequate on-the-job training in their respective employment settings. In other words, they did not have sufficient skills and knowledge to effectively implement EBP in daily practice. The majority of the participants in this study expressed a strong desire to receive more continuing education in EBP. Tansey, Bezyak, Chan, Leahy & Lui (2014) found that counselors in four state-federal VR agencies were insufficiently trained to incorporate EBP into practice. Similarly, Bezyak, Kubota & Rosenthal in their 2010 study reported a general lack of knowledge about evidence- based practice and insufficient academic preparation. Although rehabilitation counselors seem to hold positive attitudes towards EBP, they have insufficient academic preparation in EBP to implement it (Bezyak et al., 2010). This study likewise found that one of the clearest obstacles to acting on this identified interest is a lack of academic preparation for using evidence in rehabilitation counseling practice, particularly for master’s level respondents. 6 Chronister et al. (2008) stated that rehabilitation counselors had inadequate research training for understanding empirical research findings and that they needed to become more knowledgeable about evidence-based practice. It is also unclear whether rehabilitation counselors have the research training and motivation to be evidence-based rehabilitation practitioners. Therefore, rehabilitation counselor educators need to more adequately prepare counselors to effectively implement evidence-based practice. As Chan et al. stated, rehabilitation counselor educators must modify rehabilitation counseling curriculum to help students become more familiar with research concepts related to EBP (Chan et al., 2009). Studies have demonstrated that rehabilitation counselors do not have sufficient training in EBP. These studies also raise important questions about how to train the future rehabilitation counselors. As many researchers have noted, education addressing rehabilitation counselors’ EBP skills and knowledge needs to be strengthened and expanded (Chan et al., 2009). Rehabilitation counseling students need better training in EBP skills and knowledge, which will allow them to find, interpret, and apply evidence in daily practice. As Chan et al (2010) have pointed out, rehabilitation counselor educators must modify their curriculum to include a strong emphasis on evidence-based methodologies, knowledge translation, and research utilization. Purpose of the Study Given the importance of teaching EBP, it is important to understand rehabilitation counselor educators’ experience and perceptions of implementing EBP concepts in rehabilitation counseling education program. The purpose of this study was to explore RCEs’ experience and perceptions in implementation of EBP concepts into RCEP. Additionally, this study investigated where EBP concepts are addressed within the curriculum, including challenging aspects of 7 implementing and teaching these concepts, as well as what resources RCEs need to appropriately cover EBP. Research Questions As such using a grounded theory approach the purpose of study was to collect, summarize, and report data related to teaching experiences from the perspective of rehabilitation counselor educators. The following research questions were of interest to this study: 1. What are rehabilitation counselor educators’ perceptions and experience in implementation of EBP concepts into rehabilitation counseling education programs? 2. How do rehabilitation counselor educators implement EBP concepts into their curricula? 3. What are the most challenging aspects of implementing EBP concepts in rehabilitation counseling education programs? 4. What resources do rehabilitation counselor educators need to effectively cover EBP concepts in their education program? Summary To address the research questions, phenomenographic research study was conducted using grounded theory analysis. Grounded theory approach was utilized to collect, summarize, and report data from rehabilitation counselor educators’ experience and perspectives in implementation of EBP concepts in rehabilitation counseling education program. This data about the teaching experiences of current rehabilitation counselor educators. Data resulting from this study may be used to identify major barriers teaching and implementing EBP concepts in rehabilitation counseling curricula. Also, this study can be used to identify effective teaching strategies for overcoming these barriers. 8 Definition of Terms The following terms are offered for clarification: Rehabilitation Counselor. A counseling practitioner with a master’s degree who assists persons with disabilities with physical, mental, developmental, cognitive, and emotional disabilities to achieve their personal, career, and independent living goals in the most integrated setting possible through the application of the counseling process. Techniques and modalities used by rehabilitation counselors may include, but are not limited to: (a) assessment and appraisal; (b) diagnosis and treatment planning; (c) career (vocational) counseling; (d) individual and group counseling; (e) case management, referral, and service coordination, (f) program evaluation and research; (g) interventions to remove environmental, employment, and attitudinal barriers; (h) consultation services among multiple parties and regulatory systems; (i) job analysis, job development, and placement services, including assistance with employment and job accommodations; and (j) the provision of consultation about and access to rehabilitation technology (CRCC Scope of Practice, 2016). Rehabilitation Counseling. Rehabilitation counseling is focused on helping people who have disabilities achieve their personal, career, and independent living goals through a counseling process. Rehabilitation Counselor Education. As specialists in providing counseling services for persons with disabilities, rehabilitation counselors promote adaptation, achievement, social justice, and improved quality of life for clients with an incredibly diverse range of developmental, congenital, and acquired disabilities. Evidence-Based Practice (EBP). EBP is a clinical decision-making process beginning with formulating clinical questions to ask, determining the best practice, and critically appraising the 9 evidence for validity and applicability to the situation. The best evidence must then be applied by a clinician with expertise in considering the client’s unique values and needs. The final aspect of the process is evaluation of the effectiveness of care and continual improvement of the process. Clinical Decision-Making. Drawing conclusions based on clinical expertise, patient preferences, and factors such as cost-benefit analysis, availability of alternatives, etc. Randomized Controlled Trial (RCT). RCT is a type of scientific (often medical) experiment which aims to reduce bias when testing a new treatment. Meta-analysis. A systematic review that uses quantitative methods to synthesize and summarize the results; Merged statistical results from a number of related studies. CACREP. The Council for Accreditation of Counseling and Related Educational Programs. 10 CHAPTER 2 LITERATURE REVIEW The purpose of this study was to explore rehabilitation counselor educators’(RCE) experience and perceptions in implementation of evidence-based practice (EBP) concepts into rehabilitation counseling education programs (RCEP). Additionally, this study investigated where EBP concepts are addressed, including challenging aspects of teaching EBP, as well as what resources RCE need to appropriately cover EBP concepts in their training programs. General Overview of Evidence-Based Practice Sackett, Straus, Richardoson, Rosenberg, and Haynes (2000) defined EBP as the integrating of the best research with clinical expertise and patient values. DePalma (2002) provided a comprehensive description of EBP as a clinical decision-making process that involves formulating clinical questions, determining the best practice, and critically appraising the evidence for validity and applicability to the particular situation. The best evidence must then be applied by a clinician with expertise, taking into consideration the client’s unique values and needs. The final aspect of the process is the evaluation of the effectiveness of care, as well as continual improvement of the process (DePalma, 2002; Sackett, Rosenberg, Gray, Haynes, & Richardson, 1996). EBP can also be defined as an explicit and judicial method of using best practice in making decisions regarding the care and treatment of patients (Sackett et al., 1996). Moreover, EBP specifically includes “practices that are informed by research, in which the characteristics and consequences of environmental variables are empirically established, and the relationship directly informs what a practitioner can do to produce a desired outcome” (Dunst, Trivette, & Cutspec, 2002, p. 3). Inherent in both of these definitions is the development of competence in the decision-making process related to EBP implementation. 11 Brief History of Evidence-Based Practice EBP in medicine and health care is not a new concept. The history of evidence-based medicine has been well documented and includes primitive experiments to test its effectiveness (Claridge & Fabian, 2005). In 1972, Archie Cochrane pointed out the importance of properly testing the effectiveness of health care strategies and stressed the role of randomized controlled studies to provide evidence on which health care is based. The term "evidence-based medicine" was introduced by Guyatt et al in 1992 to shift the emphasis in clinical decision-making from "intuition, unsystematic clinical experience, and pathophysiologic rationale" to scientific, clinically relevant research of practices. “EBP in psychology is consistent with the past twenty years of evidence-based medicine which, advocates for improved patient outcomes” (APA Presidential Task Force on Evidence-based Practice, 2006, pg. 271). In 1996, Sackett et al explained evidence-based clinical decision-making as a combination of not only research evidence but also clinical expertise which takes into consideration the patient’s preferences. Steps for Evidence-Based Practice According to Kazdin (2008), evidence-based treatments, interventions or techniques that have produced therapeutic changes through clinical trials. This clinical trial outlines five specific steps for clinical decision-making: a) formulating the questions, b) seeking the best evidence available to answer the questions, c) critically appraising the evidence, and (d) applying the evidence to the individual client (e) evaluate (Walker, Seay, Solomon & Spring, 2006). The figure below describes five steps for EBP process. 12 1. Ask Question 5. Evaluate 4. Applying 2. Search 3. Critically Appraise al. (1996) Model Figure 1. Visual Model of 5 steps in Evidence Based Practice Based Upon the Sackett et Formulating well-defined questions for a client’s problem is the first step in evidence- based practice. The ability to ask appropriate questions plays an important role in the clinical decision-making process. Specifically, background questions explore a setting or context, whereas foreground questions focus upon a specific case within that context (Walker et al., 2006). Example of question include: what are the most effective treatments for presenting problem A? Is treatment X an effective treatment for presenting problem A? Are there any significant risks associated with treatment X? Regarding foreground questions, a practitioner should be questioned using PICO format: patient group (P), intervention (I), comparison group (C), and outcome measures (O). For example, in the case of an African American man with schizophrenia (patient group), one might question whether there is any evidence that the individual placement and support (IPS) model of supported employment (intervention) is superior to state VR services, assertive community 13 treatment, and the clubhouse approaches (comparison group) in improving his employment outcome and the quality of his outcome (outcome). After formulation of the clinical question(s), the next step is to search for evidence-based interventions related to the clinical question. In this step, practitioners can use the most reliable and scholarly approach to searching for “best evidence,” for instance, through academic databases such as Academic Search Elite, CINAHL, MEDLINE, and PsycINFO. According to Chan et al (2010), the most efficient way to find best evidence is to use databases and specific evidence-based intervention websites to search for systematic reviews. The third step is critically appraising the evidence. Critically appraising evidence from a single properly designed randomized control trial requires a relatively strong background in research methods and a working knowledge of concepts related to internal and external validity (Schlosser, 2012). After appraising the research evidence, the rehabilitation professionals must incorporate the evidence into a client's treatment plan by considering the significance of the evidence. They would then apply the evidence to the individual client with incorporating evidence into a client’s treatment plan based on the significance of evidence, client characteristic, and practitioner expertise and judgment. This clinical decision-making process is critically important and a top priority in rehabilitation counselor education (Leahy & Arokiasamy, 2010; Shultz et. al., 2007; Shaw et al., 2006). Also, these five steps can be used by rehabilitation counselors in their clinical decision- making process because clinical decision-making directs rehabilitation counselors to use the best available evidence in relation to interventions that lead to the best possible client outcomes (Chan et al., 2009). In order to provide effective services in daily practice, rehabilitation 14 counselors should have this set of skills, and they need to receive training in their master's level program for the clinical decision-making model and process. Hierarchical Levels of Evidence After formulating well-defined, answerable clinical questions and seeking the best evidence available to answer these questions, RCs must critically appraise the scientific evidence. Hierarchy of evidence obtained from empirical study can been established, with the highest levels of evidence coming from a systematic review of several randomized controlled trials (RCT) of an intervention, and lower levels of evidence obtained from expert opinion (Pruett, Rosenthal, Swett, Lee, & Chan, 2008). RCTs are given the highest level because they are designed to be unbiased and have less risk of systematic errors. For example, by randomly allocating subjects to two or more treatment groups, these types of studies also randomize confounding factors that may bias results. Holm (2000) described a leveled hierarchy of evidence from which professionals can formulate an empirical consensus and determine “best evidence” regarding the effectiveness of treatment approach. Levels of Evidence Level 1 Level 2 Level 3 Level 4 Level 5 Description Strong evidence from at least one systematic review of multiple well-designed randomized controlled trials. Strong evidence from at least one properly designed randomized controlled trial of appropriate size. Evidence from well-designed trials without randomization, single group pre-post, cohort, time series studies. Evidence from well-designed non-experimental studies from more than one center or research group. Evidence is defined as opinions of respected authorities, based on clinical evidence, descriptive studies, or reports of expert’s committees. Table 1. Hierarchical Levels of Evidence 15 This five-level hierarchical framework offers a format for determining strength of evidence based on methodological rigor (Holm, 2000; & Nathan and Gorman 1998). In the hierarchical levels of evidence, rehabilitation counseling research typically falls within Level 4 and Level 5. As Law stated (2002) rehabilitation counseling practice is still characterized as experience-based or habit-based. Rehabilitation research is usually non-experimental or quasi- experimental, therefore, rehabilitation counseling researchers must provide quality evidence with a strong theoretical basis. However, rehabilitation counseling professionals should be mindful that randomized clinical trials utilized as the gold standard for scientific evidence, while they often work well in medicine, may be too restrictive for other behavioral and social sciences since they do not always consider the full complexity of human behavior and clinical condition (Wampold, 2004). EBP in Rehabilitation Counseling Practice Employment. Work is central to the life and well-being of adults (Blustein, 2008). Employment provides the individual worker with social, financial, and psychological benefits that allow adults to live as independent and productive citizens. Employment allows individuals to earn money, pursue their interests, and interact with others, while simultaneously contributing to an individual’s feelings of self-worth (Hendricks, 2010). Employment improves well-being, is shown to enhance an individual’s relationship to both the social and economic world and allows the individual who is working to feel satisfaction and accomplishment (Blustein, 2008). In the general population, an individual’s participation in the labor market has a direct impact on his or her quality of life (Fleming, Fairweather & Leahy, 2013). The same is true for individuals with disabilities, as their participation in the work-force has been found to improve their self-reported QoL rating (Stodden & Mruzek, 2010). Further, research demonstrates that there is a relationship 16 between unemployed individuals and problems such as anxiety disorders, using alcohol more frequently, and reporting a lower level of self-esteem (Dutta, Gervey, Chan, Chou, & Ditchman, 2008). In the United States, one out of every five adults have a disability, according to a new study published by the Centers for Disease Control and Prevention (2015). According to the U.S. Bureau of Labor, the labor force participation rate for people with chronic illness and disability in the United States is unacceptably low, at 20.4%, compared to 68.3% of people without disabilities (U.S. Department of Labor [DOL], 2016). The United States vocational rehabilitation (VR) service is administered by the US Department of Education. The goal of the VR is to increase independence in adults with disabilities and to maximize employment outcomes. The State–Federal VR program serves approximately one million individuals a year and the U.S. federal government spends approximately $3 billion dollars annually (Saunders, 2005). In order to be determined eligible to receive VR services, an individual must have a physical or mental impairment that creates a barrier to employment, need VR services to prepare for, secure, retain, or regain employment, and be able to benefit from VR services. Examples of potential VR services are assessment, job placement assistance, assistive technology, and on-the-job training. VR categorizes employment in two ways: competitive employment and supported employment. Competitive employment is in an integrated setting in which the individual is making minimum wage or above, and the individual may or may not need supports on the job. An “integrated setting” is defined as a setting in the community in which the employee with a disability interacts with people without disabilities, that is, people other than those professionals providing vocational services to the individual. Supported employment does not necessarily meet 17 the minimum wage requirement and focuses on enabling the person with a disability to maintain paid employment that is in an integrated work environment (Mawhood & Howlin, 1999). One of the most important functions of this program is to assist individuals with disabilities to find and retain employment opportunities (McMahon, 2009; Chan, Leahy, Saunders, Tarvydas, Ferrin & Lee, 2003). Therefore, extensive efforts have been devoted to understanding and promoting employment situations for individuals with disabilities. VR including both public sector and private not-for-profit sector services, has been shown to be an effective method for improving the employment outcomes among individuals with disabilities. For example, over half of the individuals with disabilities have found employment in an integrated setting after receiving VR services (Dutta et al., 2008). Dean et al. (2008) analyzed data of individuals with cognitive impairments who applied for VR services in the state fiscal years 1988-2000. This study found that VR services have substantial positive returns with average benefits. However, among 326,627 individuals who either initiated or completed services through the state-federal VR in federal fiscal year 2011, only 175,441 achieved a successful employment outcome for a 53.7% rehabilitation rate (Annual Disability Statistics Compendium, 2013). Further, the average hourly earnings of all VR consumers in competitive employment are only 52% of the general workforce (RSA, 2008). The effectiveness of rehabilitation counseling has been empirically demonstrated (Pruett, Rosenthal, Swett, Lee & Chan, 2008), however there is limited research regarding rehabilitation counselor use of EBP in VR service delivery. There is still substantial room to improve the quality of VR services, which will lead to better employment outcomes and employment quality for people with chronic illness and disability. Without doubt, the state-federal VR program needs to do a better job to decrease this disparity of employment rate between people with and without 18 a disability, and to improve employment outcomes. Rehabilitation counselors (RC) must provide the most effective psychosocial and vocational interventions possible, by integrating scientific evidence with clinical expertise and client perspectives to help people with chronic illness and disability to find career pathways (Chan, Wang, Muller, & Fitzgerald, 2011). In other words, RC must to know what EBP is and how to apply it in their practice setting to improve competitive employment outcomes for their clients who are in VR settings. Accountability. With increased healthcare costs and with shrinking profit margins and public funding, stakeholders, employers, and government agencies are asking hard questions about expected outcomes, efficacy of VR, and are demanding that we provide objective evidence to justify their source of support for services (Chan, Rosenthal, & Pruett, 2008). In recent years, there has been pressure to demonstrate accountability for employment outcomes of those served by the state-federal system through the development and use of evidence-based practices (Sherman et al 2014.). Therefore, the state-federal VR agencies have an obligation to demonstrate that they have met expected service outcomes and have done so in an efficient manner (Rubin et al., 2003). While the state-federal VR program has been relatively successful in assisting individuals with disability to obtain and maintain employment, these state-federal agencies are under pressure to develop a foundation of EBP that will lead to competitive employment outcomes for people with disabilities (Del Valle et al, 2014). For example, the Work Incentives Improvement Act of 1999 and the Workforce Investment Act of 1998 require the state-federal VR program to demonstrate service efficacy in order to maintain program funding and services (Kosciulek, 2004). The state-federal vocational rehabilitation program was placed by the U.S. Government Accountability Office on a list of high-risk programs for not keeping up with scientific advances 19 and social changes. The state-federal VR agencies have been asked to: (a) set more ambitious goals, (b) keep up with scientific advances, (c) improve accountability, and (d) strengthen its management practice (U.S. General Accountability Office, 2005). Rehabilitation Services Administration (RSA) has begun to encourage state vocational rehabilitation agencies to integrate the best scientific evidence with clinical expertise and client perspectives (Thirty-Third Institute for Rehabilitation Issues, 2008). RSA has recently funded six national technical assistance centers to help state VR agencies and their counselors to provide empirically supported VR services. Furthermore, the National Institute of Disability and Rehabilitation research (NIDRR) emphasized developing EBP in all research projects publicly funded through NIDRR (Brannon, 2010). The NIDRR has emphasized development of research studies to meet standards for inclusion in EBP systematic reviews (Schlosser, 2006). The agency also funded a research and training center on evidence-based practice in VR and several knowledge translation centers. RCs are increasingly being asked to use empirically supported interventions to improve the effectiveness of rehabilitation service delivery practices (Chan et al., 2016; Leahy, Chan, Lui, Rosenthal, et al., 2014; Rubin, Chan, & Thomas, 2003). Clearly, EBP is receiving increased attention from VR professionals as the demand for accountability in rehabilitation and healthcare systems increases. The state-federal VR program is at a crossroads and must now develop EBPs that demonstrate the effectiveness of the VR services provided to individuals with disabilities which lead to competitive employment outcomes (Fleming, Del Valle, Muwoong, & Leahy, 2013). This pressure encourages an increased focus on research to improve effectiveness of VR services for people with disabilities who are in a VR setting. Rehabilitation counselors cannot ignore EBP, as the movement toward accountability will undoubtedly affect state VR agencies. As Chan et al. (2009) suggested, the 20 rehabilitation counseling discipline can no longer make clinical decisions based on logic and intuition because managed care and accountability for outcomes of government programs and providers will continue to increase. Regulatory Organizations. Two of the most important regulatory organizations for rehabilitation counselors are the Council for Accreditation of Counseling and Related Educational Programs (CACREP) and the Commission on Rehabilitation Counselor Certification (CRCC). The CACREP is the official organization that accredits graduate programs that prepare professionals counselors, counseling supervisors, counselor educators and counseling psychologists. In order for a program to receive accreditation, its curriculum must be assessed by the CACREP. These standards detail the knowledge and skills that should be included in professional preparation programs. Among the standards that stress different knowledge domains, EBP is emphasized in several areas. Another important regulatory organization for rehabilitation counselors is the Commission on Rehabilitation Counselor Certification (CRCC). CRCC’s mission is to set the standard for quality rehabilitation counseling services in the United States and Canada (CRCC, 2016). Therefore, while CACREP sets standards for professional preparation programs, CRCC sets standards for qualified individuals to become certified rehabilitation counselors (CRCs). The Commission on Rehabilitation Counselor Certification (CRCC) Scope of Practice Statement (2014) states it is “a systematic effort to collect, analyze, and interpret quantitative or qualitative data that describe how social characteristics, behavior, emotions, cognition, disabilities, mental disorders, and interpersonal transactions among individuals and organizations interact”. RCs are ethically bound as part of the Code to follow such EBP. The use of EBP enables a counselor to fulfill their ethical obligation to clients by better protecting consumers from harm 21 (non-maleficence), improving efficiency in the utilization of scarce resources (justice), and allowing consumers to exercise knowledgeable self-determination and informed choice (autonomy) (Chan et al. 2016; Chan et al., 2011; Chan, Tarvydas, Blalock, Strauser, & Atkins, 2009; Leahy & Arokiasamy, 2010; Tansey, Bezyak, Chan, Leahy, & Lui, 2014). The CRCC’s professional code of ethics reveals that rehabilitation counselors have a responsibility to the public to engage in practices that are based upon accepted research methodologies and evidence- based practices (CRCC, 2016). Since January 1st, 2017, CRCC has modified the professional responsibility area in its professional code of ethics. Specifically altered were section D, which requires competency to provide competent services to their clients and not to provide services not personally competent to render. For example, Section D.6.a. is a new addition to the 2017 Code and requires that “Rehabilitation counselors use techniques/procedures/modalities that are grounded in theory and/or have an empirical or scientific foundation” (CRCC, 2017, p. 15). Similarly, Section D.6.c., states that “rehabilitation counselors do not use techniques/ procedures/ modalities when evidence suggests the likelihood of harm, even if such services are requested” CRCC,2017, p.16). To provide services that have a scientific foundation and to ensure that resources are valid, rehabilitation counselors need to be able to access the research literature, assess its scientific rigor, and implement it in practice. Under the new ethics code, rehabilitation counselors will need to be aware of EBP, so they can develop effective plans that will work with specific clients in specific situations. Knowledge about EBP should strengthen a counselor’s ability to assess a plan’s viability and effectiveness and modify the plan as needed. Barriers to EBP Implementation in Rehabilitation Counseling Practice Multiple barriers to adopting EBP in rehabilitation counseling have been identified in the literature (Fleming et al., 2011; Johnston et al., 2009; Leahy & Arokiasamy, 2010; Tarvydas et 22 al., 2010, Chan et al., 2010). One of the major barriers to EBP implementation in rehabilitation counseling is the challenge of training practitioners to utilize research and results appropriately in practice (Chan et al., 2008). Current rehabilitation practitioners reported insufficient graduate training in basic concepts of research, evidenced-based methods, knowledge, and utilization (Fitzgerald, Leahy, Kang, Chan, & Bezyak, 2017; Graham et al., 2006; Pfaller et al., 2016; Tansey, Bezyak, Chan, Leahy, & Lui, 2014; Yaeda, Iwanaga, Fujikawa, Chan, & Bezyak, 2015). According to Tansey et al. (2014) rehabilitation counselors agree that the application of EBP is necessary and useful and will lead to improved quality of client services. In addition, attitudes of rehabilitation counselors surveyed were positive regarding interest in learning skills. Despite positive attitudes, insufficient academic preparation and a lack of knowledge about EBP were reported by rehabilitation counselors (Tansey et al. 2014; Bezyak et al. 2010). Rehabilitation counselors may not possess the skills and knowledge necessary to evaluate and incorporate research findings into practice setting and so may not be familiar with research- related terms (Chan et al., 2010). For example, a lack of understanding of terms such as “systematic reviews,” “randomized controlled trials,” “meta-analysis,” and “effect size” may easily prevent rehabilitation counselors from applying EBP interventions and techniques. Another major challenge to implementing EBP interventions in rehabilitation counseling is the scope of rehabilitation services (Chan et al. 2010). The rehabilitation process encompasses a broad array of services and is provided through a number of disciplines, such as physical therapy, occupational therapy, and rehabilitation counseling for individuals within diverse populations. The process involves both personal and environmental factors and interactions, which make it difficult to determine which aspects of service delivery affect which outcomes (Johnston, Stineman, & Velozo, 1997). Bolton (2004) stated that it would be impossible to 23 design and implement an experimental study to explore the overall effect of state VR services because of ‘order of selection’ criteria. The spread of rehabilitation counseling in multiple service settings has led not only to specialty areas of rehabilitation counseling practice, such as transition, but has made rehabilitation research very complex (Strauser & Wong, 2010). Lack of scientific rigor and sophistication is another common criticism of rehabilitation counseling research (Arokiasamy, 1994). Rehabilitation researchers must provide quality evidence with a strong theoretical basis to implement EBP effectively. As Bolton (2004) suggests, rehabilitation research is frequently non-experimental or quasi-experimental. Lack of randomized controlled trials is often seen as a major barrier to the successful implementation of rehabilitation counseling research in practice (Chwalisz & Chan, 2008). Insufficient time continues to be reported as a barrier to applying research evidence to practice. O’Donnel (2004) reported that time is a major barrier to implementing evidence-based practice since practitioners are often required to spend at least eight hours per day providing direct service, and as a result, there is limited time for training on new research evidence. Similarly, Nelson & Steele (2007) pointed out that limited access to evidence, big caseloads, and limited time in the work setting serve as a barrier to evidenced-based practices. In another study, an estimated 83% of respondents ranked insufficient time as one of top three barriers to implementing EBP (Beyzak et al., 2010). The inability to consistently devote time to collecting new research evidence, carefully reviewing findings, and developing implementation strategies must be addressed before EBP can be successfully applied to all areas of rehabilitation counseling. Additional barriers are inherent to the nature of rehabilitation research, as explained by Strauser & Wong (2010), who include studies in rehabilitation that use small sample sizes and do 24 not use or analyze blinding or placebo effects; the broad multidisciplinary nature of rehabilitation and rehabilitation counseling, in which multiple outcomes are appropriate for the individuals served (i.e. employment, independent living, etc.); the complexity of rehabilitation service delivery, which complicates the rehabilitation research process; and ethical challenges to including a control group. To sum up, although EBP has been identified as powerful and efficient in improving the quality of VR services, rehabilitation counselors report insufficient training to incorporate EBP into practice. This presents problems, because no matter what the challenges of utilizing EBP may be, EBP plays an important role in providing effective VR services for people with disabilities. Researchers and educators must work together to prepare effective evidence-based practitioners, since rehabilitation practitioners are responsible for providing effective rehabilitation services that lead to successful employment outcomes for their consumers. Evidence-Based Practices and Environmental Factors Environmental factors may inhibit the use of EBP. To improve the effectiveness of rehabilitation counseling services, rehabilitation counselors should be aware of environmental factors that can affect the outcomes of services. However, little is known about which environmental factors have a high probability of predicting successful employment outcomes for people with disabilities. Additionally, characteristics of the organizational setting itself may serve as barriers to the use of EBP (Nelson & Steele, 2007). For example, rehabilitation counselors have reported a higher level of perceived barriers at the agency level than at the individual level (Tansey, Bezyak, Chan. Leahy, & Lui 2014). Lack of support among agency colleagues, little support and encouragement from senior management, and lack of experience 25 among supervisors and counselors trained in EBP were reported as problems of EBP in rehabilitation counseling (Tansey, Bezyak, Chan. Leahy, & Lui 2014; Sherman et al., 2014). Similarly, Winch, Henderson and Creedy (2005) reported that barriers to the use of EBP are often symptoms of organizational contexts that have not been established to integrate evidence into practice. Winch et al. (2005) noted professionals often point to limited motivation and interest, poor confidence, negative attitudes, and limited understanding of the value of research as barriers to EBP. Included among both organizational and individual characteristics are lack of knowledge for properly assimilating EBP on the part of service providers, poor leadership, bureaucratic constraints, and a change-averse culture (Corrigan, Steiner, McCracken, Blaser, & Barr, 2001). Clearly, vocational rehabilitation researchers and scholars have begun to recognize the need to consider environmental factors in the development of effective VR practice. EBP Interventions in Rehabilitation Counseling Practice RCs work with individuals who have disabilities in order to improve the quality of their lives. In recent years, the EBP movement has received increased attention within rehabilitation counseling (Leahy et al., 2014). In the discipline of rehabilitation counseling, the underlying philosophy of EBP is for rehabilitation counselors to use the best available evidence in relation to interventions that lead to the best possible consumer outcomes (Chan et al., 2009). Therefore, rehabilitation counselors need to answer a long-standing question posited by Paul (1967): “What treatment, by whom, is most effective for this individual with that specific problem, and under which set of circumstances?” (p. 111). Answers to this question will provide VR counselors with information regarding which services and interventions have a high probability of predicting successful outcomes for VR customers (Fleming, Del Valle, Muwoong, & Leahy, 2012). 26 Although the general effectiveness of vocational rehabilitation counseling has been empirically demonstrated in the literature (Pruett, Rosenthal, Swett, Lee & Chan, 2008; Leahy & Arokiasamy) there is a serious lack of targeted EBP to accurately define what specific VR services improve employment outcomes of people with disabilities. State-Federal VR agencies must understand which service provision patterns a high probability of have predicting employment outcomes for individuals with disabilities. As Chan, Taryvdas, Blalock, Strauser, & Atkins (2009) stated: “Rehabilitation counseling must embrace an EBP paradigm to remain a vital and respected member of the future community of professional in rehabilitation and mental health care. Included among the many services that are both supported by strong scientific evidence, and provided by counselors in the VR agencies, are counseling, skill training, and supported employment (Chan et al. 2011). Recently, researchers identified two practices: promising organizational management practice and clinical services delivery practice, both of which are used by state VR agencies to improve employment outcomes of people with disabilities (Del Valle et al., 2014; Leahy et al., 2013; Sherman et al., 2014). Del Valle et al. (2014) conducted a multiple-state case study to discover promising VR interventions that help improve employment outcomes of people with disabilities across four states. The researcher sought to discover what the specific best practices are that appear to be evidence-based and transportable to other state VR agencies. The result from the multiple case study research indicated that twenty-nine agency practices, subdivided into Promising Organizational Practice and Promising Service Delivery Practices, were identified as being supportive of successful employment outcomes for persons with disabilities. The practices that were identified from the case studies were valforce, community rehabilitation program 27 certification, career exploration services, soft skill training, benefit counseling, and individual placement. These practices affected service delivery at multiple levels within the agency, community rehabilitation organizations, employers, and specific disability population. The authors indicated that the result of this study appears to be sustainable, replicable, and transportable to other VR agencies. Similarly, Sherman et al (2014) worked to identify promising best practices and organizational factors associated with successful employment outcomes for clients served in the public VR program. The authors found that leadership style and management practice have a significant impact upon the success of a state VR agency. Sherman and her colleagues (2014) also found that the likelihood of engaging in innovative best practices is affected by several organizational and environmental factors, such as culture, leadership, support for innovative and promising practice, staff training, and client-centered services. Fleming, Del Valle, Muwoong, and Leahy (2013) conducted a literature review focused upon identifying empirical studies published in the last 25 years for “active VR Services” at the state VR agency level. Active services was defined as actual state VR agency service delivery practices that lead to competitive employment outcomes for VR consumers. The research team located a total of 561 articles during this search. According to this study, the best practices of the literature were: interagency collaboration, counselor education and consumer outcomes, services to a targeted group, EBP supported employment, empowerment and consumer self concept, essential elements of service delivery, and miscellaneous VR services and outcomes. Rehabilitation counselors use psychological and vocational counseling in daily practice to improve the quality of service for their clients. Psychological and vocational counseling is identified as a major job function and knowledge domain of rehabilitation counselors (Leahy, 28 Chan, Sung, & Kim, 2013). Wampold (2001) concluded that the effect size of absolute efficiency for counseling is relatively large and falls within the range of .75 to .85. In addition, the counseling services, skill training, and supported employment services that are provided by rehabilitation counselors in state VR agencies are supported by strong scientific evidence (Chan et al., 2011). Rehabilitation Counselors’ Perceptions and Insufficient Preparation in EBP Rehabilitation counselors’ perceptions about EBP have been noted in the literature of rehabilitation counseling for many years (Fitzgerald et al., 2016; Pfaller et al.,2016; Tansey et al., 2014; Del Valle, 2015; Grahamn et al., 2013; Bezyak et al., 2010; Yaeda, Iwanaga, Fujikawa, Chan, & Bezyak, 2015). Recently, Fitzgerald et al. (2017) explored certified rehabilitation counselors’ perceptions of the effectiveness of EBP and their preparedness to implement EBP within rehabilitation counseling settings. The majority of participants agreed that the application of EBP is necessary, useful, and will lead to improved quality of client services. Also, they believed EBP to be an effective practice in rehabilitation counseling, and that EBP is important for learning new techniques, services, and practices best suited to individual with disabilities. Moreover, the study’s findings highlighted that some of the participants reported they received EBP in their more recent graduate training. However, the majority of the participating rehabilitation counselors (61%) indicated that they had not received training in EBP in their master’s level curriculum and were not provided adequate on-the-job training in their respective employment settings. In other words, they did not have sufficient skills and knowledge to effectively implement EBP in daily practice. The majority of the participants expressed a strong desire to receive more continuing education in EBP in this study. 29 Pfaller et al (2016) examined the readiness of community-based rehabilitation organization (CBRO) practitioners to implement EBP. Participants were in moderate agreement that they were ready to use EBP in their clinical settings. Participants were in high agreement about the practical value of EBP (M = 7.38) and interested in learning more about EBP (M = 7.11). However, they were in the least agreement as to whether they learned the foundations of EBP in their academic training (M = 4.13). There is a lack of sufficient training to incorporate EBP into practice (M = 3.48) Similarly, Tansey, Bezyak, Chan, Leahy & Lui (2014) have worked to determine the level of understanding VR professionals have of EBP, giving specific attention to perceived self- efficacy, outcomes expectancy, barriers, and readiness to use evidence in current practice. Three hundred 96 rehabilitation counselors participated this study. The result of this study indicates that counselors’ self-efficacy and outcome expectancy were positively associated with readiness to use EBP. Participants reported a relatively moderate confidence in their ability to use EBP in making decisions about their client. This study revealed that participants have a moderate outcome expectancy regarding the benefits of EBP. Perceived barriers were reported at the higher level, at the agency level rather than at the individual level. Beyzak, Kubota & Rosenthal (2010) described and examined certified rehabilitation counselors’ attitudes about EBP knowledge and skills related to obtaining and evaluating evidence. Although rehabilitation counselors may not possess all of the skills necessary to incorporate research into daily practice, they reported generally positive attitudes towards the use of EBP. According to a survey, application of EBP is necessary and will lead to improved quality of consumer services. Rehabilitation counselors typically feel equipped to use generic search engines in professional practice. Despite this skill, many participants reported receiving limited 30 training in academic search strategies or strategies for the implementation of EBP. Approximately 34% of rehabilitation counselors who were in this study reported an inability to apply research findings to provide EBP interventions for their clients. The survey also reported some familiarity with research-related terms, but gaps in knowledge of terminology contribute to problems incorporating evidence into rehabilitation counseling practice. Other barriers to using EBP include insufficient time, lack of generalizability to current practice, and inability to apply research findings to individual clients. The result of their study indicates a need for increased training in skills that will allow rehabilitation counselors to find, interpret, and apply evidence to practice (Beyzak, Kubota & Rosenthal 2010). Grahamn et al. (2013) conducted a study to identify and describe vocational rehabilitation counselors’ perspectives about EBP in the setting of vocational rehabilitation agencies. The researcher asked state-federal vocational rehabilitation counselors what they knew about EBP and how they used the information. The focus of the study was upon how rehabilitation counselors defined EBP. The majority of the participants defined EBP in these terms: “research- based”, “documented evidence”, “proven effective” and, “practice or experience”. The result indicated that they valued research for practice and that they understood how to interpret research literature and apply the information at their job. Many of participants reported that they were willing to try new ideas based on research, but they reported not having time to read the available research. In addition, fewer respondents indicated that they used research findings and EBP in the performance of their duties with clients, such as developing Individual Plans for Employment. Many participants used informal networks of collaboration with other professionals, meeting with clients, and informal conversations at their offices or with client’ families as methods to do their job better and gather information for practice. 31 EBP plays an important role in providing effective rehabilitation services that support individuals with disabilities in their rehabilitation process. However, rehabilitation counselors reported that they did not receive adequate training in EBP in their master’s level curriculum. As the regulatory organizations (e.g., CRCC and CORE) specify, it is rehabilitation counselors’ professional and ethical responsibility to have EBP skills and knowledge. The rehabilitation counseling literature has suggested that EBP knowledge and skill training should be emphasized in the rehabilitation counselors’ preparation programs. EBP Training in Rehabilitation Counseling Education Program As identified above, EBP knowledge and skills are crucial for helping rehabilitation counselors to assist their clients effectively in the rehabilitation counseling practice. Educational needs have also been identified through a review of past literature. The next task is to understand how the EBP trainings are offered in rehabilitation counselor preparation programs. Training students in EBP as a model for clinical decision making (Thomason, 2010) includes a process to help practitioners address issues of treatment selection, ethical practice and application issues with a scientific influence (Gambrill, 1999). However, the literature suggests that practitioners are aware of the importance of treatment effectiveness but feel they lack the training and resources to effectively perform these interventions (Bezyak et al. 2010). As stated before, EBP is increasingly recognized in health disciplines including rehabilitation counseling. However, recognition of the need for EBP is only one step toward it becoming a reality for a profession; this cannot occur unless practitioners are properly trained and know how to practice in this way. Evidence-based practice requires a set of skills and knowledge including the ability to find and appraise evidence and to apply the best evidence at the point of clinical decision making (Dawes et al., 2005). These attributes must be taught in 32 undergraduate and continuing rehabilitation counseling education, acquired by students and practitioners, and hopefully maintained throughout practice life. In order to provide EBP skills and knowledge to rehabilitation counselor education students, EBP must be a core part of rehabilitation counseling curricula. Kosciulek (2010) has explored how rehabilitation counselor educators can aid students and practitioners in learning about and engaging in evidence-based rehabilitation counseling practice. The author stated that disability and rehabilitation service systems that exist today are vastly different from past systems, therefore rehabilitation education needs to transform and move to an evidence-based rehabilitation counseling practice approach. Kosciulek (2010) suggested that rehabilitation educators must both join in and lead the effort to advance an evidence-based rehabilitation counseling practice approach in order to prepare better future practitioners. The author also indicated that a rehabilitation counselor educator must be highly skilled in three areas: (1) helping their students learn how to engage effectively in EBR, (2) developing and using an evidence base for their own practice of teaching, and (3) conducting high quality research that yields useful evidence for guiding rehabilitation counseling practice. In order to meet the demands of today’s diverse practice environments effectively, rehabilitation counselors must be well prepared in the EBP area. EBP is identified as a clinical decision process. The ability to make clinical decisions is a critically important skill and top priority in rehabilitation counselor education (Leahy & Arokiasamy, 2010; Shultz et. al., 2007; Shaw et al., 2006). Given the importance of clinical decision-making skills training within rehabilitation counseling practice (CRCC, 2016), the timing and need for the explicit training of clinical decision-making skills is now. In order to provide the philosophical underpinnings of 33 EBP, rehabilitation counselor educators need to cover clinical decision-making skill training in rehabilitation counselor preparation programs. In order to provide effective clinical decision-making skill training, rehabilitation counseling discipline can consider the EBP model for teaching and/or practice of rehabilitation counseling. This idea is not new and has been advocated for many years (Bellini & Rumrill, 1999; Leung, 1987; Chan et al, 2009; Dellario, 1996; Koch et al., 2009). An EBP model can be infused throughout the master’s program curriculum. A new approach to teaching research skills can be taught in a way that favors its applicability to delivering counseling services (Heppner et al., 1999). Further, master’s students can be informed from the beginning on how to use research to inform practice as an ethical duty (CRCC, 2009) and be taught that using research to inform practice is what rehabilitation counselors are expected to do. Spengler et al. (1995) provide an example of an EBP model that can be infused throughout the master’s rehabilitation counseling curriculum. This model prioritizes counselor clinical judgment and de-biasing techniques to improve judgment accuracy (Spengler et al., 1995), as well as decision-making throughout the counseling process Although implementation of EBP in rehabilitation counseling practice would be a highly challenging process given the uniqueness of rehabilitation counseling field, EBP education is important component of rehabilitation counseling. Rehabilitation counselor educators have a critical role to play on preparing rehabilitation counselors at master’s level who are knowledgeable about the process of EBP and skilled in using available tools to make clinical decision that are supported by the best available evidence (Leahy & Arokiasamy, 2010). From this point, it is time to understand rehabilitation counselor’s perspective about teaching EBP in rehabilitation counseling program. 34 Major Barriers to EBP Training in Rehabilitation Counseling A lack of adequate or appropriate training for behavioral healthcare providers affects quality of care directly when providers make erroneous decisions. There are considerable challenges for the teaching of EBP. Research skills are a key and important element of EBP skills and knowledge. However, Chan et al. (2004) found that rehabilitation counseling master’s students often view research as their least favorite class. As Leahy, Muenzen, Saunders, & Strauser (2009) demonstrated, certified rehabilitation counselors view research as the least important domain related to practice, compared to other knowledge and skill domains. This may be a result of curricula within master’s programs that do not emphasize the importance of research in relation to practice. Another reason could be problems with the integration of research and practice within graduate level coursework. This has been a challenge for educators (Olson,1996). Another barrier to EBP is limited time for continuing education or advanced training in EBP (O’Donnell, 2004). As mentioned before, rehabilitation counselors spend approximately eight hours in direct service. This high volume of face-to-face client contact has been purported to leave providers with limited time for continuing education. Rehabilitation counselors can see the process of locating and appraising research evidence as a too time consuming (Meier, 2003). Moreover, rehabilitation counseling practitioners may not consider research to be relevant or applicable to their needs (Johnston et al., 2009). Clearly to an effective EBP practitioner, knowledge of basic concepts of research methods acquired from traditional courses in master’s level rehabilitation counseling research and statistics is insufficient. Rehabilitation counselor educators can begin to more adequately prepare counselors to effectively implement EBP. As Chan et al (2010) have pointed out, 35 rehabilitation counselor educators must modify their research curriculum to include a strong emphasis on evidence-based methodologies, knowledge translation, and research utilization. Johnston et al. (2009) suggested that more research involving individuals with disabilities and practitioners in the field is needed to increase the relevancy of research in relation to practice. Another way to make the connection between research and practice is to train future rehabilitation counselors to be active researchers, in order to improve their own counseling practice (Kiener & Koch, 2009; Koch et al., 2009). Summary EBP has been identified as an important and effective method for helping rehabilitation counselors improve clients’ quality of life, and it has become popular in the rehabilitation counseling discipline. At this point rehabilitation counseling practice is still characterized as experience-based or habit-based. In fact, rehabilitation counselors may not possess the skills and knowledge necessary to evaluate and incorporate research findings into a practice setting. The lack of EBP training among current rehabilitation counselors could be one of the reasons for the lack of skills. As identified by rehabilitation literature, professional EBP training and continuing education are needed for rehabilitation service providers to provide better VR services. 36 CHAPTER 3 METHODOLOGY Introduction The purpose of this study was to explore rehabilitation counselor educators’(RCE) experience and perceptions in implementation of evidence-based practice (EBP) concepts into rehabilitation counseling education programs (RCEP). Additionally, this study investigated where EBP concepts are addressed, including challenging aspects of teaching EBP, as well as what resources RCE need to appropriately cover EBP concepts in the RCEP. Phenomenographic approach were used for the current study. Phenomenography is the study of how people experience understands or conceive of phenomenon in the world around us (Larsson & Holmstrom, 2007). Phenomenographic approaches were used in this study because it explores study participants’ perceptions and experience in teaching EBP concepts. The current study also employs grounded theory methodology during the data analysis phase to generate a framework explaining what rehabilitation counseling education programs need to integrate EBP concepts. Grounded theory is an approach to analyzing data which is based on inductive reasoning rather than deductive reasoning; therefore, existing theory and conceptual frameworks are not used explicitly to guide the research. A comprehensive review of the literature which details insufficient academic preparation and lack of knowledge in EBP concepts that rehabilitation counseling students experienced was detailed in Chapter 2. While other studies have quantitatively examined the challenges and barriers of teaching EBP in master's level curriculum, the present study gave a voice to RCEs explain the complexities of teaching EBP knowledge and skills. The current study provided a detailed description of the teaching experiences of rehabilitation counselor educators. 37 The research questions of interest in the current study were: (1) what are rehabilitation counselor educators’ perceptions and experience of implementing EBP concepts into curricula? (2) how do rehabilitation counselor educators implement EBP concepts into their curricula? (3) what are the most challenging aspects of implementing EBP concepts in rehabilitation counseling education programs? (4) what resources do rehabilitation counselor educators need to effectively cover EBP concepts in their education program? Research Design A phenomenographic methodology was selected for this study in order to ascertain in- depth and rich experiences of the participants. A basic definition of qualitative research is that it “involves an interpretive, naturalistic approach to the world” (Denzin & Lincoln, 2005, p. 3) where the researcher engages with things and people in their natural settings “attempting to make sense of, or interpret, phenomena in terms of the meanings people bring to them” (Denzin & Lincoln, 2005, p. 3). Researchers who execute qualitative studies generally identify a phenomenon in an open-ended way and “develop hypotheses and theoretical explanations that are based on their interpretations of what they observe” (Creswell, 2007). The phenomenographic branch of qualitative studies describe the representative categories of experience in particular contexts (Lepp & Ringsberg, 2002; Marton, 1981, 1986; Van Manen, 1996) and investigates the qualitatively different ways in which people experience something or think about something. This methodology documents participants' own perspectives and experience, by listening to the voices of participants through their words. In the current study, qualitative data were collected through semi-structured, and open-ended questions. 38 In the current study, grounded theory is employed for analyzing research data since there is no existing theory which can be used to guide analysis, and since this study aims to generate theory suitable to generate a framework for understanding how RCEs think and work in relationship to EBP. Social scientists Barney Glaser and Anselm Strauss (1967) developed and implemented grounded theory in order to assist sociologists in systematically analyzing data in the process of theory development. Grounded theory has received increased attention because this method emphasizes understanding the “voice” of the participant to build a theory about phenomena. Strauss and Corbin (1990) defined grounded theory methodology as theory is “discovered, developed, and provisionally verified through systematic data collection and analysis of data pertaining to that phenomenon” (p. 23). In order to examine RCE’s perception and experiences of teaching EBP concepts in their training programs, grounded theory was used to guide the analysis, and sub-theme of a teaching framework. Grounded theory development requires an in-depth level of data analysis. The researcher must employ a method that generates new knowledge. This new knowledge reflects the researchers’ data analysis and is expressed as theoretical concepts (Charmaz, & Belgrave, 2012). Stringing together concepts is a way of grouping varied sources of the data under one representation. Grounded theory begins with the data and, through processing, a theory emerges. The ultimate goal of a grounded theory approach is to develop substantive theory/model from data. In this study, data had been collected from rehabilitation counselor educators in the U.S. The data described the experiences based on the educators’ perception of teaching EBP in their master's level curriculum. In the current study interviews with rehabilitation counselor educators was analyzed to develop a framework that explains their EBP teaching experiences in RCEP. 39 Sampling. The quality of a study’s sample is a common concern in both qualitative and quantitative research. Most quantitative researches strive for a random sampling of study participants, but phenom graphic and grounded researchers select a sample within a particular social context (Munhall, 2007). A purposeful criteria sampling technique was thus used to recruit participants for current study. Purposeful sampling is described as choosing “particular subjects to include because they are believed to facilitate the expansion of the developing theory” (Bogdan & Biklen, 1998, p. 65). Purposive sampling also facilitates the selection of individuals who offer rich information on a particular topic of interest (Shaw, 1999). In grounded theory research, the selection of research participants is guided by the aim of creating a theory or framework grounded in the data (Breckenridge & Jones, 2009). Valid sampling focuses on the representativeness of the sample when compared to the population of interest (Englander, 2012). Purposeful criterion sampling which “involves searching for cases or individuals who meet a certain criterion” (Palys, 2008, p. 697). was used to ensure that all participants have experienced the phenomenon being studied (Creswell, 2013). Specific criteria for participation in this study was: (1) rehabilitation counselor educators (2) self-report of teaching experience in master level class in rehabilitation counseling education program, (3) currently working in university, and (4) to be in the top 20 graduate schools for rehabilitation counseling record to the US news report national rankings. The population of interest in this study was rehabilitation counselor educators who have been working in accredited rehabilitation counselor education program in the United States. Phenomenographic and grounded research requires the gathering of data from a relatively small number of participants through extensive interviews to identify the essence of their lived experience (Creswell, 2013), and data is collected until categories of experience are saturated. 40 Data saturation occurs when additional participants provide no new data, no new themes, no new coding, and future researchers will be able to replicate the study (Guest, Bunce & Johnson, 2006). The current “saturated” study sample was comprised of 12 rehabilitation counseling faculty participants: four females and eight males from universities around the United States. Participants’ length of experience in rehabilitation counseling education ranged from six to 25 years; their years of teaching experience ranged from six to 20 years. All participants have taught master’s and PhD-level rehabilitation counseling courses in accredited rehabilitation counselor education program. The participants have a wide range of experience in teaching rehabilitation counseling education courses at all levels. Data Collection Procedure and Ethical Consideration This study’s research methods were approved by the Michigan State University Social, Behavioral, and Education Institutional Review Board (SIRB). After receipt of IRB approval, the researcher firstly contacted with top twenty graduate schools of the rehabilitation counseling education program (based on 2017 U.S. News and World Report rankings) in order to disseminate the study recruitment flyer to appropriate participants who fit the study inclusion criteria. The researcher used the 20 institutions as a basis to identity the first participants. However, response rate early in the data collection process from first 20 institutions required adjustments to be made to the widen the research sites. The researcher contacted with 26 graduate schools of rehabilitation counselor education program (based on 2017 U.S. News and World Report rankings). The final sample consisted of 12 participants who are from different institutions. Respond rate for this study was %46.1. Recruitment emails were sent to potential participants in these school. The potential participants that contacted the researcher by telephone or email participated in an initial 41 information telephone meeting. Then an initial informational email was sent to enlist participation by researcher. The initial email covers the purpose and process of the study, and answers any questions the potential participant may have, as result 12 participants reported that they would like to participate in the study. Once a written agreement was obtained by the researcher, interviews were scheduled at a convenient time and location as indicated by the participant. Although data was saturated with 10 participants, the researcher collected data from all 12. The researcher assessed whether potential participants met the study inclusion criteria. If participants met the criteria, the researcher sent a pre-interview packet to those 12 participants via email. The pre-interview packet contained the informed consent document and list of interview questions. Providing the interview questions to the participant ahead of time allows her or him to consider the experience being studied with plenty of time to ponder (Englander, 2012). Once participants were identified, the researcher held interviews in appropriate environments where the participant felt encouraged to share and answer questions asked honestly. Data was collected from semi-structured interviews that were carried out with each participant on a single occasion either in-person or online through Zoom as appropriate; seven semi-structured interviews occurred online via Zoom. The remaining five interviews were completed in person. In-person interviews were completed in a private room to allow for complete privacy during the interview. Informed Consent. Upon meeting the participant for the interview, the researcher verbally explained and provided a copy of the Informed Consent Document (Appendix A) for the participant to keep if desired. The researcher explained the importance of confidentiality in this study and would use pseudonyms names to identify participants through the data collection and 42 analysis process. The researcher assured participants that their involvement in the study was voluntary and they were able to end their involvement and stop the interview at any time. Participants were requested to sign an informed consent form, which served as documented proof of participant approval for their involvement in the study. At the start of each interview, the researcher explained that the participants could refuse to answer any question with which he or she was uncomfortable. After the Informed Consent Document (Appendix A) was provided, each interview began with the researcher asking each participant specific demographic information (Appendix C). The Participants Demographic Information form (Appendix C) form included gender, ethnicity, level of education obtained, year of experience in human services, and year of teaching experience in rehabilitation counseling education. The Participants Demographic Information form was used to create a profile of each of the 12 research participants in order to provide a summary of the study sample. Confidentiality. Participant and institutional identifying information remains confidential and pseudonyms have been allocated to protect all identities. All information will be kept safe under a locked computer file with identifying information separated from all other data as required by the institutional review board. The Interview Guide. The interview guide contains seven broad questions. The questions were written to elicit as much detail as possible about the rehabilitation counselor educator’s perception of teaching evidence-based practice in rehabilitation counseling education program. The questions are broad, open-ended, and allow the participants to answer in a variety of ways. In semi-structured interviews, “the questions can be very open, and the conversation can take many directions before all the areas you want to address are covered” (Fylan, 2005, p. 43 66). As such the interview agenda created “doesn’t determine the order of the conversation and is subject to revision based on the responses of the interviewees” (Zhang & Wildemuth, 2009, p.2). Given the nature of the semi-structured format of the questions, participants were asked to elaborate and expand upon responses through probes that were unique to each interview. Traditionally, interviews have very few follow-up questions and allow the participant to guide the researcher through their experiences with the phenomenon being explored. The researcher ensured that each participant was asked each of the interview question at some point during the interview process. Interviews were semi-structured in order to allow flexibility for follow-up questions and additional issues to be explored by both the interviewer and interviewee (Esterberg, 2002), therefore the researcher asked unique follow-up question in each interview in order to get more idea about participants responses. The length of the interviews ranged from 26.40- 60.45 minutes. The participants interview varied in length depending on the participant, with the average interview time being approximately 35 minutes. All interviews were audio-taped, and the digital audio recordings were sent to a third- party transcription service. The researcher also maintained a journal of handwritten field notes throughout the interview process. The field notes comprised details that occurred to this researcher during the progression of the interview. The field notes also included observations of participant’s nonverbal behavior, key phrases participants use throughout the interview, and the researcher’s own thoughts and emotions during the interviews. Field notes are considered a secondary data storage method and are vital in qualitative research to retain the data gathered (Lofland & Lofland, 1999). 44 Reliability, Validity and Trustworthiness In any research study the reliability and validity or trustworthiness of the research findings is an important concern (Creswell, 2014). In determining the trustworthiness of qualitative studies, researchers must consider the data collection, analysis, and interpretation methods used (Charmaz, 2014). Triangulation of data, member checking, thick description, theoretical sampling, peer debriefing, and auditing were some of the techniques used to ensure trustworthiness of the present project (Charmaz, 2014). Below is a description of steps and strategies that has been taken to ensure reliability and validity throughout the study. These strategies included clarity of instrumentation, member checks, triangulation, data saturation, and debriefing. Clarity of Instrumentation. Data for this research was gathered from in-depth semi- structured interviews with each of the participants. Semi-structured interviews allow for each participant to guide the conversation (Fylan, 2005). Semi-structured interviews also minimize researcher-participant power relations by allowing for the participants to direct the conversation (Mills, Bonner, &Francis, 2006). The Interview Guide (Appendix B) was used to maintain consistency across the interviews. The guide was developed based on the research questions and grounded theory methodology. The style in which the interview questions were worded is based upon grounded theory literature (Moustakas, 1994; Kvale, & Brinkmann, 2009). The intent is that the interview questions be evocative and give the participant much to discuss. For this study, the Interview Guide was vetted by three experts in the field of rehabilitation counselor education in order to eliminate questions that were too abstract or lacking clarification. One expert was an individual with 20 years of experience in evidence-based practice, though master level courses and currently working as a director of a rehabilitation counselor education program. Another 45 expert had several publications on evidence-based practice. The third expert has worked in public vocational rehabilitation counseling practice, taught courses at the college level. Once approved by those experts The Interview Guide (Appendix B) was reviewed to ensure the clarity of the interview questions. Member Checks. Member checking is when interview transcriptions and later the research findings are shared with the participants for verification and validation (Goldblatt, Karnieli-Miller, & Neumann, 2011). Maxwell (2013) states that member checking, or soliciting feedback from participants, is the “single most important way of ruling out the possibility of misinterpretation of the meaning of what they say and the perspective they have on what is going on” (p. 94). In the current research participants were sent a list of the main ideas as interpreted by the researcher, organized by each interview question. Participants were asked to review these documents to detect if the researcher accurately captured the participants' perceptions and identified main ideas the participant thoughts were important. Triangulation. Triangulation is a technique used to increase the validity of qualitative research (Maxwell, 2013). Triangulation refers to the process of comparing results from different sources, or gathered using different methods, to validate finding. Creswell (2014) indicate that triangulation is a common strategy used in qualitative research that reduces the risk of chance associations and of systematic biases due to a specific method, and allows a better assessment of the generality of the explanations that one develops. For triangulation purposes in the current study multiple participants were included and their experiences were compared for similarities and differences. Triangulation refers to three points of triangle and means that at least three data points are located to support all significant findings. As Glesne (2016) explains triangulation involves “the use of multiple methods, sites, and/or respondents” in this study triangulation 46 occurred by locating least three data points among the 12 respondents for all findings. Only two sub-themes in the study are supported with only three data points; most findings occurred among about %75 of participants as the Figure 2 illustrates. The teaching EBP framework (the current study’s major and sub-theme) was based on the accumulation of common experiences, rather than data unique to any one participant.Data Saturation. Data saturation is used in qualitative research as a criterion for discontinuing data collection and/or analysis (Garrett et al., 2012). Data saturation origins lie in grounded theory (Glaser and Strauss 1967). The two types of samples that are valid are statistically significant samples (quantitative research) and data saturated samples (qualitative research). Data Saturation occurs when additional participants provide no new data, no new themes, no new coding, and future researchers will be able to replicate the study (Guest et al., 2006). This method means that a phenomenon has been accurately captured among participants who fulfil the identified purposeful criteria of the sample. This sample was saturated at ten participants however, I included twelve participants in the study to add richness to my data set, and further test my saturation. These two additional participants conformed my findings. Debriefing. Researchers state that “soliciting feedback from others is an extremely useful strategy for identifying validity threats, your own biases and assumptions, and flaws in your logic and methods” (Maxwell, 1996, p. 94). Several individuals, in addition to the researcher, engaged in coding and interpreting the data for the current study. Two peer debriefers participated in the analysis process. Characteristics of both debriefers include experience in rehabilitation counseling field and qualitative study. The peer debriefers assisted in the coding of data and provided feedback on coding categories and data interpretations. Peer debriefers served several roles throughout the data analysis and interpretation stages including coding data, 47 providing feedback on interpretations, and being a sounding board for the researchers’ emerging insights and concerns. Debriefers also provided background information about the study and grounded theory methodology. After becoming familiar with the raw data, debriefers provided feedback on category lists and participated in category coding of all data during the open coding process. In axial coding, the debriefers reviewed selected passages for the main categories and provided feedback on the emerging theory. Finally, debriefers reviewed an outline of the final draft of the framework and provided feedback on the fit between the framework proposed and the data provided by participants. Data Analysis Procedures Data were gathered and analyzed from in-depth semi-structured interviews that focused on the rehabilitation counselor educators’ perception of evidenced-based practice in teaching rehabilitation counseling education programs. In the current study all of the audio recorded interviews were transcribed immediately by a third-party transcription service to allow for initial open coding to inform subsequent interviews. After getting familiar with the data and ensuring the accuracy of the transcriptions, the researcher then began open coding the data. The participants were assigned a number to protect their identities in the data. After the transcriptions were returned to the researcher, the researcher reviewed each transcription document while listening to the recording to ensure the accuracy of the transcription. Each participant interview was read along with interviews memos multiple times. Strauss and Corbin (1998) suggest several analytic strategies (tools) for enabling, “description that embodies well- constructed themes/categories, development of context, and explanations of process or change over time… [in order to] generate new knowledge and deeper understandings because [the analysis] tends to go beyond what everyone knows” (1998, p. 51). In accord with procedures 48 outlined by Strauss and Corbin (1998) and regarding data analysis for grounded theory research, the following coding procedures were implemented in the current study: (1) open/initial coding, (2) axial coding, and (3) selective coding. Strauss and Corbin (1998) state that first step in theory building is conceptualizing. The purpose of open coding is to begin the process of breaking data down into concepts or representations of objects and events. Open coding involves a line-by-line analysis of the data to identify emergent codes that are treated as provisional but help in identifying gaps in the data at an early stage of data collection (Charmaz, 2014). Initial codes ensure that no categories are overlooked and provide guidance for researcher on areas to focus on in the next interviews (Charmaz, 2014). As a part of grounded theory study, the researcher was able to keep the “voice” of participants in this research. Each transcription was also reviewed multiple times, while the researcher continued to take notes about thoughts and reactions. Note taking while listening to transcriptions is one of the most useful and powerful sense-making tools at hand for researchers to use during analysis (Lawrence & Tar, 2013). Open-coding generated unique codes which closely resembled the participants’ words. During open coding, interviews were reviewed and broken down into phrases and sentences that represented the participant's’ overall experience in a brief summary format and main ideas. To ensure that all potentially important data had been captured, multiple codes were created during the first review of the participant interviews. Using computer software MAXQDA researcher re-read each interview multiple times before engaging in line-by-line open coding identifying key phrases and terms that served as emergent codes. Open-coding generated 365 unique codes which closely resembled the participants’ words. Examples of the open codes included “importance of evidence-based 49 practice”, “generation to generation”, “importance of teaching evidence-based practice”, “adult learner”, “integrating to curriculum”, “workshop to professor” and students’ level. The second step was axial coding which involved linking categories identified in open coding into clear subcategories/themes (Charmaz, 2014). Axial coding “specifies the properties and dimensions of a category and reassembles the data you have fractured during initial coding to give coherence to emerging analysis” (Charmaz, 2014, p. 147). Strauss and Corbin (1998) state that the purpose of axial coding is to “begin the process of reassembling data that were fractured during open coding” (p. 124). According to Strauss and Corbin (p. 125), axial coding answers questions such as 'when, where, why, who, how, and with what consequences. This phase of analysis began by grouping category notes into main and subcategories. In the phase of axial coding process, the researcher began with grouping category notes into main and subcategories. The categories, codes, and the supporting data were then transferred to a grid in a second working document. Supporting participants’ statements were also placed in each category and codes in the grid. With each comparison of participant interviews, new codes and categories became apparent; past documents were also reviewed for similar data. Through axial coding the researcher consolidated final categories with sub-categories. During focused coding researcher grouped the 365 initial open codes into 14 categories. Examples of the categories included “EBP training in counselor education”, “places that you focused EBP in your curriculum”, “challenges teaching EBP”, and “what rehabilitation counselor educators need”. In the final step, selective coding involved identifying core categories which identify relationship between categories (Mills et al., 2006). Strauss and Corbin (1998) state that “selective coding is the process of integrating and refining categories” (p. 142). The primary goals of this step of analysis were to develop an overarching theoretical scheme explaining how 50 each of the categories related to each other, and to identify a core category that explained the experiences of participants. In this step of analysis, the main categories were examined for similarities and differences. Passages representing each of the main categories were sorted and reviewed with peer debriefers. Through final phase of coding researcher consolidated these 14 categories into four final major categories with sub-categories. Additionally, the qualitative data analysis computer software MAXQDA version 12.3.2 was utilized to supplement the researcher own analysis. Creswell (2013) supports the use of software to both store and organize data. Each participant interview transcript was uploaded to the software. The researcher then coded specific words, phrases and quotes. MAXQDA was used to organize the data as well as assist the researcher and debriefers to locate text associated with a code or theme, and also helped find passages that were identified with two or more code labels (Creswell, 2013). This grounded analysis process resulted in the emergence of four main themes included (1) personal perceptions about EBP, (2) perceived barriers toward the implementation of EBP, (3) necessary resources for EBP training, and (4) educational strategies for teaching EBP. The conceptual framework of the Implementing Evidence-Based Practice (IEBP) was created based on rehabilitation counselor educators’ perspectives and experience in implantation of EBP and using these four emerging major themes identified from the data. This model describes the emergent framework and helps to explain RCE’s perceptions of teaching EBP. 51 CHAPTER FOUR RESULTS In this chapter findings from the study will be shared. Grounded theory methodology was used in an effort to analyze interview data from 12 RCE working in different universities. The study explored how rehabilitation counselor educators (RCE) integrate evidence-based practice (EBP) concepts in rehabilitation counseling education programs (RCEP). This study also explored and emphasized where EBP concepts were placed in their program, including challenging aspects of teaching and implementing EBP concepts, as well as what resources RCE need to appropriately cover this skill area. Evidence supporting the four main themes were provided. These include (1) personal perceptions about EBP, (2) perceived barriers toward the implementation of EBP, (3) necessary resources for EBP training, and (4) educational strategies for teaching EBP. The conceptual framework of Implementing Evidence-Based Practice (IEBP) concepts in counselor education based on rehabilitation counselor educators’ perspectives was created using these four emerging themes identified from the data. This model describes the emergent framework and helps to explain RCE’s experience and perceptions in implementing EBP concepts in a rehabilitation counselor education program (RCEP) and what resources RCE need to appropriately cover EBP concepts for RCEP needs. The findings reveal rehabilitation counselor educators’ perceptions and experience of teaching evidence-based practice (EBP) knowledge and skills in RCEP. Additionally, this study provides data to explain where EBP concepts are addressed, including challenging aspects of teaching EBP. In order to provide better training on EBP knowledge and skills this framework can be used as a conceptual framework by RCEP. 52 Participants Before detailing the findings, the following section provides demographic and career background information for the 12 study participants. Each of the participants met sampling criteria that included: (1) being a rehabilitation counselor educator, (2) self-reporting of teaching experience in a master level class in rehabilitation counseling education program, and (3) currently working in a university. Each of the 12 participants had graduate-level teaching experience before and/or at the time of data collection in RCEP. They were interviewed regarding teaching experience in EBP concepts, barriers they encountered in the implementation of EBP in RCEP, needed resources for EBP training, and educational strategies for teaching EBP. Four of the participants were female and the other eight participants were male. Each of the twelve participants held a PhD as their highest education qualification. Seven participants identified themselves as White/Caucasian Non-Hispanic. There were also two Asian, two Hispanic, and one African American who joined the current study. Each participant shared their own individual experiences and perceptions. To ensure participant confidentiality, the real names of individuals have been removed from quotes. Participants’ demographic characteristic are shown in the Table-2. 53 Name (pseudonym) Gender Race Credential Highest Level Education Experience in Human Services (Years) Experience in Teaching (Years) Current Teaching Yang Steven William Angela Yolanda Victoria Harold Roger David Fred Laura M M M F F F M M M M F Asian CRC White or Caucasian CRC White or Caucasian CRC White or Caucasian CRC White or Caucasian CRC White or Caucasian CRC Hispanic Hispanic CRC CRC White or Caucasian CRC African American Asian CRC CRC Robert Table 2. Participants Demographic Characteristics White or Caucasian CRC M 20+ 15-20 20+ 20+ 10-15 5-10 15-20 10-15 10-15 20+ 5-10 5-10 Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes 20+ 15-20 20+ 20+ 20+ 5-10 20+ 15-20 15-20 20+ 5-10 5-10 PhD PhD PhD PhD PhD PhD PhD PhD PhD PhD PhD PhD 54 Participants Profiles Yang. Yang has a PhD in Rehabilitation Counseling. He has worked at different institutions as a researcher and professor. He currently serves as the program director for the Rehabilitation Counseling Education program at his institution. He has more than 25 years’ experience in Human Services, and he has been teaching masters and doctoral level classes for more than 20 years. Steven. Steven has a PhD in Rehabilitation Counseling, and he is an Associate Professor at his institution. He currently serves as the program director for his institution’s Rehabilitation Counseling Education program and coordinates the Rehabilitation Counseling Specialization in the Disability Disciplines’ doctoral program. He has been teaching a masters level class for more than 15 years. His research interests include clinical supervision, counselor education, and counselor professional identity development. William. William has a PhD in Rehabilitation Counselor Education. His area of expertise is in counselor education, and he conducts research in the areas of counselor education. He has 30 years’ experience in Human Services and has been teaching graduate level classes since 1997 in rehabilitation counseling education programs. Angela. Angela is chair and associate professor in the Department of Rehabilitation Counseling at her institution. She has been involved in advocacy, education, and employment issues related to individuals with disabilities for over 33 years. She received her PhD in Rehabilitation Counselor Education. Yolanda. Yolanda received her PhD in Rehabilitation Counselor Education. Her area of expertise is evidence-based practice in Rehabilitation Counselor Education. She has been involved in teaching at the university level over 10 years. 55 Victoria. She is an Assistant Professor of Education in the Department of Educational Psychology, Counseling, and Special Education. She received her Ph.D. in a Rehabilitation Counselor Education Program. She has been teaching a graduate level class over 7 years in Rehabilitation Counseling Education. Harold. Harold is an Associate Professor in the Counseling Program. He is knowledgeable about the profession and professional practice of Rehabilitation Counseling and is committed to defining and better-preparing rehabilitation counselors to work effectively with persons with disabilities. He has been teaching graduate level class since 2000. Roger. Roger is an Associate Professor and has a Ph.D. in Rehabilitation Counselor Education. He has been teaching graduate level Rehabilitation Counseling classes since 2005. David. David is a Professor of Rehabilitation Counseling. He is the Coordinator of the Masters in Rehabilitation Counseling Program. He completed his Ph.D. in Rehabilitation Psychology. He has experience in teaching master level classes. His research interests focus on professional issues within Rehabilitation Counseling Education. Fred. Fred earned a doctorate in Rehabilitation Counseling. He is a Certified Rehabilitation Counselor and has been a full Professor since 2004. His areas of expertise are in counselor education, multicultural counseling, and emotional intelligence. He has been teaching graduate level classes science 1995. Laura. Laura is an Associate Professor of Counseling and Rehabilitation Science. She received her Ph.D. in Rehabilitation Psychology. She is teaching graduate-level courses and she also supervises Master's degree students. Robert. Robert is an Assistant Professor in Rehabilitation Counseling. His area of expertise is evidence-based practice in Rehabilitation, Rehabilitation Administration, and 56 Leadership. He received his Ph.D. in Rehabilitation Counseling. His teaching interests include the applications of behavior analysis to rehabilitation and evidence-based practice in rehabilitation. He is teaching graduate level courses. Themes The next section of this chapter provides the rich data that was discovered during this study, as well as results from the various levels of the coding of that data. In order to answer the above listed research questions, it ultimately identifies the grounded theory methodology that was discovered, based on RCEs’ experience and perceptions of teaching EBP knowledge and skill. The four major themes are organized to first present beliefs about importance and barriers, and then to discuss implementation factors, including resources and teaching strategies. The section begins with an exploration of the first major theme, labeled: “Personal Perceptions about EBP”. This theme describes study participants’ perceptions about EBP. Following this is a description of the major challenges of implementation of EBP in RCEP when teaching EBP: “Perceived Barriers toward the Implementation of EBP”. The rehabilitation counselor educators’ needs for teaching EBP are explained under the heading “Necessary Resources for EBP Training”. Finally, the major theme entitled “Educational Strategies for Teaching EBP” is discussed. In discussing each theme, direct quotations are included to assist in illustrating the teaching EBP framework. Following a format similar to Richie et al. (1997) results are discussed using particular terms to indicate the frequency of endorsement. Note that the phrases “the majority of,” “many,” and “most” were used to discuss concepts expressed by at least 10 of the 12 participants. The words “several” and “a number of” show that 7-9 of the participants supported the concept. The word “some” shows that 4-6 of the participants supported the 57 concepts. “A few” was used to indicate concepts expressed by 3 or fewer participants. Additionally, Table 3 provides an outline of the main themes and sub-themes with indications of how many participants endorsed each. Main Themes Number of participants endorsing sub-themes Endorsement of Main Themes (N=12) Personal Perceptions about EBP Positive Attitude towards EBP 12 Perceived Importance of Teaching EBP 12 Integrating EBP throughout the Curriculum 10 Perceived Barriers toward the Implementation of EBP Students’ Academic Development 8 Students’ Attitudes toward Research 10 Educators’ Level of Knowledge 9 Lack of Empirical Support 12 Common Terminology 7 Lack of Time 5 Necessary Resources for EBP Training Professional Development 10 Keeping Updated 7 Creating Connection 9 Reshaped Master Curriculum 5 Regulatory organizations 9 Focusing on PhD Education 5 Educational Strategies for Teaching EBP 10 Case Study Approaches 12 Using Research Methods Class Creating a Connection with Practice 8 Analyzing Empirical Research Article 9 Table 3. An Outline of the Main Themes and Sub-Themes Personal Perceptions about EBP Before focusing on the challenging aspects of teaching EBP knowledge, and what resources the study participants need to appropriately cover EBP concepts in their training programs, it is helpful to have in mind each of the twelve participants’ perceptions about EBP. The major theme of Personal Perceptions about EBP first focuses on discussing in detail each 58 study participant’s perceptions about EBP and their teaching experience. Then, this theme focuses on the places EBP is usually located in RCEP. This theme includes participants’ descriptions of their belief about EBP and experience in implementing EBP concepts. This major theme also describes the places where EBP is usually located in their graduate curriculum. Three sub-themes were identified in this major theme: (1) positive perception about EBP, (2) perceived importance of teaching EBP, and (3) integrating EBP throughout the curriculum. Positive Perceptions about EBP. As the first step of this study, it is important to understand the study participants’ perceptions of EBP. Participants were asked to describe their perception of EBP. This sub-theme emerged from this question. Each of the twelve participants mentioned their positive perception of EBP. It was not surprising to find that each of the twelve participants hold generally positive attitudes toward EBP, have positive perceptions, and that they are integrating EBP concepts into their curriculum. Participants discussed how EBP is important for rehabilitation counseling education. When discussing participants’ perceptions about EBP, Yolanda stated, “It (EBP) is critically important because rehabilitation counselors need to know what works best for whom and under what circumstances.” Steven is another study participant who expressed his positive perceptions about EBP: “EBP is obviously a crucial piece. It is absolutely fundamental to practice of rehabilitation counseling.” Harold was asked to describe his perception of EBP, and he shared a response on its future role for current students. He described: It (EBP) is important, vital, critical to the future of the profession and the professional practice of rehabilitation counseling. My perception is how critical it is to the future. I feel like that is why it is so critical to use it to help 59 inform, to make more informed decisions as practitioners, as students become practitioners. Laura indicated that EBP is not only an essential part of rehabilitation research, but that it is therefore also an important component of practice settings; rehabilitation counseling students must have EBP knowledge and skills to provide the best available services and interventions for their clients after graduation. She explained her perception of EBP thus: EBP is one of the essential components of rehabilitation counseling, because we actually are actively involved with the research activity. We mainly produce a lot of rehabilitation counselors in the field. I want the students...to run their practice based on empirical evidence and be able to track back what is out there, what research outcomes, so they will be able to use it when they are actually incorporating those interventions they are using. A few participants also discussed how generational differences play an important role for an individual’s perspective on EBP, emphasizing that EBP is relatively new concept for the field of rehabilitation counseling. Yang stated: It (perceptions about EBP) depends on the educator’s age. Because in my time (older generation) there was no EBP. But the newer PhDs are more aware of them. If you look at, say, how likely they are trained or are aware of the EBP, it depends on how old they are. The younger generation will probably be more open to it (EBP), more aware of this movement. Each of the twelve participants expressed their positive perceptions about EBP. It appeared that the RCE were open to teaching EBP knowledge and skills in RCEP. The next 60 theme of this study was focused on the participants’ perceptions about teaching EBP concepts in their education programs. Perceived Importance of Teaching EBP. RCEs’ perception about teaching EBP also helps clarify how EBP integrates into the rehabilitation counseling curricula. Teaching EBP with graduate rehabilitation counseling students is especially important, because this is the first step towards preparing them for their professional roles as practitioners. Therefore, during the interviews, it became evident that most study participants thought EBP should be included in RCEP. Each of the twelve participants believed teaching EBP to be an important concept in the master’s level rehabilitation counseling program. The majority of the participants were supportive of teaching EBP. They stated that teaching EBP is one of their major responsibilities in their counseling training program. For example, Victoria stated: "It (teaching EBP) is very important to have counselors in training understand the value of EBP while we do make that shift." The discussion of this sub-category continued with participants detailing why they thought rehabilitation counseling education should introduce EBP concepts, Angela explained: In the field of rehabilitation counseling, what we do is that human connection with other people and facilitating relationships and establishing that human connection. I value that above all in terms of relationships with students and in terms of relationships with people with disabilities in the community. However, beyond social and emotional intelligence in our curriculum, it is really important that we educate our students in EBP so that when they go into the communities to do their jobs, they will hopefully make a real difference in the lives of people with disabilities. 61 Many participants articulated their perceptions of the importance in teaching EBP. Roger described his idea about importance of teaching EBP thus: “Teaching EBP is really important and I try to meet the students where they are and hopefully get them all on the same level of understanding of the importance and the applicability of EBP.” Some study participants mentioned both their perceptions of importance EBP and teaching students to ask “why” in a structured and purposeful manner. Steven stated: Teaching EBP obviously a crucial piece. I try and teach my students this: at any given point you should be able to discuss ‘why you are doing’ and ‘what you are doing’ with your client. That needs to be grounded in something other than, it just seemed like a good idea at the time. We need to make that explicit, and we need to help the students understand that this is what it (EBP) means. Several participants mentioned a relationship between the clinical decision process and EBP in the rehabilitation practice settings. Harold discussed how RCE should teach students knowledge and skills required for EBP, so that upon commencing clinical practice, they can confidently incorporate research evidence into their clinical decision-making. He stated: It is important to the future of the profession and the professional practice of rehabilitation counseling. EBP is a little bit about separating the body of research that is out there and what we know to inform decisions and put our best foot forward knowing that we cannot control ultimately. I feel like that's why it is so critical. To use it (EBP) to help inform, to make more informed decisions as practitioners, as students become practitioners. 62 This sub-theme data showed each of the twelve participants are aware of the importance of teaching EBP in their programs. Integrating EBP throughout the Curriculum. Participants were asked where EBP is usually located in their graduate curriculum. A number of participants discussed the courses in which they have chosen to implement EBP concepts. Most participants emphasized the integration of EBP and how they integrate it into their masters’ level counseling curriculum. Most have focused on EBP throughout the curriculum; they do not have specific class to cover it. Steven has been teaching master’s level classes more than 15 years. He taught a research class for a number of years, and this class is one of the classes in which the RCE covers most EBP concepts. He was one of the study participants who mentioned an infusion model. In describing his teaching pedagogy, he stated that he has been utilizing an infusion model for EBP. He stated: In our program, we utilize an infusion model. I am utilizing this concept in all of my classes. It (EBP) is integrated throughout the curriculum, which is more topically based. So, if we have a psychiatric rehab program or course, then we are working on looking at the evidence-based realm. William also described where EBP is addressed in their program. He similarly mentioned that EBP was integrated throughout their curriculum: “We have a research course, but that is not where we do EBP. We do it (EBP) in all the courses. It is integrated across the curriculum.” Roger stated: "I used EBP for different classes. I have done it for the research class, I did it for our assistive technology class, as well, for the job placement development." 63 Study participants were also asked to provide an example of how they integrate this concept in their classes. Laura explained her teaching strategies for integrating EBP into her specific class: I am actually teaching group counseling, and one of the big assignments is to let them come up with the group counseling manual. And in order to do that, I encourage the students to go out there and look for the current literature, and I am emphasizing outcome measurement, so how they can measure the effectiveness of their intervention. In order to see the effectiveness, what kinds of outcomes they can actually select and use. It is part of the ways I am incorporating EBP into my curriculum. At the time of data collection, all study participants were teaching master’s level classes. Overall, each of the twelve participants have been integrating EBP into RCEP. The participants have been using several teaching styles and pedagogical approaches to integrate EBP knowledge and skills into their curriculum. Perceived Barriers toward the Implementation of EBP While study participants discussed many positive aspects of EBP, this section highlights some of the challenges they encountered. The perceived barriers toward implementation of EBP in RCEP conveyed by study participants established the second major theme of the current study. Participants discussed several common barriers to teaching and implementing EBP in RCEP. Study participants also addressed issues surrounding lack of empirical studies in the field of rehabilitation counseling research. The barriers to utilization of EBPs across the rehabilitation counseling field threatens to further impede future practice and derail accountability. Unfortunately, barriers also exist at the training level for rehabilitation counseling professions. 64 Therefore, it is imperative that counselor educator programs adequately provide students with the knowledge and skills needed to deliver counseling services within an EBP framework. This major theme derived from study data focused on barriers and challenges to the implementation of EBPs across the rehabilitation counseling curriculum. As shown previously, each of the twelve participants claimed that teaching necessary knowledge and skills for EBP is a critical part of their training programs. However, all study participants encountered some barriers and challenges for implementing EBP concepts. In this major theme, there are six sub-themes: (1) students’ academic development, (2) students’ attitudes toward research, (3) educators’ level of knowledge, (4) lack of empirical support, (5) common terminology, and (6) lack of time. Students’ Academic Development. Participants were asked to describe barriers and challenges to implementing and teaching EBP in RCEP. From the interviews with participants, among the first and most common responses to this question was the students’ lack of necessary knowledge and skills to understand EBP concepts. Most of the study participants mentioned that a student’s level of understanding research and research methodology plays a critical role in gaining necessary knowledge and skills for EBP. The study participants were aware of this barrier, and they were working with students to adequately prepare them for the practice setting. In this sub-theme, study participants typically mentioned their students’ level of ability, previous work experience, knowledge of research and methodology, and professional identity as all impacting students’ levels of understanding. Most study participants emphasized that the students’ previous work and research experience were important to understanding EBP concepts during their graduate education. Steven described the professional training and professional identity development process for 65 graduate rehabilitation counseling students. He mentioned that the way new graduate students thought could be a barrier to understanding EBP concepts in RCEP. He stated: The challenge…this comes back to the nature of professional training and professional identity development and the comprehension that graduate students have regarding professional activity. Our students are described as thinking black and white. It is a challenge for us to start to integrate EBP, when developmentally they may not be interested or at the point where they would understand the ‘why’ behind it. The challenge for us to really teach this effectively is to integrate those discussions of EBP into conversations that make sense for early developing counselors. Further illustrating this sub-theme, some participants discussed how previous work experience was necessary to gain EBP knowledge and skills. Angela explained her experience with young students who were just starting their careers in the field of rehabilitation counseling. She mentioned that previous work experience in the field is an important key factor to understanding EBP concepts: Some of the challenges are just that translation and application. Some of the young people coming in, say, this is an EBP, but if they have not worked in the profession in the field they do not have a clue about what their job is going to look like on a day-to-day basis. Several variables that impact students’ levels of understanding were acknowledged. As discussed above, students’ previous work experience impacted their ability to understand EBP concepts. Fred mentioned a relationship between learning experience EBP and students’ level of 66 understanding. He mentioned some potential barriers when he tries to implement EBP concepts into his classes: When they (students) first come in, I do not think they have any idea. My hope would be that by the time they finish the program that they have more appreciation of the overall idea of trying to go to the available research on any particular area that they may be working in to see if there are proven approaches or empirically validated approaches for doing different types of practices. A few participants discussed their experience with the students who are at the beginning of their career path. Robert stated, "Students at that early stage do not even necessarily know what they are observing. There is not the experience level yet." When we talked about students’ experiences, William explained: “When you have a master's student, that is just totally overwhelming. They do not have the ability to put that into the right framework to make all the evidence useful. They do not have that experience.” Additionally, some participants identified challenges students encounter in understanding empirical research article methodology. The study participant indicated that her students do not know how to read empirical literature. Yolanda described her students’ experience reading an empirical research article: “I was teaching the research class, and I had the students read an article that looked like a great article to them but methodologically I went back and just shredded it.” Similarly, Victoria stated, “I do see students who do still struggle with reading research articles.” The study participants all cited students’ inexperience with research concepts as a disadvantage of teaching EBP in their education programs. Most of the discussion regarding the 67 students’ levels of understanding concerned students’ previous work experience, ability to put all the evidence together, and professional identities. Students’ Attitudes toward Research. In addition to students’ previous work, research experience, and their academic development, their attitude toward research is a key factor to understanding potential barriers to implementation of EBP in RCEP. The previous section documented the importance of students’ levels of understanding and work experience in understanding and interpreting EBP concepts. While these experiences were identified as a primary barrier, most of the current study participants mentioned students' attitudes toward research as important in exploring barriers to EBP implementation in RCEP. A number of study participants emphasized that the students’ attitudes toward research were typically negative. Based on the study data, one of the main reasons for these negative attitudes is that students who are in master’s level training typically have expectations that they will be employed as a rehabilitation counselor in a practice setting following graduation. Therefore, most of the rehabilitation counseling students believe they will not become researchers, and they are not going to use research in their practice setting. The majority of participants reported that rehabilitation counselor educators have challenges teaching EBP skills and knowledge to the students who have this belief. Most study participants discussed some of the challenges that arose when the students had negative attitudes toward research. These attitudes were impacted by student’s future expectations to be practitioners rather than researchers. Yang described his experience teaching EBP concepts in his rehabilitation counselor preparation program: The master's level students, most of them will not become a researcher. You spend time teaching all of those traditional ways of thinking about research 68 design and research methods and qualitative and quantitative analysis. They will put up with that, but it is not like they are really interested in that. Roger explained his experience with students’ attitudes toward research in his training program. He mentioned that students who would like to work in a practice setting after graduation were not interested in research classes, and he noted the difficulty of highlighting the value of research to them: Once in a while, you have students who are clear on what they want. Either it is to be a practitioner, or they want to go on to a Ph.D. program. Or sometimes you have those students who are really clear that research is important. In my experience, about 50% of the students that I have seen have the idea of getting into a master’s program that is a practicing degree, and they are not thinking about the role that research plays in a practicing degree like counseling. And sometimes getting that thought correct is one of the challenges. I think the main barrier is just the attitudes toward research, especially when they first start the program. Laura stated, “It is challenging in terms of where the students start and the different attitudes toward research that the students have.” David also mentioned that most of the rehabilitation counseling students would like to go into a practice setting after graduation. He explained, “A lot of students going into rehabilitation counseling programs want to work right in the field. They are socially oriented; they may not get excited about looking at data and research, so trying to get them interested is a challenge.” Similar to David, William mentioned the following: 69 I think another key thing is that we place a lot of importance on our research methods class - and that is a class where I find there is some resistance among the students to taking it because they may say things like, I am going to be a rehabilitation counselor, not a researcher. In addition to the students’ negative attitudes toward research, a few participants highlighted students’ level of anxiety about the research component. Robert stated: “They (students) experience anxiety about the research component and then they are thinking that it is going to be number crunching and learning formulas and all that.” David described his experience teaching EBP knowledge and skills in RCEP. He discussed negative students’ attitudes toward necessary EBP concepts, such as statistics, and also mentioned there are some challenges to getting students’ attention focused on EBP: Sometimes there is a challenge to getting students excited about this (EBP). I mentioned the research methods class; some students try to put it off until the end because they have the perception that it is statistics, and they are not going to understand it, and it is going to be boring. Trying to get the point across that they really need to know this. Some study participants also discussed student’s attitudes toward continuing education for research. Study participants highlighted the importance of in-professional development for current rehabilitation practitioners to improve their skills and knowledge of EBP concepts. A few study participants mentioned that students must to educate themselves in EBP concepts after graduation by going to conferences, reading journals, and participating in research activities. 70 Steven mentioned, “You (current practitioners) have to continually educate yourself, go to conferences, read journals, and maybe participate in your own research.” Similarly, Steven, Harald stated: I have finished my master’s degree so now I am done. Or, I will do my minimum 100 hours on continuing education, and then I'm done. You have to have the mindset that you have to continually be educating and training yourself. As is evident from the experiences shared by the participants of this study, student attitudes toward research classes can prove to be challenging for teaching EBP. For some participants, it appears the teaching challenges resulted from students’ attitudes or students’ expectations after graduation. Educators’ Level of Knowledge. Another significant barrier to the implementation of EBP is evident in the educators' perceived lack of knowledge related to EBP concepts and pedagogy. Many study participants expressed concern about whether educators truly know what EBP is, how to attain knowledge, barriers to mastering knowledge, and ultimately how to convey EBP knowledge to students. In order to implement EBP in rehabilitation counseling curriculum, rehabilitation counselor educators must be adequately trained in the general evidence-based philosophy. However, the majority of the study participants emphasized lack of knowledge and skills in EBP as another barrier toward implementation of EBP to RCEP. This significant barrier to the implementation of EBP derived from study data focused on study participants’ perceived lack of knowledge and skills related to EBP. 71 The majority of the participants shared their concerns about faculty competencies on EBP. Yang discussed current RCEs’ level of knowledge of research methodology, which is an important component for EBP. He stated: Just randomly talk to any rehabilitation counselor educators you run into and say, "Hey, I am doing an experiment. ‘Can you tell me what the effect size for counseling psychotherapy is?’ I would imagine that over 75% cannot tell you that. Then you say, ‘Let me make it easier. Small effect size or medium effect size or large effect size?’ Now we are not demanding much now. We are not asking the number now. We are only asking if it is small, medium, or large. And they would have to guess. That tells you why the master's students are not appropriately trained. In addition to the necessary knowledge of research methodology, a number of study participants suggested that RCE might have different approaches to EBP, and they might define EBP differently, Angela described: Faculty might have different approaches for how they teach EBP and what those practices are. When I think of EBP, it can mean anything. What is the definition or the parameters for EBP? It could be counseling. It could be employment. It could be leadership. It is almost like you have these interdisciplinary approaches that you try to apply in the graduate program to help educate your students so that they can become the best that they can become. A few participants mentioned the importance of educators’ competencies to cover EBP in RCEP. Roger explained faculty competency in teaching EBP as the following: 72 Faculty side I would say, I have seen this with some other colleagues that I have consulted with, but they have the desire to implement EBP into their classes, the challenge starts with the faculty in terms of, do I understand everything clearly? Am I ready to sell the students on the research aspects of this? Faculty competency is important. So, is the faculty somebody who understands research and who understands the importance and applicability of all these concepts? Lack of Empirical Support. Lack of empirical support in rehabilitation counseling research was seen as a barrier to the implementation of EBP in RCEP by many participants. A lack of strong theory-driven research and empirical evidence to inform assessment, planning, and intervention in the rehabilitation process has been a common criticism for rehabilitation research (Chan et al. 2009). In order to fully implement EBP in rehabilitation counseling curriculum, researchers must provide quality evidence with a strong theoretical basis. Each of the twelve study participants pointed out that rehabilitation research is frequently non-experimental or quasi-experimental in nature. At this point, the study participants addressed the challenges of teaching EBP without strong theory-driven research and empirical evidence in rehabilitation research. Yang was one of the study participants who is trying to contribute to the literature on EBP. He mentioned lack of randomized controlled trials literature in the field of rehabilitation counseling: Do we have enough evidence that we can integrate into our practice? We (researchers) are beginning to contribute to the literature on EBP, but we still do not have enough randomized controlled trials. In order to fully implement 73 EBP, we have to have enough randomized controlled trials and supported interventions studies. There is not enough EBP in rehabilitation research. Several participants explained the current situation for rehabilitation literature on EBP. Roger stated that rehabilitation research is frequently non-experimental. He mentioned current evidence in the field of rehabilitation counseling may not be satisfactory due to problems such as weak external validity and difficulty standardizing treatments: Unfortunately, if you look at our research in rehabilitation counseling we have a lot of survey research, and we should be doing more, even some more quasi- experimental stories. Some more intervention stories where we can really get a grasp on them getting EBP. I think we are slowly seeing more of that but sometimes I think the profession is moving faster than our research. We just need to catch up. And in that process of catching up, I think our competence will build more in faculty members. The first step in the use of EBP in rehabilitation counseling practice is to understand the value of it, and most study participants indicated that their students are aware of this value. Victoria described her experience teaching EBP concepts in RCEP programs. She explained that, although students understated the value of EBP, the reality is that they have difficulties finding empirical studies in the field of rehabilitation counseling. Students generally understand the value of it (EBP), they understand why it is important, but then when they go to the literature to search, I think they become confused and sometimes frustrated when they go to the literature. They do not see much in the rehabilitation literature. They struggle a little bit. 74 David has been teaching master’s level classes for fifteen years and shared his concern about the quality of available evidence in rehabilitation research. He stated: The difficulty is just teaching in certain areas where there may not be really good data to work from. One example is I just taught a lecture on intelligence testing and the use of standardized IQ testing. We talked about how for different groups, like for African-Americans, some of these tests there's evidence of bias against African-Americans. Having to rely on alternative forms and looking at the evaluation of IQ. But I would say there is not great data to work with in terms of offering alternatives. Steven mentioned the same point about available experimental research design in the field of rehabilitation counseling. He pointed out that current evidence in the field was conceptual, with no data collected, and was qualitative: We did a literature review on this component of professional practice, and we found that 80% of the articles were conceptual only, with no data collected. 15% of the articles were qualitative in nature and so generalizability is limited, just based on the nature of qualitative research. Then 5% were empirical quantitative pieces that were still not randomized controlled trials, standards of research. That is more of a lower level of evidence as opposed to the RCT standard of evidence. Further discussions with study participants focused on this sub-theme for RCT, highlighting the need for more articles specifically about the intervention outcome. Fred explained, "There needs to be a comment on the quality of the research that is being done in the field and how it fits into the context of the larger question of EBP." 75 From another point of view, some study participants discussed how rehabilitation counseling students would use a general counseling field or related fields for finding experimental studies, such as randomized controlled trials. The study participants indicated that when the students have a problem finding the empirical research articles in the field of rehabilitation counseling research, they have to know how to use other related fields, such as psychology and general counseling. Laura stated, “A lot of times those articles you can find, not in the rehabilitation counseling journal, but in other areas like psychology journals.” Similar to Laura, Yang mentioned students have to be educated to use different fields to find evidence and empirical research: There is not enough EBP in rehabilitation. But that is not totally true, because a lot of the things that we use are not from rehab, but from counseling in general, from psychology in general. What if we find this person has a coping skills problem? Well, in rehabilitation we may not have randomized controlled trial evidence, but in the generic psychology and counseling literature they may. I think the new generation of students has to be educated to not think scientific evidence is only limited to rehabilitation counseling interventions, but any counseling interventions, any psychology interventions that we can use as part of EBP. Victoria also explained that students might find more theory-driven research in the field of general counseling, but most the students are not visiting other fields: They see right away that our literature base is not as full as some other closely related professions. ‘Okay, I have got my treatment problem; I see that in other fields this is an EBP, but I am wondering if that is appropriate to my 76 population or not.’ I think they are kind of struggling with some of the application. In the field of rehabilitation counseling, there is a lack of literature and evidence about and from empirical research pertinent to rehabilitation counseling. Lack of scientific rigor and sophistication is common in the field. The same holds true for the sample in the current study. However, some study participants stated that rehabilitation counseling students have to be educated to not think scientific evidence is only limited to rehabilitation counseling interventions. Common Terminology. In this subcategory, the participants expressed common terminology as barriers for EBP implementation within RCEP. The common terminology described within this sub-category includes both a different term and definition for EBP. Some participants mentioned that they often do not call EBP as such in their classroom setting, even when they were covering it. When one of twelve participants mentioned challenges and barriers to EBP implementation in the curriculum, David stated, “I think sometimes if we do not say that is an EBP, students may not necessarily recognize that it is based on evidence.” When asked to describe what the most challenging aspects of teaching EBP in the RCEP are, Steven stated: We are struggling with terminology and having consistent terminology and being able to even really create a common dialogue around this issue. The differences in terminology and concepts… and when we're talking about certain things, are we all on the same page? There is some work on the back end that we need to do in this area to bring us all to a common understanding of what this means and how it is implemented in practice. 77 Angela also provided data related to common terminology as a challenge for teaching EBP: “We (educators) may not point out to the students, you are learning the evidence-based practice." The study participants continued to describe a general lack of common terminology for EBP. Yolanda stated: Sometimes we (educators) are not using the term "evidence-based practices." They are talking about a particular set of circumstances that maybe is working with people and so from that, we have this theory. Well, people (students) might not always realize that we are talking about evidence-based practices, particularly if we do not call it that as we are talking about different theories and research. We do not actually always call it that. I think it also depends on the professor and the program that they are in. Some professors and programs probably pay more attention to that than others. Lack of Time. The last sub-theme under the major theme of Perceived Barriers to EBP Implementation is lack of time to cover EBP in the rehabilitation counseling curricula. A few study participants described a general lack of time available to include EBP in the classroom setting in light of existing course content and implementation. When discussing the lack of time, participants noted the time necessary to gain EBP knowledge, particularly when balancing the other roles required of their positions. Laura stated: They are more likely to complete their degree program within two years. And then they are really pretty busy meeting those practicum requirements and internship requirements within those given years, so there is no way they can actually have enough research experience. For example, like data collection. 78 They can do part of the data collection but within the given two years, I do not think they actually can see it from the beginning to the end. In light of university and academic requirements, accreditation and licensure guidelines, not surprisingly, rehabilitation counselor educators experience a lack of time in the curriculum to dedicate to empirically validated methods. Necessary Resources for EBP Training In this major theme, the study participants discussed the resources they need to appropriately cover EBP concepts within their training programs. During the interviews, it became evident that most participants thought that EBP should be included in RCEP. Many participants discussed their ideas for effectively covering EBP in rehabilitation counseling education programs. Within this theme, the need for EBP training in rehabilitation counseling includes six different sub-themes that emerged as participants described their teaching experiences. These five sub-themes included: (1) professional development, (2) creating connections, (3) reshaping the master’s curriculum, (4) CACREP standards, (5) focusing on PhD training, and (6) keeping updated. Professional Development. Professional development for practitioners was one of the needs for EBP training the participants described. The majority of participants emphasized that professional development is not only important for current practitioners, but also continuing education is necessary for RCE. As discussed previously, the current research data reveals that educators need training on how to teach EBP, how to integrate EBP into their curriculum, and how to advance their research methodology and skills. Each of the twelve participants in this study described resources necessary to effectively cover EBP in their program. The discussion of 79 this sub-theme began with educators detailing why they thought current RCE should have professional development. Most participants explained that RCEs need professional development not only concerning research methodologies, but also concerning how to teach EBP in the counselor training program. The participants were asked to describe what resources an educator needs to appropriately cover EBP within training programs. Yang discussed professional development for current RCEP. He explained that the field of rehabilitation counseling needs a workshop for current RCEs to improve their knowledge and skills on how to teach EBP: For the people who are already professors, we can start doing workshops for them but with more of a teaching focus. We will have to provide in-service training for people who are already teaching. Say, ‘This is how you teach the EBP.’ And better yet, if they attend your workshop, you have a take-home curriculum that they can use. If you can build on EBP teaching resources, then I think it's more likely that they will teach it, because they can just use the materials that you give them, and then they can easily move some of those materials into different classes. The RCE must be adequately trained in general evidence-based philosophy. Further illustrating the sub-theme of professional development, participants discussed continuing education for rehabilitation educators about EBP. Yolanda, a participant who discussed how continuing education is important for improving EBP skills and knowledge, stated the following: It is important to have continuing education for professors about EBP to help disseminate that information. You look at studies and people present their research in places like this and that makes you stop and think about different 80 things. I think it is also important to continually provide opportunities for educators to learn more about EBP. A few participants stated their interest in being trained on how to teach EBP concepts. David described current educators' needs for covering EBP in their programs. He first mentioned that describing best practice approaches on teaching EBP would be helpful for educators. Then he stated: It would be helpful maybe in Rehabilitation Research Policy and Education to have a special issue devoted to this: pedagogy methods to teach EBP or promote a program. So, I think to have some specific training on how to effectively teach EBP could be helpful. Similar to David, Joe provided data related to the professional development issue. He explained that current educators should have professional development related to how to teach EBP. Joe reported, “We (the field of rehabilitation counseling) will have to provide professional development for people who are already teaching.” Roger stated, “I think if we had a more structured approach on how to teach this and get students to understand and appreciate the use of data and EBP, I think that would be all the more helpful.” The information provided in the participant interviews above shows continuing education for RCE on the EBP is considered necessary to enhance their knowledge and skills in acquiring appropriate research methodologies and how to teach them. Keeping Updated. In this sub-theme, the participants discussed several common themes related to keeping RCEs up with current best practices. The participants mentioned reading current articles, spending time keeping their knowledge current, and perfecting skills in teaching 81 EBP. David discussed a common criticism of educators related to educators' skills in teaching EBP. He stated: Do we need to ask why faculty don't have the skills for how to teach this? I think, like my own doctoral program at…I think we just talked about research all the time, but I am not sure if we talked about it as a way of teaching it or as a specific skill and how to teach students about the use of EBP. I think it is kind of a given in a sense that we would just have that assumption. Williams mentioned, “It's a very rapidly changing field (rehabilitation counseling). You have to stay on top of these things to be current for EBP”. Roger stated, “We (educators) need to keep ourselves current with the literature.” By having knowledge and understanding of EBP concepts, educators need to keep themselves up-to-date with the latest evidence. Fred focused on spending time keeping knowledge current: We (educators) have got to try to stay abreast of what the current research is and sometimes we do not even have the time to keep our knowledge current. I worry about myself as a faculty member. But I wonder sometimes, am I reading enough to keep current on that particular practice? While some study participants stay current in specific areas, they were not able follow current evidence for some topics. Steven mentioned: Which (EBP) requires us to know the literature and the research and the evidence. It is easier to do in some classes than others. When I teach a clinical supervision class... I could go in and just kind of wing it, whereas another area 82 of instruction that I do not have as much experience in, I might need to review some of the literature and see what the newest stuff is. Some participants identified challenges in staying current, including limited time to search for literature and incorporate it into teaching, lack of commitment from other academics, and heavy workloads. Steven stated, “We spend most of our time particularly for teaching”. Creating Connections. Creating connections was categorized as a third sub-theme within this theme. In this sub-theme, the study participants discussed creating a connection between universities and practice, collaboration with different universities to effectively cover EBP in their curriculum, creating a common language for EBP terminology, and learning different pedagogical approaches from other institutions. When different institutions and faculties share their knowledge and experience with EBP with others, the ability to cover EBP can be enhanced in graduate rehabilitation counseling education programs. Angela explained that it is better to know what other institutions and faculties are implementing to cover EBP concepts in their curriculum. She stated: I would love to have a discourse with another faculty from other institutions. What are you doing? How do you define EBP? How does EBP show up in your courses in your curriculum? And then maybe share, ‘Oh, we're doing that,’ to kind of reinforce what we are doing… This is a gap and we need to make sure we cover this. David mentioned creating an online connection with other institutions would beneficial to share how to teach EBP in their curriculum. He stated: I think the more connections (between universities) we can have on this topic (EBP) the better. Like the use of things like Zoom or having specialty groups 83 in the area where people could get together on a regular basis, like they do in the other divisions of NCRE. It would be great to compare notes with another faculty, like how you are teaching this in your program. In order to further bridge the gap between research and practice, rehabilitation educators must improve and promote research application and dissemination. Collaboration between those in practice and academics will promote the use of multiple EBPs in various settings. Harold mentioned a connection between academia and practice: I think it gets into ways to bridge the connection between academia and research and scholarship to what people are doing in the field. It is not only in the programs but when people get out of the programs… How do you keep them interested in going to conferences and reading journals and doing all those things? Similar to Harold, Steven mentioned collaboration between universities, “I think that the collaboration (with other universities) piece would be helpful, primarily to help create dialogue to facilitate common understanding, so we are all on the same page.” Reshaped Master Curriculum. In this sub-theme, study participants focused on rehabilitation counseling curriculum for master’s degree. Most of the participants expressed that their education mostly emphasizes how to become a practitioner rather than a researcher which this attitude affects students’ mindset toward EBP. Fred mentioned RCEP is mostly focusing on practitioner side of the field and he emphasized that the program needs to have more focus on the scholar side: It really is moving our students to become scholar practitioners. They really need to have that mindset…I have looked at the CRC results of our students over more than 20 years. 84 Then you get into the research questions and the assessment questions like, ‘Tell me about what validity is?’ They do less well. The scholar side of the equation. I think they're really there for the practitioner side, but the scholar side of the equation is less appreciated… Some study participants mentioned importance of thesis project to deliver EBP philosophy to rehabilitation counseling students. Study participants emphasized that they cover necessary knowledge and skills for EBP in different classes including research methodology, practice and theory of counseling. In order to get the whole picture of EBP, rehabilitation counseling students need to deliver a master’s thesis. Laura was one of the study participant mentioned master’s thesis in RCEP. She mentioned that thesis should be part of the master curriculum to better understanding of EBP and apply it. The students perceive, and they are actually getting information piece by piece and they do not really get the whole picture. If we ask them to deliver a master’s thesis maybe that is a point where they can actually put those pieces of information together. By looking at data and by analyzing data and by delivering it in a scientific manner, they will get a deeper understanding. Students don't really get the big picture of what EBP actually is. The majority of the study participants mentioned integrating EBP concepts with advanced research and research methodology topics through the master level rehabilitation counseling curriculum. However, current format of training on EBP in the RCEP does not answer effective training for EBP. Most study participants mentioned that changing tradition format of research methods class for the master's curriculum. Study participants mentioned that in the traditional research class, they are more focusing on search design, statistics, and sampling which are not enough knowledge for rehabilitation counseling students to be effective EBP 85 practitioners after their graduation. Also, rehabilitation counselor educators spend so much time teaching all of those traditional ways of thinking about research design and research methods and qualitative and quantitative analysis. As a result, some study participants mentioned that rehabilitation counselor educators must teach research methods class in a different way. To illustrate teaching research methods class in different way, Yang mentioned: We (educators) will still cover research design, sampling populations and things like that, however, we should shorten their (students) path so that they will be very intelligent consumers. But we should actually try to teach research methods by integrated EBP and teach them how to search for EBP. Also, to teach them how to evaluate research that they are thinking about integrating into their practice. Some study participants mentioned importance of training of systematic reviews, scoping reviews, and knowledge translation to teaching EBP. Participants in this study mentioned that those concepts are not in the master curriculum. Angela mentioned “We did not teach meta- analysis as an easy level for the master's. We do not teach enough of those things.” Similarly, Steven mentioned: We teach research methods first and then after that we will go into concepts that are related to EBP like systematic rebuilds, scoping rebuilds, keyword search, databases and then have examples. Then after that we will teach meta-analysis. Teach them how to analyze the quality of the systematic rebuild so that they can also use some judgment” In sum, many study participants emphasized to train master’s level rehabilitation counseling students to be intelligent consumers of research, it is important, though not sufficient, to teach them the basic concepts of research design and statistical methods. They also need to be 86 knowledgeable about EBP methodologies and concepts including research databases, meta- analyses, knowledge translation, and implementation science. Regulatory Organizations. CACREP mission is to accredit graduate programs that provide professional rehabilitation counselor preparation. In order for a program to receive accreditation, its curriculum must be assessed by CACREP. In this theme, the participants expressed how their curriculum has been impacted by CACREP standards. Each of the twelve participants in this study described that they would like to see EBP concepts as a main standard of CACREP. David stated, "It (EBP) should be a key part of the establishment of accreditation standards." Yang also expressed that EBP was not specifically emphasized in the accreditation body; Yang explained: One thing would be, say the CACREP standards, very explicitly say that that is one of the standards that you need to cover. It (EBP) can be embedded in your research design and research method as one line, right? But people would have to agree. It does not mean that people agree EBP is important. It (EBP) will be a major heading in the standard then It would give a different emphasis to it. Then it will have a higher chance. When the researcher asked to what resources rehabilitation counselor educators need to effectively cover EBP in their curricula, Roger stated: To have clear standards on our accreditation, licensure, whatever having clear standards on what is expected of the practice the professionals in the field.... EBP, that would be important, too, because if there are clear standards then we will be obligated to use it in our curriculum When considering the implication of the CACREP standards, Robert described in his interview the field of rehabilitation need clear CACREP standards; He explained 87 The CACREP standards indicate to cover EBP, but exactly what that looks like and what it entails is really open-ended. There need to be more specific practices discovered and utilized to incorporate into the core curriculum, but it seems like there's just such a lack of EBP in rehabilitation counseling that even if an educator was incentivized to cover EBP, there's so few of them in rehab counseling. The information provided in the participant's interviews show clear evidence of the impact of accreditation standards in relation to their curriculum. Overall, the CACREP standards described by participants were not enough for EBP. Focusing on PhD Education. Some participants in the study described the importance of PhD education for EBP knowledge and skills, such importance related to better training for doctoral rehabilitation counseling students. Study participants emphasized implementation of EBP concepts in a PhD level curriculum. This implementation in rehabilitation counseling education programs is important because current doctoral students are future educators. Unfortunately, barriers exist at the PhD level for rehabilitation counseling professions. Therefore, it is imperative that rehabilitation counseling education programs adequately provide PhD students with the knowledge and skills needed to deliver counseling services. Yang mentioned: I am trying to begin to integrate some of this training (EBP training) in our advanced research methods class for doctoral students. And you would be surprised - even the doctoral students are not well trained in systematic reviews, scoping reviews, meta- analysis. Even our doctors are not well trained in those because it's not in the doctoral curriculum. So how are they going to teach that to their students, right? We have to teach the doctoral students better and then the other side would be the curriculum side. We 88 have to give them enough resources in the curriculum, so they can just pick up and start teaching those things. David stated, “I think it would be helpful to have this a standard thing in doctoral programs to teach doctoral students how they will teach this to their students in their programs.” Victoria claimed that “My own doctoral program at the…, We just talked about research all the time, but we did not really talk about it as a way of teaching it” Educational Strategies for Teaching EBP In order to meet the demands of today's rehabilitation counseling practice environments effectively, rehabilitation counselors must be well prepared for EBP. To prepare today's rehabilitation counselors, educators must introduce EBP in their graduate education with variety of educational strategies. Effective educational strategies integrating EBP competence into graduate rehabilitation counseling education programs is essential to ensure that the future practitioner is well-prepared for their practice challenges. Implementation of a variety of teaching and learning strategies can also provide enrichment to students in adapting EBP into their professional practice. Many study participants indicated that using different educational strategies for teaching. Faculty needs to keep giving that message and reinforcing that message. This major theme focuses on educational strategies employed by study participants to influence EBP integration into RCEP. Each of twelve study participants discussed which teaching strategies to use for teaching EBP in their program. Most study participants reported four main teaching methodologies as follows: (a) case study approaches, (b) using research class, (c) creating a connection with practice, and (d) analyzing empirical research articles. 89 Case Study Approaches. Study participants provided pedagogical approaches to teaching EBP implementation for RCEP. In this sub-theme, many participants expressed case study approaches as a main teaching strategy used by them to teach EBP concepts among their students. Case studies were one of the main teaching strategies used by the majority of participants. To improve their students’ EBP skills and knowledge, most study participants brought real problems in the classroom. In the classroom setting, participant indicated that working on real problems allow to rehabilitation counseling students apply EBP skills knowledge including formulating clinical questions, searching best evidence and finding most appropriate interventions for the presenting problems. Yang stated: To make it (EBP) interesting, we will be using a case study approach. We can use, say, this case and then say that the presenting problem of this case is this and how do you formulate the clinical hypothesis of this case, and then how do you formulate questions that are searchable so that you can use different keywords to search them. And then after you retrieve that information, what do you think will be the most appropriate, culturally sensitive interventions for this person? Steven is another participant expressed that he was trying to find best educational strategies for teaching EBP to his students. He indicated that case study approach is one of the effective ways to teach EBP concepts: If I am sitting in a class and we have just gone through a case study and we get to the point where we are making a clinical decision, I may say, "Okay, so what's the intervention?" And let's say the case study has something to do with treating depression. I could say, "What are you going to do?" And the student says, "I do not really know 90 what to do next. What should we do?" And I could say, "Let's go with a cognitive approach," and we would do the exercise. It was evident from participant interviews that case study presented to students reflected the use of latest evidence. Students were encouraged during case study activity to substantiate their work with relevant current evidence. Angela is another participant who use case study approach to teach EBP in her classroom. She stated: We use case studies to teach EBP. We have experiential opportunities for folks in terms of their interactions and how they absorb content and knowledge. For example, in the clinical class, every student is responsible for identifying a case study or bringing a case scenario to class. They are doing their hours out in the community and the agency, but they have to identify a case study and present that and then they discuss that with all their classmates in terms of gaining insight and knowledge and sharing with one another their growth. But then also inquiring about, "Well, how would you have addressed this," to try to expand the learning potential of that particular situation. Students were exposed to the case study and they were encouraged to discuss this case in the light of current evidence by study participants. Victoria stated: During our case conceptualization that every student has to do a presentation of a case study, right after you explain your case conceptualization you also have to come up with what are some areas of treatment goals that you are considering and what is your theoretical background and your evidence base for addressing this issue. So, students have to say, ‘Here is the problem that I would like to address first and here is the evidence that I am going to apply to this particular person. 91 Using Research Methods Class. It is important that counselor educators always strive for the most effective approach to teaching students the knowledge and skills required for EBP so that upon commencing clinical practice they can confidently incorporate research evidence into their clinical decision-making. In order to generate a comprehensive dataset regarding participant teaching strategies, participants were asked to describe how to use research methods class that help to cover EBP concepts. In this sub-theme participants expressed how to use research methods class as a teaching method. The majority of study participants noted using master level research method class cover EBP in their program. William explained that one way he used to cover EBP in his research class was reading research articles. He pointed that reading those articles helps students interpret literature and assess articles’ methodologies. In research class, he mentioned that reading empirical articles are also beneficial for counseling students to show how to analyze the literature. William described how he has used research articles in his research class: In research class, we (Educators) require students to read first source articles a review of EBP....We do teach a research class that focuses on reading EBP in which they have to do first source readings but that is more geared toward how to interpret the literature, how to read the literature and what to do with it rather than, ‘Here is all the EBP.’ Some participants were passionate about having EBP in their research class. Yang reported focusing on master level research methodology class in a different way; he explained: In the traditional research methods class, we are more focusing on research design, statistics, sampling the traditional way of teaching research methods. I think eventually we will have to teach research methods in a different way. We should actually try to teach research methods with integrated EBP and teach them how to search for EBP. To 92 teach them how to evaluate research that they are thinking about integrating into their practice. So just being intelligent consumers of research findings will be more important. Some study participants also described developing critical thinking skills and asking clinical questions in research methods class. Most participants indicated that these skills were helpful for improving their students’ knowledge and skills required for EBP. Angela explained that she has been focusing on this area in her research class: In our research class, …our students learn how to evaluate, research and scholarship with critical thinking. Then when they see research or studies that are out there, they know how to review an article or a publication or study and determine, ‘Is this sound science? Is this something that I might be able to utilize in my day-to-day work?’ I think one of the keys that we make sure we say is that they are familiar with the phrase EBP. In research class, many participants emphasized using effective educational strategies in for teaching EBP that engaged students. Participants perceived challenges when teaching EBP related concepts to students if pedagogical approaches were not innovative and motivating for students. Victoria stated: In the research methods class, we talk about what exactly EBP means. I think you get the point across to students about why they should pay attention to the use of evidence to guide their practice. In research class, we really stress the importance of understanding how to read a research article and how do you understand basic statistics to try to look at what are significant findings, what is the difference between correlation and causality in looking at data, what is a well-designed study versus a study that's not as well designed. The majority of participants mentioned importance of asking clinical question for EBP. They emphasized that this ability is a key part of EBP concepts and the students must to learn 93 how to ask clinical question to find best available evidence. In order to promote asking clinical questions skills, most participants use empirical research articles in their research methodology class. Steven reported: When I taught a research class for a number of years. We would read a research article on something and I would say, ‘what do we take away from this,’ and they would go, ‘This is no good because this does not tell me what to do.’ Their rudimentary understanding of research is that it gives them a step-by-step instruction on what to do and that is not the case. The reality of it is that the research informs their clinical reasoning process. It does not give them a destination; it is part of the trip that they are taking if that makes sense. The research class is obviously one of the classes that the majority of participants would talk about EBP. Participants interviews showed that educational strategies were used in research class and most participants mentioned this class helps to improve students’ overall EBP competence. Creating Connections with Practice. In this sub-theme, the majority of participants mentioned rehabilitation counseling students must to know how to use and apply EBP in the practice. All study participants emphasized differences between educational and work settings. They reported several strategies may be needed to move EBP interventions into practice, and students must learn these strategies during their master level education. Laura explained that students must to know how to deliver EBP in clinical settings. She reported: My focus is more on the research outcome and helping the students understand what that means when it is actually applied to the clinical settings. I can actually help them to see that the theory can connect to the practice and practice can connect to the outcome and outcome can also feed into the research. 94 The current study participant interviews demonstrated students need to learn how to deliver evidence into their professional practice. Yolanda reported rehabilitation counseling students must learn how to integrate EBP into services: In order to provide evidence practice, they need to know how to deliver evidence. Although you can ask, ‘Hey, did you like my approach?’ That is definitely a subjective question. I just try to incorporate how students can utilize and standardize measurement as a part of their intervention. That way they can actually track back. Few participants emphasized creating a connection between theory and measurement to teach EBP knowledge and skills. Robert stated rehabilitation counselor educators need to teach the students how to create the connection between research and practice. Helping students to make the connection in between theory and also measurement. Also, how they can actually deliver the counseling intervention. Because of a lot of curriculum, we focus on research method. It is all about research. Sometimes we focus on the theory and so it is all about the theory. Students actually get the information piece by piece and sometimes they are lost somewhere with how they put all those things together and then make EBP. In order to create connection between practice and education, most participants mentioned having guest speakers for their classes. Some participants stated that to help in their efforts at teaching evidence-based approaches, consultants or external speakers should be invited to share practical expertise in the classroom. Angela mentioned having guest speakers in a classroom is very beneficial for students to demonstrate real work problems. Angela stated, “We try to bring in professionals who are already working in the community and knowledgeable 95 about EBP. We will have them to present how they have been able to incorporate aspects of best practices into their work.” Harold is another participant using guest speaker in his classroom to create connection between education and practice. He mentioned that inviting community members in the classroom helps students to improve their knowledge and skills on EBP. He also emphasized this interaction was seen positively by study participants and promotes them to gain necessary knowledge and skills for EBP. This study participants provided insight into the importance of the connection between practice and educational settings and events that fostered the student's skills. Some participants tried to engage clinical experts in delivering lectures. Roger stated, “The curriculum is so tight that I think we just have to be creative and encourage the students to access all of their resources and come to the conferences and bring guest speakers to the classrooms.” The data obtained regarding creating the connection with practice in rehabilitation counseling program demonstrated that most participants mentioned these connections were beneficial to promote a learning environment for EBP and the students were able to use best available services. Analyzing Empirical Research Articles. Research skills are a key element of EBP skills and knowledge. During the interview process, the majority of participants mentioned negative attitudes toward research. The study participants use empirical research articles to change their students' attitudes toward research and bring current evidence to the classroom setting. The data showed that most participants placed significance on building activities around reading research articles, critiquing research studies, and the conduct of research. Many participants perceived these to be vital skills for students to apply EBP in their future practice. Laura is one of the 96 participant to encourage her students for literature review; “I encourage students to go out there and look for the current literature and encourage them find what is actually current. Like more recent articles using the therapeutic approach to the disability population.” Some participant mentioned explained that they use the research articles to show effectiveness of interventions that they choose their current problem. Fred mentioned “To see the effectiveness, what kinds of outcomes they can actually select and use. It was evident from participant interviews that most participants distributed research articles to students during class to read and appraise the evidence. The study data showed rehabilitation counseling students were encouraged during lectures to substantiate their work with relevant current evidence. Roger mentioned using research articles in his classroom; “I supplement a lot of the chapters with research-based articles from the journals. Students stay current as much as they can.” Some participants were passionate about using different approaches to deliver content other than lectures, such as using problem-based learning and inquiry-based learning to facilitate students’ skills. To improve students’ EBP skills, Steven placed significance on building activities around asking clinical questions regarding research articles. This approach facilitated students’ understanding of the importance of asking clinical questions, finding relevant evidence, appraising the evidence and using evidence to inform practice. Steven mentioned: I had them read it (article) and I said, ‘Okay, what did you see? What did you think?’ Everyone was like, ‘Yeah, this is great. Here is what we take away from this.’ ‘Okay, let's go back now and look at the methodology.’ We started talking they were like, ‘Oh, wait,’ and they reversed their point. They were like, ‘This is a terrible article!’ I said, "But if you skip all of that stuff in the middle, you take away what they conclude. And so, 97 a couple elements of that - there's the teaching the students how to understand research and to be consumers of research, Participant interviews explicitly showed that students were expected to find current empirical literature, critically analyze it, and support their work with appropriate references. David mentioned how to use current research articles to promote his students’ level of EBP skills and knowledge in his classroom. “We really stress the importance of understanding how to read a research article and how do you understand basic statistics to try to look at what are significant findings” Lastly, William emphasized the students need to learn difference between statistical methodologies “what is the difference between correlation and causality in looking at data, what is a well-designed study versus a study that is not as well designed?” As observed in these quotes, the most participants used current research articles to deliver EBP concepts in their classrooms. The study participants did express use of current research articles to teach research skills can facilitate EBP teaching. It was also evident from participant interviews that using research articles in the classroom presented to students reflected the use of latest evidence. Overview of Implementing Evidence-Based Practice Framework This chapter ends with an overview connecting all of the pieces of the puzzle explaining how rehabilitation counselor educators integrate EBP concepts in rehabilitation counseling education programs. The Framework is represented in Figure 2. The framework postulated that participants’ perceptions and implication in EBP concepts are best understood as an interaction between four major variables: knowledge, persuasion, program, and organization. These factors are bound together by the unifying the Framework of Implementing Evidence-Based Practice (IEBP) in counselor education. The Framework IEBP was created based on study participants’ 98 perspectives and experience and using this study four major themes included (a) personal perceptions about EBP, (b) perceived barriers toward the implementation of EBP, (c) necessary resources for EBP training, and (d) educational strategies for teaching EBP. Implementing Evidence-Based Practice framework helps explain RCE’s experience and implementation of teaching EBP and understand what resources RCE need to appropriately cover EBP concepts in RCEP. 99 INDIVIDUAL LEVEL of EBP EBP KNOWLEDGE PERSUASION Pedagogical Knowledge of EBP Disposition towards Learning EBP Disposition toward Teaching Professional Knowledge Figure 2. Implementing Evidence-Based Practice Framework (IEBP) in Counselor Education Professional Development Regulatory Organizations Research Community ORGANIZATION SYSTEM LEVEL PROGRAM Curriculum EBP IMPLEMENTATION in COUNSELOR EDUCATION 100 Narrative Explanation of the Framework The implementation of evidence-based practice in a rehabilitation counseling education program has the potential to improve the students’ general knowledge and skills, so that upon commencing clinical practice, they can confidently incorporate research evidence into their clinical decision-making. Effective implementation depends upon understanding at both the individual and system level in RCEP. There are two main factors included at the individual level of teaching in the evidence-based practice framework: knowledge and persuasion. For the knowledge factor, implementation of EBP in RCEP depends upon the educator’s professional and pedagogical knowledge of EBP. Rehabilitation counselor educators must be adequately trained in general evidence-based philosophy, as well as in at least one to two evidence-based approaches. Pedagogical knowledge of EBP tries to answer the critical question of how EBP should be taught in order to increase students’ engagement and foster students’ learning experience. This study found that educational strategies play an important role in improving students’ overall EBP competence and their EBP knowledge and skills. Therefore, current rehabilitation counselor educators must be aware of these strategies in order to better implement EBP in their curriculum. The second important factor for the individual level of EBP implementation is the persuasion component. This component is an important part of EBP implementation in counselor education. Rehabilitation counselor educators have the responsibility to teach future rehabilitation counselors, and they have an opportunity to promote client outcomes. Therefore, rehabilitation counselor educators must have a positive attitude toward the implementation of EBP in their curriculum. Also, students’ attitudes toward learning EBP concepts are an important component in the implementation of EBP. 101 The next component of Implementing Evidence-Based Practice framework (IEBP) in a rehabilitation counseling education program is at the system level. At this level, there are two main components included: program and organization. It is imperative that rehabilitation counseling education programs adequately provide students with the knowledge and skills needed to deliver counseling services within the EBP framework. Curricular modifications that effectively integrate EBP concepts can begin with assessments of current educational design. Rehabilitation counselor educators must review program content and competency distribution, assess relevant barriers, and design plans for overcoming these barriers to improve themselves and their students as educators. Essentially, EBP learning environments need to be purposefully and intentionally designed curricula. When a program coordinator understands the benefits of EBP, the program, the faculty, and students are more apt to embrace EBP as a value-added component in their curriculum. Lastly, rehabilitation counselor education programs must work with purpose and intention to create rich learning experiences for students. In order to better implement EBP in RCEP, external resources are needed. These resources were identified as “Organization” in the Implementing Evidence-Based Practice framework (IEBP). First, external resources are CACREP and CRCC accreditation. CACREP must explicitly claim that EBP is one of the standards that rehabilitation counselor educators need to cover in their curriculum. Finally, to fully implement EBP in the rehabilitation counseling curriculum, researchers must provide quality evidence with a strong theoretical basis. Rehabilitation counseling research must have more RCT studies and supported interventions. Summary The purpose of this study was to explore rehabilitation counselor educators’ experience and perceptions in the implementation of evidence-based practice concepts in rehabilitation 102 counseling education programs. Additionally, this study investigated where EBP concepts are addressed, including challenging aspects of teaching EBP, as well as what resources RCE need to appropriately cover EBP concepts in the RCEP. Although each of the twelve participants had unique EBP teaching experience, their experience could be described under four major themes. The four themes were (1) positive perceptions about EBP, (2) perceived barriers to EBP implementation, (3) necessary resources for EBP training, and (4) educational strategies for teaching EBP. Participants shared their perceptions about EBP, potential barriers to EBP implementation, and need for EBP training. Results showed that different and innovative teaching approaches are essential to preparing rehabilitation counseling students with the knowledge and skills to access, appraise, and integrate evidence into practice. Study results provided a deeper understanding of processes used by participants. Chapter 5 discusses and describes the implication of the study results relating to teaching EBP for educators, practitioners, and researchers in the field of rehabilitation counseling. 103 Introduction CHAPTER FIVE DISCUSSION The literature has emphasized the importance of teaching evidence-based practice (EBP) concepts in rehabilitation counselor education (Bezyak et al., 2010). Rehabilitation counselor educators (RCE) are in a unique position to train future professionals to follow EBP concepts in their practice settings. However, considerable research has highlighted a lack of knowledge and insufficient academic preparation in EBP concepts as major barriers to the use of current best evidence to guide the clinical decision-making process (Bezyak, Kubota & Rosental, 2010). In order to identify the reason for these major barriers and insufficient academic preparation, researchers need to focus on master’s level rehabilitation counseling education and need to identify the underlying causes of these major barriers. Little research focuses on teaching EBP, and there is a significant dearth of research from the perspective of RCE who are involved in teaching EBP concepts in rehabilitation counseling education programs. The purpose of this study was to explore RCEs’ experience and perceptions in the implementation of EBP concepts in RCEP. Additionally, this study investigated where EBP concepts are addressed within the curriculum, including challenging aspects of implementing and teaching these concepts, as well as what resources RCEs need to appropriately cover this skill area. Given that there is insufficient first-hand information regarding the implementation of EBP concepts in RCEP, additional data is needed to help guide educators on how to teach EBP in their curricula. Therefore, a qualitative study was needed to describe the complexities of this EBP teaching experience in RCEP. 104 There were four research questions that were used as directives during this study. They provided the general structure needed in this grounded theory study and are: (a) What are rehabilitation counselor educators’ perceptions and experience in the implementation of EBP concepts in rehabilitation counseling education programs? (b) How do rehabilitation counselor educators implement EBP concepts into their curricula? (c) What are the most challenging aspects of implementing EBP concepts in rehabilitation counseling education programs? (d) What resources do rehabilitation counselor educators need to effectively cover EBP concepts in their education program? To address these research questions, the researcher used a grounded theory methodology to gather data through semi-structured interviews with 12 participants who had graduate-level teaching experience before and/or at the time of data collections in RCEP. The study participants not only shared their perceptions of the importance of teaching EBP, but also discussed implementation of EBP, barriers to teaching EBP, and resources necessary to increase their students’ EBP knowledge and skills. This chapter discusses four research questions that were mentioned above and the four major themes, chronologically in the same order they were presented in Chapter 4. The themes that will be discussed are as follows: (1) personal perceptions about EBP, (2) perceived barriers to implementation of EBP, (3) necessary resources for EBP training, and (4) educational strategies for teaching EBP. Personal Perceptions about EBP In this part, the first major theme and the first research question are addressed concerning rehabilitation counselor educators’ perceptions and experience in the implementation of EBP concepts in rehabilitation counseling education programs. The EBP movement has had significant influence on the professional practice of rehabilitation counseling (Chan et al., 2016; 105 Leahy, Chan, & Lui, 2014). Considerable research has focused on rehabilitation counselors' (RC) attitudes toward EBP. Not surprisingly, these studies found that rehabilitation counselors generally hold positive attitudes toward the use of EBP to improve their service and counseling outcomes (Fitzgerald, Leahy, Kang, Chan, & Bezyak, 2017; Graham., 2016; Pfaller et al., 2016; Tansey, Bezyak, Chan, Leahy, & Lui, 2014). However, there is no study showing RCEs’ perceptions and experience in the implementation of EBP concepts in RCEP, and thus the current study is unique, because it focuses on their perceptions and experience. The study participants were first asked to describe their perceptions of EBP and then their experience regarding teaching and implementing EBP in RCEP. In relation to the current study results, most participants emphasized the importance of EBP and the need to have EBP concepts in their training program in order to increase their students’ knowledge and skills. The study results indicate that each of the twelve participants in this study held generally positive and favorable attitudes toward EBP. Many study participants mentioned that EBP is critically important for rehabilitation counselors because it is a fundamental piece of rehabilitation practice and improves employment outcomes for their clients. Therefore, the study participants emphasized that rehabilitation counseling students must learn the required knowledge and skills for EBP during their graduate training. The current study is also unique in that, in addition to showing RCE’s perceptions about EBP, participants noted the experience of teaching EBP to graduate students. The majority of participants stated that their training program for rehabilitation counseling students is the first step towards preparing students for their professional role as a practitioner. Most participants indicated that RCEs need to teach their students about the concepts of research design and statistical methods. Many participants were supportive of teaching EBP, and they would like to 106 teach the applicability of EBP in rehabilitation counseling practice. Participants also emphasized that to be an effective evidence-based practitioner in rehabilitation counseling practice, it is important that RCEs educate their students in EBP. Therefore, during the interviews with study participants, it became evident that they all emphasized EBP concepts should be included in graduate RCEP. The current study reached conclusions similar to previous literature (Chan et al., 2016), demonstrating the importance of teaching EBP to rehabilitation counseling students. Moreover, the existing literature also indicates that rehabilitation counseling students need to be knowledgeable about EBP concepts, including research databases, systematic reviews, and knowledge translation (Leahy, Chan, & Lui, 2014). Seven of the twelve participants in the present study reported teaching these concepts to their students. In the current study, participants were asked to describe the places EBP is usually located in their master’s level rehabilitation counseling curriculum. Each of the twelve participants discussed the courses in which they chose to implement EBP concepts. The current study findings showed that participants were not choosing a specific class in which to teach EBP concepts. Participants mentioned integration of EBP into their graduate-level curriculum. For example, many participants emphasized that they were integrating EBP across their curriculum to provide better understanding of EBP to students. In other words, EBP had been taught in all of the classes in the curriculum and internships. Perceived Barriers toward the Implementation of EBP The second major theme and second research question are discussed in this section. Multiple barriers to adopting EBP in rehabilitation counseling have been identified in the literature (Fleming et al., 2011; Johnston et al., 2009; Leahy & Arokiasamy, 2010; Tarvydas et 107 al., 2010; Chan et al., 2010). The literature suggests that rehabilitation counselors are aware of the importance of treatment effectiveness and applying EBP in their practice settings, but they lack the training and resources to effectively perform EBP and perform evidence-based interventions (Chan et al., 2010; Tansey, Bezyak, Chan. Leahy, & Lui, 2014; Bezyak et al., 2010). They also reported insufficient academic training in basic concepts of research, evidenced-based methods and knowledge, and research utilization (Bezyak et al., 2010). In rehabilitation counseling education programs, these barriers must be addressed to facilitate the development and implementation of EBP. The data from the current study indicated that participants perceived that there are several reasons for insufficient graduate training on EBP. Many study participants reported barriers to the implementation of EBP in their respective programs, which corresponds to the literature. Literature suggested that randomized controlled trials (RCT) are the gold standard for evaluating EBP, due to their ability to establish a causal connection between treatment and outcome (Soloman, Cavanaugh, & Draine, 2009; Foster, 2010). The data gathered in the current study is consistent with how previous literature shows a lack of strong theory-driven research in rehabilitation counseling research (Chan et al., 2003). These studies also reported that rehabilitation counseling research must have RCT studies and supported interventions to fully implement EBP in the RCEP. However, a considerable amount of research has focused on the lack of empirical studies in rehabilitation counseling research, and they emphasized that rehabilitation research is frequently non-experimental or quasi-experimental (Bolton., 2004; Chan, Keegan et al., 2009; Chan et al., 2016; Parker & Hansen, 1981). The lack of RCT is seen as a major barrier to successful implementation of EBP in rehabilitation (Chwalisz & Chan, 2008). 108 Although RCTs are useful in medicine, the emphasis on experimental studies may be too restrictive for the behavioral and social sciences. This judgment is based on the fact that randomized controlled trials do not always consider the full complexity of human behavior and clinical condition (Wampold, 2001). The complex nature of vocational rehabilitation makes it impossible to rely solely on experimental research to determine the effectiveness of treatment contributing to successful outcomes. The current study results are consistent with the literature listed above. Many study participants emphasized the lack of empirical studies in rehabilitation counseling research as a major barrier to the implementation of EBP in their programs. The majority of participants reported a lack of strong theory-driven research in rehabilitation counseling research. Many participants also emphasized the importance of empirical studies for effectively implementing EBP in rehabilitation counseling curricula. The majority of participants had also had a negative experience finding RCT in rehabilitation counseling literature. For example, Yang reported that rehabilitation counseling literature does not have enough RCT. Such a limited amount of experimental studies in rehabilitation counseling literature was seen as a barrier to teaching students the knowledge and skills required for EBP, with the result that upon commencing clinical practice, they could not confidently incorporate research evidence into their clinical decision-making process. This finding aligns closely with Bolton’s (2004) study, which documents that rehabilitation research is frequently non-experimental or quasi-experimental. The lack of empirical studies in rehabilitation counseling research is seen as a major barrier to the successful implementation of EBP. 109 Nevertheless, the majority of study participants noted that rehabilitation counselors can use related fields, such as general counseling, mental health, and psychology, to find theory- driven research for their current problems. The study provided data about using other fields’ resources to find stronger EBP literature. Some study participants indicated that when students have a problem finding a good research article, they need to know that other related fields can be used to find good evidence. Findings from the current study suggest that using related fields to find best evidence is a necessary skill for rehabilitation practitioners. For example, Laura stated that rehabilitation counseling literature does not have many empirical studies, but a rehabilitation counselor must know how to use other resources to find best evidence for their practice. Similar to Laura, Robert mentioned that graduate rehabilitation counseling students have to be educated to use different fields in order to find the evidence. This finding confirms what has been reported in the rehabilitation counseling literature regarding using other fields such as psychology or general counseling (Chan et al., 2016). Chan et al. (2016) suggested that the EBP movement may present a window of opportunity for rehabilitation counseling professions to promote and support a systematic agenda for theory-driven rehabilitation research. The use of evidence from theory-driven research could improve employment outcomes of people with disabilities. Rehabilitation researchers, educators, and practitioners must work collaboratively to promote theory-driven rehabilitation research. In order to develop awareness of EBP, rehabilitation educators must be knowledgeable of existing research and the resources that are available to assist rehabilitation educators. Counselor educators are also ethically obligated to train future practitioners to appropriately evaluate and utilize EBP (Gambrill, 2006). To fulfill this ethical obligation, they must be adequately trained in general evidence-based philosophy. However, faculty lack of training and knowledge in EBP 110 was identified by study participants as a barrier to the implementation of EBP in RCEP. In addition, there is no existing literature to show current RCE’s level of EBP knowledge and skills. From this perspective, the question that needs to be answered is: “What are the current RCEs’ level of knowledge concerning EBP methodologies and concepts, including research databases, systematic reviews/meta-analyses, knowledge translation, and implementation science?” The current study provides valuable information for this question. The data gathered in the current study indicates that while study participants noted the importance of EBP for rehabilitation counseling practice, they also emphasized the challenges of limited knowledge about EBP methodologies and concepts. In relation to the current study, participants highlighted the question of whether current rehabilitation counselor educators are adequately trained in the general evidence-based philosophy and methodologies, as well as issues of how to teach EBP in their graduate programs. Most participants discussed current RCEs’ lack of knowledge about EBP methodologies and concepts. For example, one study participant, Yang, reported RCEs are not well trained in systematic reviews, scoping reviews, and randomized control analysis. Another participant discussed RCEs’ skills for teaching EBP. David mentioned that rehabilitation counselor educators might not have the necessary skills for teaching EBP, because they were not specifically trained on how to both teach EBP and teach students how to use it. Other participants mentioned that EBP is relatively new and is inconsistent with the training and clinical orientation of some faculty. The present study findings highlighted that more rehabilitation counseling research needs to be conducted on current RCEs’ level of EBP knowledge and skills. The current study is unique in that, in addition to RCEs’ level of EBP methodologies and concepts, participants noted that using EBP terminology is important when teaching EBP 111 concepts. Study participants indicated that they often do not call it EBP when they are covering these concepts in their classroom. The data gathered in the current study is consistent with previous literature in showing rehabilitation counseling students’ attitudes toward research. Although rehabilitation counselors reported generally positive attitudes toward the use of EBP (Bezyak, 2010), research methodology classes are not popular with rehabilitation counseling students (Chan et al., 2010). As Leahy, Muenzen, Saunders, & Strauser (2009) demonstrated, certified rehabilitation counselors view research as the least important domain related to practice, compared to other knowledge and skill domains. In relation to the current study results, participants highlighted that rehabilitation counseling students typically have the expectation that they will be employed as a rehabilitation counselor in a practice setting following graduation. Therefore, they are not interested in research methodology classes. Additionally, participants noted that rehabilitation counseling students do not think research plays an important role in their practice setting. Nevertheless, based on research evidence, EBP advocates for rehabilitation counselors (RC) to deliver the best possible services to people with disabilities, whenever possible (Chan et al., 2016). High-quality research evidence allows the RCs to identify the best possible services for their clients. Also, use of high-quality research evidence to guide the RCs helps fulfill their ethical obligations to consumers. For these reasons, rehabilitation counseling students must be educated in EPB methodologies and concepts, including research databases, systematic reviews/meta-analyses, knowledge translation, and science implementation to improve employment outcomes and employment quality for people with disabilities. 112 Necessary Resources for EBP Training Given the emphasis on EBP in the 2017 Code of Professional Ethics for Rehabilitation Counselors (CRCC), it has become even more critical for rehabilitation educators to understand EBP and how to implement it in teaching. Nevertheless, the words “evidence-based practice” are not explicitly used in the CRCC (Commission on Rehabilitation Counselor Certification [CRCC], 2017). EBP has also been identified as a professional standard for counselors and counselor educators through accrediting and professional bodies. The use and training of EBP is a part of the Standard of the Council for the Accreditation of Counseling and Related Educational Programs (CACREP): Section II, F.8.a. of the 2016 CACREP standards refers to the use of research for the “identification of evidence-based counseling practice” (CACREP, 2016, p. 12). CACREP also highlights the “understanding and use of resources for research and evidence-based practices applicable to rehabilitation counseling” (CACREP, 2016, p. 22). Lister and Moody (2017) noted that the need for EBP is identified in the CACREP standards, but specific guidelines are lacking on the implementation and training of EBP. They also indicated that, while the CACREP standards refer to EBP, the counseling field is still lacking in guidelines on how EBP can be integrated more fully into counselor training (Lister & Moody, 2017). In the current study, participants were asked to describe resources that are needed to effectively cover EBP in the RCEP. The study findings showed that professional development is necessary for current RCE in order to improve knowledge and teaching skills. For example, many participants emphasized that they would like to attend a workshop to learn more about how to effectively teach EBP, as well as how to integrate EBP across the rehabilitation counseling curricula. Previous literature has revealed useful resources about how to teach EBP in a vocational rehabilitation curriculum. Chan and Tansey (2016) have published an EBP toolkit in 113 the Rehabilitation Research and Training Center on Evidence Based Practices in Vocational Rehabilitation (RRTC-EBP-VR). This toolkit provides specific examples, including well- formulated, answerable questions that guide how to seek the best evidence available, critically evaluate the evidence, and apply the evidence to one’s individual consumer. Furthermore, scholarly websites, useful articles and books, and EBP interventions that are used in vocational rehabilitation are available in this toolkit for counselor educators. This toolkit also provides modules, such as individual placement and support (IPS), to show educators how to integrate IPS methods and programs into services, as IPS is important for a practitioner. Current RCE can use the toolkit to teach and integrate EBP into their graduate programs. The data gathered in the current study confirms previously documented findings relevant to creating connections with practice, research, and education in order to effectively cover EBP in all settings (Leahy et al. 2014). The study results indicate that there needs to be a connection between academia and research. The study participants noted that collaboration with different universities is necessary to effectively cover EBP in their curriculum, because they believe that when different institutions and faculties share their knowledge and experience on EBP with others, the ability to cover EBP will be enhanced in RCT. The data gathered in the current study is consistent with previous literature, which describes widespread insufficient training of basic concepts of rehabilitation research methods (Chan et al., 2016). To be an effective evidence-based practitioner in rehabilitation counseling, the RC must become more knowledgeable about EBP and research utilization, but a current master’s level course is insufficient to get more knowledge about that (Chan et al., 2016). In relation to the current study results, participants emphasized the importance of changing the traditional way of teaching a master’s level course. Rehabilitation counseling curricula must 114 provide comprehensive information about EBP, knowledge translation, research utilization, and how to search for EBP in the literature. Further, the participants in the current study proposed that a research methods class would be more relevant to application. The participants also emphasized that rehabilitation counseling students should be comfortable with searching databases on the Internet, as that is essential for finding best evidence. Rehabilitation practitioners need to find a reliable and scholarly approach to searching for scientific research papers and reviews through academic databases, such as CINAHI., Cochrane Library, Campbell Library, ME PsycINFO. Rehabilitation counseling students need to be familiar with academic databases. Educational Strategies for Teaching EBP Literature suggests that training on EBP significantly improves practitioner confidence, expected benefits, and readiness to adopt EBP into their practice (Kosciulek, 2010; Burker & Kazukauskas, 2010). Therefore, RCEs need to focus on EBP in their curriculum to improve students’ level of knowledge and skills in EBP. Counselor educators use different educational methodologies when they are teaching this concept. Many different educational interventions and strategies have been utilized in RCEP to teach EBP (Chan et al., 2016). Literature shows that educational strategies adopted were found to improve students’ overall EBP competence and their EBP knowledge and skills. Students felt more confident they were accurately interpreting the literature, could better assess the reliability/validity of information on the web, and felt more comfortable with EBP concepts (Kyriakoulis et al., 2016). The results of this study are consistent with previous literature in using different educational strategies to teach EBP. Most participants in the current study indicated that EBP is a learnable skill and the question is not whether EBP can or should be taught, but how to best 115 teach it. The challenge for RCE is to find innovative ways to engage students in a way that facilitates the development of positive attitudes towards research, so that future RCs actively use EBP in their practice settings. However, there is little robust evidence for guidance in the most effective way to build EBP knowledge and skills in students. From this point of view, the question that needs to be answered is how EBP should be taught in order to increase students’ engagement and foster students’ learning experience. The current study findings provide educational strategies for teaching EBP in the rehabilitation counseling curriculum. Findings from the study also showed that RCEs incorporated several teaching and learning strategies, including lectures, current research articles, inquiry-based learning, group discussions, case study approaches, and guest speakers, or a combination of these. The results of the current study are consistent with reports in the literature regarding effective educational strategies for teaching EBP. Chan et al. (2013) claimed that using case study approaches, inquiry-based learning, and current research articles as pedagogical tools improves students’ overall EBP competence and their EBP knowledge and skills. Specifically, the two most effective educational strategies, as reported by the participants in this study, were case study approaches and current research articles. One of the major barriers to EBP implementation in rehabilitation counseling is the challenge of training practitioners to utilize research and results appropriately in practice (Chan et al., 2008). A necessary component for EBP is the ability to know how to deliver evidence in the practice setting. The participants in this study reported several skills may be needed to move evidence-based interventions into practice, and students must learn these skills during their education. To teach these skills, RCEs use a variety of educational strategies. One study participant, Laura, reported focusing on the research outcomes and helping students understand 116 what that means when actually applied in clinical settings. Such barriers to EBP implementation in practice have been reported by other study participants. One study participant claimed that to provide evidence practice, rehabilitation counseling students need to know how to deliver evidence. Angela uses guest speakers in her classes to create a connection between practice and education. It is important that RCEs always strive for the most effective approach to teaching students EBP concepts, so that upon commencing clinical practice, they can confidently incorporate research evidence into their clinical decision-making (Kosciulek, 2010). These study findings showed that the research methodology class is one of the classes in which RCEs would talk about EBP. Participant interviews showed that effective educational strategies, including lectures, group discussion, using web-based research, and inquiry-based learning, were used in research methodology classes, and current study participants stated these strategies helped improve students’ overall EBP competencies. As previously mentioned, study participants highlighted challenges, such as students’ negative attitudes with EBP and research. RCEs need to create positive perceptions of research and EBP. To create this perspective, teaching must incorporate creative and interactive strategies that motivate students and have relevance to their clinical practice (Geum Oh et al., 2010). Based on the results from the current study, study participants use different educational strategies to create a positive environment and motivate students for research and EBP in their training program. As a result, a multifaceted approach may be best suited when teaching EBP to rehabilitation counseling students. 117 Strengths and Limitations of the Study A strength of the current study is that the participants are rehabilitation counselor educators themselves who are speaking and reporting on their own teaching experiences and perceptions about EBP within their training programs. Previous literature has examined rehabilitation counselors’ attitudes toward EBP. However, there is no study available that has focused on rehabilitation counselor educators’ perceptions of EBP, potential reasons for insufficient academic preparation for EBP from a rehabilitation counselor educator’s perspective, and barriers and challenges to the implementation of EBP concepts in RCEP. Another strength of this study has been its ability to obtain valuable information, including educational strategies for teaching EBP, barriers involved in teaching EBP, and necessary components for implementing EBP training. As a result, this study provides insights into the mindset of the rehabilitation counselor educators who are participating in teaching EBP, which is imperative for promoting EBP at other training sites. An additional strength of this study is that the twelve participants who were teaching at the time of data collection were from different institutions. Further, the study’s data collection was analyzed until no new data appeared, and all concepts outlined in the TEBP framework were well developed (Strauss & Corbin, 1998). The researcher adapted the number of participants to reflect the needs of the study after seeing more data was necessary. All participants in the study were asked the same questions. During interviews, the researcher noticed participants discussing similar concepts to those others in previous interviews had discussed during the data collection process. When analyzing the data, it became apparent that the data was saturated, because no new concepts and themes emerged (Strauss & Corbin, 1998). 118 There are limitations inherent to every research study. This study has several limitations that need to be noted when considering the results. First, the data for this study was gathered using a qualitative methodology; therefore, the results described are not generalizable to all RCEs and are only limited to the twelve participants. A second limitation is that the participants shared their subjective experiences as they saw and experienced them. This subjective experience may or may not be consistent with the objective realities of what is taking place within various institutions. Lastly, participants may have had very strong feelings in support of, or against, EBP that may have influenced their responses. Nevertheless, this in some ways may also be seen as a strength, because the participants’ previous associations with EBP allow them to express clearer and stronger opinions based on their experiences. The data obtained in this study is limited based on the small number of research participants. This limitation affects data collection and the grounded theory framework. The limited number of participants coupled with the lack of population representativeness preclude generalization of findings to all rehabilitation counselor educators and institutions. The amount of data is also limited because the participants were involved in only one interview. Additional data may have been obtained from other sources such as reviewing syllabi, observations, and curriculum objectives. This further triangulation could have provided more grounding with other sources for the theoretical framework. The lack of triangulation will be addressed in future studies conducted by the researcher. Data was collected from semi-structured interviews that were carried out with each participant on a single occasion either in-person or in an online meeting, as appropriate. Online interviews can be seen as a limitation of this study. The use of the online interview method may affect rapport and non-verbal cues by creating limitations. 119 Researcher bias effects presented another potential limitation. In the current study, the researcher conducted all of the interviews with participants. Further, the researcher was the primary person involved in protocol development and guided the interpretation process. Thus, there is the possibility that researcher biases influenced the study result. However, several strategies were implemented to reduce researcher bias. Two peer debriefers assisted in the coding and interpretation phases, and a peer auditor reviewed study information to audit the study methods and product. Implications of the Study The study findings have implications for a wide variety of stakeholders, including rehabilitation counselor educators, program coordinators, rehabilitation counseling education programs, rehabilitation practitioners, and researchers. As such, the results of this study will be discussed in the context of counselor education, practitioners, research and theory, and future research. Implications for the Rehabilitation Counseling Education Program Rehabilitation counselors gain necessary knowledge and skills for EBP concepts in their graduate education. Therefore, rehabilitation counselor educators are ethically obligated to train future practitioners to appropriately evaluate and utilize EBP concepts. Teaching EBP concepts is an ethical obligation for RCEs, and they have an ethical responsibility to provide the necessary mindset, skills, and training to prepare future counselors to use evidence-based interventions in their practice settings. In order to provide this skills area to their students, RCEs must have sufficient knowledge to meet these needs. Therefore, they must involve themselves in clinical issues and approximate clinical, educational, and research activities in the process of teaching EBP to students. Nevertheless, the results of this study indicate that the level of knowledge that 120 rehabilitation counselor educators have in EBP concepts is limited. The lack of knowledge described by the participants in the current study includes a general lack of knowledge in EBP, in how to deliver EBP to their students, and in relation to staying current, all of which affects their student outcomes negatively. Some study participants reported a lack of knowledge in how to teach EBP concepts and methodologies to rehabilitation counseling students. As a result, current rehabilitation counselor educators may need to attend workshops and in-service training to improve their knowledge and skills in EBP. The study results showed that rehabilitation counseling students have negative attitudes towards research, which is an important component of EBP. At this point, it is important to support students so that they can overcome these attitudes toward research and EBP. The current study offers teaching strategies for incorporating EBP into counselor education courses. In this way, rehabilitation counselor educators may incorporate these educational strategies that motivate students and create positive perceptions of research and EBP. When RCEs teach students about EBP, the information will be helpful to students in their research and clinical experiences and will help them be successful in their eventual careers. RCEs might remind students that EBP enables rehabilitation counselors to offer choices, as well as information about the advantages and disadvantages of each choice. Rehabilitation counselors provide a large portion of vocational and psychological services to a diverse client population that presents unique problems in different settings. It is important that rehabilitation counselor educators always strive for the most effective approach to teaching students the knowledge and skills required for EBP, so that upon commencing clinical practice they can confidently incorporate research evidence into their clinical decision-making. 121 Utilizing concepts of EBP, the current study highlights the benefits of teaching EBP within learning environments, as well as challenges in teaching EBP to rehabilitation counseling curriculum. The current study findings showed that RCEs should utilize effective educational strategies for teaching EBP to provide quality and beneficial training to rehabilitation counseling students. The results of the study demonstrated that counselor educators should consider an intentionally designed hands-on curriculum, including those educational strategies discussed in Chapter 4. The data showed that educators should also provide training that prepares rehabilitation counselors to be aware of the impact of a lack of EBP skills, in order to incorporate the issues into practice. The inclusion of EBP skills and knowledge in RCEP will fully equip rehabilitation counselors to work more effectively with their clients with disabilities, by ensuring that their clients receive quality education, employment, and other related services in an inclusive environment. Based on the results from the study, it is important to cover EBP concepts in counselor education. However, the study participants reported a lack of knowledge among counselor educators regarding EBP and research utilization. These barriers must be addressed to facilitate the development and implementation of EBP in RCEP. To that end, several study participants mentioned that workshop and professional development for current RCE was important and helpful for improving their skills and knowledge about EBP and research. For example, the majority of participants emphasized that they needed training on how to teach EBP, how to integrate EBP into their curriculum, and on advanced research methodology. The field of rehabilitation counseling education needs workshops and continuing education for the current RCEs to improve their knowledge and skills in EBP. These workshops should provide examples of the curricula that integrate EBP into current RCEs. 122 RCEs must provide the most effective psychosocial and vocational interventions possible by integrating scientific evidence with clinical expertise (Chan et al., 2016). When educating future rehabilitation practitioners about EBP, it is important that future RCEs are aware of the most effective psychosocial and vocational interventions. They also need to be educated in how to use current EBP and interventions in their work with individuals with disabilities. As previously mentioned, the lack of knowledge and academic preparation is often seen as a major barrier to employing EBP (Grahman et al., 2006; Tansey, Bezyak, Chan, Leahy, & Lui, 2014). In order to overcome this barrier, the development of a strong EBP pedagogy is essential for enhancing students’ knowledge about EBP methodologies and concepts, including research databases, meta-analyses, knowledge translation, and implementation science for all students. In designing EBP education and training curricula, first and foremost, educators should be aware of their own knowledge about EBP and increase their personal knowledge, so that they can incorporate EBP concepts into coursework. It is also recommended that educators use EBP concepts and methodologies as course objectives and set clear expectations and guidelines about EBP. In recognition of the positive features of EBP within learning environments, educators are encouraged to design teaching and learning activities, using a case study approach and using current empirical research articles that require students to engage with real problems. Educators should provide students with adequate guidance and assistance in the development of the required EBP knowledge and skills. This means that educators must have sufficient knowledge and skills in EBP before taking on the role of being EBP educators. Rehabilitation counselor educators must be highly efficient and effective EBP learners and possess the skill to engage with and utilize EBP to develop their own and their students’ EBP knowledge and skills. They must be self-reflective and critically aware 123 of how their own knowledge influences what they do, including the way they design the curriculum and select learning activities, and how they respond to students. Rehabilitation counselor educators should also ensure that EBP concepts are infused throughout the entire education training program. Essentially, teaching EBP needs to be purposefully and intentionally designed, not just at the educator level, but also at the institutional and program levels. When the program coordinator understands the benefits of EBP training for their program, faculty, and students, they are more apt to embrace EBP as a value-added component. Embracing an EBP philosophy ensures that programs work with purpose and intention to create rich and diverse learning experiences for students. Chan et al (2016) highlight the need for long-term curricular and organizational planning that addresses the unique needs of institutions with a growing demand for EBP. They also explain that programs have both an opportunity and obligation to capitalize on EBP. Finally, the results from the study and TEBP framework have the potential to assist in strengthening existing curricula for rehabilitation counseling programs. Educators and/or program coordinators may integrate this framework to provide comprehensive training on EBP for their students. Implication for Practitioners The use of EBP enables rehabilitation counselors to fulfill their ethical obligation to their clients by better protecting consumers from harm (non-maleficence), improving efficiency in the utilization of scarce resources (justice), and allowing consumers to exercise knowledgeable self- determination and informed choice (autonomy) (Chan, & Tarvydas, 2009). Due to the current emphasis in EBP on enhancing the effectiveness of rehabilitation services and ethical considerations, RCEs have devoted energy to understanding how EBP can be helpful to their 124 clients and how to best use EBP in their practice settings. Although it is important to consider the theoretical applications, the result of this study indicated that students have negative attitudes toward learning EBP. To deal with these negative attitudes, after graduation RCEs should work closely with institutions to develop better attitudes toward EBP. RCEs also need to join continuing education activities, such as conferences and workshops, to stay current with EBP. Corrigan et al. (2001) suggested that pre-service or professional development makes EBP more accessible to practitioners. RCEs need to attend continuing education activities to improve their EBP knowledge and skills. Additionally, RCEs must develop a strong focus on theory development, empirical evidence, and clinical application. It is vital that rehabilitation counselors understand the importance of EBP in their clinical setting when they are working with people with disabilities. To be an effective practitioner, rehabilitation counselors not only need to become knowledgeable about EBP and research, and but also need to have positive attitudes toward these concepts. Implications for Research and Theory Theoretically, the findings show that the teaching of EBP in rehabilitation counseling education programs needs to be reconsidered. Teaching EBP concepts in this study was mostly dependent upon the persuasion of rehabilitation counselor educators and students. In fact, most of the participants indicated that they pursued training of their students in the concepts of EBP. Participants emphasized the importance of teaching EBP concepts in RCEP, but they also mentioned students’ negative attitudes toward EBP and research. Therefore, RCEP needs to think about improving students’ and educators’ attitudes toward EBP concepts. The evidence-based practice framework developed in the current study can be used as a toolkit to develop rehabilitation counseling curricula for teaching EBP. The toolkit can be used 125 to identify rehabilitation counseling curriculum needs when educators teach EBP to their students. Additionally, such a toolkit can be useful for disseminating effective educational strategies for teaching EBP. Additional quantitative and qualitative studies are needed to further develop TEBP frameworks for rehabilitation counseling education. Additional research is also needed to validate the framework through non-participant class observations and document analysis. Observations of participants allow for the discrepancy between what participants share and their actual practice, which may reflect the reliability of the study findings. Finally, this study shows that implementation of EBP depends on both the individual and system level. In order to provide better training on EBP, RCEs must focus on these two levels. As students become more knowledge about EBP, they will use evidenced-based intervention in their practice settings after graduation and when pursuing personal interests. Implications for Future research This study consisted of semi-structured online and in-person interviews with participants to gauge their perceptions of and teaching experiences with EBP. While the data that the researcher gathered in the study sheds light on RCEs’ perceptions and teaching experiences, further research can obtain more data through a mix of quantitative methodologies. An extension of the current study utilizing quantitative methods or mixed methods would provide comprehensive data about RCE’s attitudes toward EBP, potential barriers to teaching it, and educational strategies to cover it. Extending the current study would also expand our understanding of the EBP teaching experiences of RCEs. The use of quantitative methods may also offer information that is not gleaned via qualitative methods, such as the elements listed above. 126 The study participants were based in twelve different institutions throughout the U.S. Replication of this study with a larger and more representative sample would strengthen the findings and provide valuable information on perceptions of RCEs that are located in other institutions. The study participants were selected on a volunteer basis. Future research should attempt to include more participants selected on a random basis. This approach would allow the result to be more applicable to a broader audience of RCE. The identified population for this study were rehabilitation counselor educators who are the only stakeholder within RCEP. They were asked to provide their perceptions and teaching experience of EBP. The self-report of RCEs pertaining to student knowledge of EBP, attitudes, and barriers within their departments is not likely to be representative of other stakeholders and faculty within those departments. As such, other stakeholders within the programs, including program coordinators and students, should be invited to participate in similar future investigations. This will provide better representation of faculty training in EBP, the implementation of EBP in curricula, the barriers, and attitudes associated with faculty. An extension of the current study that examines rehabilitation counseling Master’s and PhD students, program coordinators, and rehabilitation counseling researchers’ perspectives of EBP would facilitate the training experience of EBP within RCEP. It is possible that rehabilitation counseling students and RCEs may have viewed EBP training differently as a function of their unique perspectives. Accordingly, do RCEs or students have a better vantage point to speak to what forms of learning take place in classrooms? For example, the results of this study demonstrate that RCEs use various educational methodologies in their classrooms to effectively cover EBP in their classes. A follow up study that includes interviews with rehabilitation counseling students is necessary to provide additional data about their ideas of 127 educators’ strategies and their learning experiences with them. Further, additional qualitative and quantitative research studies are needed to identify the underlying reasons that contribute to the major barriers for EBP, including lack of student knowledge and insufficient academic preparation in EBP. Further inquiries should also clarify that adherence to EBP is being tracked and measured by CACREP accredited programs. RCEs indicated that EBP is consistent with the goals of their programs, and they are integrating EBP into their curriculum. Are these program curricula or syllabi being written to reflect specific goals and objectives pertaining EBP? The current study’s results also trigger additional topics for research that can continue to define needs of EBP training in the rehabilitation counseling curriculum. One topic for future research is to examine the effectiveness of educational strategies which were identified in the results section. The task of training and preparing ethical, knowledgeable, and skilled rehabilitation counseling practitioners is the primary objective of professional RCEP. Governments and third- party payers have identified EBP as the standard of practice. Rehabilitation counselors need to demonstrate they are using an array of evidence-based interventions in their practice to improve employment outcomes for customers with disabilities (Rubin, Chan & Thomas, 2003). To answer these external and internal pressures, rehabilitation counseling education programs must provide training to future practitioners to appropriately evaluate and utilize EBP. RCEs must have knowledge about concepts of research, methods, knowledge, and utilization. However, there is no study that shows current rehabilitation counselor educators’ level of EBP knowledge. Further inquiries should clarify their level of knowledge in these concepts. 128 Conclusion Given the importance of EBP in rehabilitation counseling practice, teaching EBP concepts with rehabilitation counseling students is especially important because rehabilitation counselors provide a large portion of vocational and psychological services to a diverse client population with unique presenting problems in varied settings. The findings in the present study demonstrate that rehabilitation counselor educators provide EBP training to master’s level rehabilitation counseling students. The training and challenges describe by participants in this study were complex. The study results demonstrated the need for additional professional development for current rehabilitation counselor educators in relation to how to teach EBP. The findings in the present study also demonstrate that RCE face challenges in educational setting when they are teaching EBP. To provide effective services, rehabilitation counseling professionals must become more knowledgeable about EBP methodologies and concepts including research databases, systematic reviews, and knowledge translation. Therefore, rehabilitation counselor educators have an ethical responsibility to encourage the training and implementation of EBP across all counseling settings. They also must include evidence-based concepts in their curricula, regardless of barriers present in their academic programs. The purpose of this study was to explore RCEs’ experience and perceptions in implementation of EBP concepts into RCEP. Additionally, this study investigated where EBP concepts are addressed within the curriculum, including challenging aspects of implementing and teaching these concepts, as well as what resources RCEs need to appropriately cover EBP. Specifically, the findings in the present study stress challenging aspects of implementation of EBP concepts, and what resources RCE need to appropriately cover EBP within their training 129 programs. From the results, participant's responses were coded into four major themes: (1) personal perceptions about EBP, (2) perceived barriers toward the implementation of EBP, (3) necessary resources for EBP training, and (4) educational strategies for teaching EBP. The findings from this study may be used to understand the challenges, as well as needs of rehabilitation counselor educators who teach EBP concepts in their training program in order to improve and develop teaching experience in the master level curriculum. It will be important to continue to examine RCEs’ perceptions of teaching EBP with various of data collection models. Although each interview was unique to the rehabilitation counselor educators in the study, the responses were consistent with and expanded findings of previous research. 130 APPENDICES 131 Appendix A: Informed Consent Document Research Participant Informed Consent Form Study Title: Counselor Educators’ Experience Implementing Evidence-Based Practice Concepts in Rehabilitation Counseling Education Program Researcher and Title: Bilal Urkmez, Doctoral Candidate Department and Institution: Department of Counselor Education, Psychology, and Special Education, Michigan State University You are being asked to participate in a research study. The researcher is a doctoral candidate at Michigan State University. The information in this form is provided to help you decide if you want to participate in the study. This form describes what you will have to do during the study and the risks and benefits of the study. You should feel free to ask the researcher any questions you may have. If you have any questions about this form or do not understand something you should ask the researcher by the email address provided or call (517) 517 7781. Do not participate in the study unless the researcher has answered your questions and you decide that you want to be a part of this study. Purpose of the Research: The researcher is interested in learning about rehabilitation counselor educators’ perception of students’ preparedness and teaching in evidence-based practice. What Will Happen: For this study we will meet once at a mutually agreeable time via online meeting. During this meeting we will talk about your experiences with teaching. The interview will be audio recorded to be transcribed at a later date and I will take written notes throughout our conversation. This study will take no more than a 60 minute commitment on your part. 132 Potential Benefits: I hope that will find our conversation helpful, interesting, and beneficial as it relates to your experiences with teaching. Potentially the results of this study may be helpful for rehabilitation counseling students training. Potential Risks: In every conversation, there are risks: these risks include hurt feelings, feeling let down, and misunderstandings. In the research context of your interview there are additional risks. I may accidentally ask you a question you find upsetting, too personal, or uncomfortable. You never need to answer a question or talk about a topic that you do not want to. Privacy and Confidentiality: The data for this project will be kept confidential. I will use pseudonyms for each participant so that your actual name will never be used. Any information you share with me will be kept confidential to the maximum extent allowable by law. The only persons who will have access to the data gathered in this study are myself and my advisor Dr. Michael Leahy. The results of this study may be published or presented at professional meetings, however your identity and those of all the other research participants will remain anonymous. The interview will be audio recorded with your permission. The audio recordings will be stored in a digital format on my laptop. My laptop will be password protected to protect the files. Participation is Voluntary: Your participation in this study is completely voluntary. You have the right to say “no” at any time. You may choose not to answer a specific question and can stop the interview at any time. Choosing not to participate or withdrawing from this study will not make any difference in your relationship with me or the social skills group. Contact Information: If you have any questions or concerns about this study, please contact me (Bilal Urkmez), Michigan State University, Department of Counselor Education, Psychology, 133 and Special Education. Erickson Hall, East Lansing, MI 48824. 517 5757781 urkmezbi@msu.edu If you have any questions or concerns about your role and rights as a research participant, would like to obtain information or offer input, or would like to register a complaint about this study, you may contact the Michigan State University’s Human Research Protection Program at (517) 355-2180, Fax (517) 432-4503, or email irb@msu.edu. This communication can be kept anonymous. --------------------------- ----------------------- Name Date 134 Appendix B: Interview Guide 1. Please describe your perception about evidence-based practice. 2. Please describe your perception about teaching evidence-based practice in your curriculum. 3. Where is evidence-based practice in your master level curriculum 4. How do you integrate evidence-based practice in your curriculum? 5. What are the most challenging aspects of teaching evidence-based practice in rehabilitation counseling training program? 6. What resources do you need to effectively cover evidence-based practice in your curriculum? 7. Would you like add something? 135 Appendix C: Participant Demographic Information 1. With what race do you identify? o White or Caucasian Non-Hispanic o Hispanic or Latino o Black or African American o American Indian or Alaskan Native o Asian or Pacific Islander o Other: ______________ 2. With what gender do you identify? o Male o Female o Other: ______________ 3. What is your age? __________ 4. What is your education level (highest degree earned)? o Bachelors o Masters o Doctorate o Other: ____________________ 5. What is your current job title? ___________________________ 6. How many years of experience do you have in human services? o 0-5 years o 5-10 years o 10-15 years o 15-20 years o 20+ o Other: __________ 7. How many years of experience do you have in least master level teaching? 136 o 0-5 years o 5-10 years o 10-15 years o 15-20 years o 20+ o Other: __________ 8. What is your current role/position in your university? (For example: program director, _______________________________ 137 REFERENCES 138 REFERENCES Training Center on Disability Statistics and Demographics. Annual Disability Statistics Compendium (2013). Dunham, NH: Rehabilitation Research and Arokiasamy, C. V. (1994). A theory of rehabilitation? Rehabilitation Education, 7, 77-98. Bezyak, J. L., Ditchman, N., Burke, J., & Chan, F. (2013). Communities of practice: A knowledge translation tool for rehabilitation professionals. Rehabilitation Research, Policy, and Education, 27, 89–103. Bezyak, J.L., Kubota, C., & Rosenthal, D. (2010). Evidence-based practice in rehabilitation counseling: Perceptions and Practices. Rehabilitation Education, 24(1&2), 85-96. Bezyak, J. L., Yan, M. C., Kang, H. J., Burke, J., & Chan, F. (2014). Communities of practice to improve employment outcomes: A needs assessment. Journal of Occupational R Rehabilitation, 24, 597–604. doi:10.1007/s10926-013-9494-1. historical, and public policy perspective. American Psychologist, 63, 228-240. Blustein, D.L. (2008). The role of work in psychological health and well-being: A conceptual, Bolton, B. (2004). Counseling and rehabilitation outcomes. In F. Chan, N. L. Berven, & K. R. Thomas (Eds.), Counseling theories and techniques for rehabilitation professionals (pp. 444—465). New York: Springer. theory and methods, Third Edition. Boston: Allyn and Bacon. Bogdan, R.C. & Biklen, S. K. (1998). Qualitative research for education: an introduction to Brannon, R. (2010). EBP Issue: Forward. Rehabilitation Education, 24(3 & 4), 177-178. Breckenridge, J., & Jones, D. (2009). Demystifying theoretical sampling in grounded theory research. The Grounded Theory Review, 8(2), 113-126. Centers for Disease Control and Prevention (2015). Facts about Autism Spectrum Disorders (ASDs).Retrieved Janurary 1 , 2017, from https://www.cdc.gov/media/releases/2015/p0730-us-disability.html Chan, F., Bezyak, J., & Lui, J. (2013). Development and validation of the evidence-based practice-VR survey. Madison, WI: Rehabilitation Research and Training Center on Evidence-Based VR Practice. Chan, F., Bezyak, J., Romero Ramierz, M., Chiu, C., Sung, C., & Fujikawa, M. (2010) Concepts, Challenges, Barriers, and opportunities related to evidence-based practice in rehabilitation counseling. Rehabilitation Education, 24, (3&4), 179-190. 139 Chan, F., Chronister, J., & da Silva Cardoso, E. (2009). An introduction to evidence-based practice approach to psychosocial interventions for people with chronic illness and disability. In F. Chan, E. da Silva Cardoso, & J.A. Chronister (Eds.), Understanding psychosocial adjustment to chronic illness and disability: A handbook for evidence-based practitioners in rehabilitation (pp. 3- 16). New York: Springer Chan, F., Keegan, J., Muller, V., Kaya, C., Flowers, S., & Iwanaga, K. (2016). Evidence-based practice and research in rehabilitation counseling. In I. Marini & M. A. Stebnicki (Eds.), The professional counselor’s desk reference (2nd ed., pp. 605–610). New York, NY: Springer. Chan, F., Leahy, M. J., Saunders, J. L., Tarvydas, V. M., Ferrin, J. M., & Lee, G. (2003). Training needs of certified rehabilitation counselors for contemporary practice. Rehabilitation Counseling Bulletin, 46(2), 82-91. Chan, F., Miller, S. M., Lee, G., Pruett, S. R., & Chou, C. C. (2004). Research. In T. F. Riggar&D. R. Maki (pp. 159-170) Handbook of Rehabilitation Counseling. New York: Springer. Chan, F., Sung, C., Muller, V., Wang, C. C., Fujikawa, M., & Anderson, C. A. (2011). Evidence based practice and research utilization. In D. Maki & V. M. Tarvydas (Eds.), Professional practice of rehabilitation counseling (pp. 391–412). New York, NY: Springer. Chan, F., Tarvydas, V., Blalock, K., Strauser, D., & Atkins, B. J. (2009). Unifying and elevating rehabilitation counseling through model-driven, diversity-sensitive evidence-based practice. Rehabilitation Counseling Bulletin, 52, 114-119. Charmaz, K. (2014). Constructing grounded theory (2nd ed.). Thousand Oaks, CA: Sage Publications Inc. Charmaz, K., & Belgrave, L. (2012). Qualitative interviewing and grounded theory analysis. The Chronister, J.A., Chan, F., Cardoso, E., Lynch, R., & Rosenthal, D.A. (2008). Evidence-based SAGE handbook of interview research: The complexity of the craft, 2, 347-365. practice movement in health care: Implications for rehabilitation. Journal of Rehabilitation, 74(2), 6-15. Chwalisz, K., & Chan, F. (2008). Methodological advances and issues in rehabilitation psychology: Moving forward on the cutting edge. Rehabilitation Psychology, 53, 251 253. Commission on Rehabilitation Counselor Certification. (2016). Code of professional ethics for 140 rehabilitation counselors. Retrieved December, 10th, 2017 from https://www.crccertification.com/filebin/pdf/Final_CRCC_Code_Eff_20170101.pdf Cook DJ, Jaeschke R, Guyatt GH (1992). "Critical appraisal of therapeutic interventions in the intensive care unit: human monoclonal antibody treatment in sepsis. Journal Club of the Hamilton Regional Critical Care Group". Journal Intensive Care Medicine. 7 (6): 275– 82. Corrigan, P. W., Steiner, L., McCracken, S. G., Blaser, B., & Barr, M. (2001). Strategies for disseminating evidence-based practices to staff who treat people with serious mental illness. Psychiatric Services, 52, 1598–1606. Council for Accreditation of Counseling and Related Educational Programs (CACREP). (2009). CACREP 2009 standards. (2016). Retrieved, 01/01/2018 from http://www.cacrep.org/2009standards.html Council on Rehabilitation Education. (2016). Standards for Rehabilitation Counselor Education Programs. Retrieved Creswell, J. W. (2007). Qualitative inquiry and research design: Choosing among five approaches. Thousand Oaks: Sage. approaches (3rd ed.). Los Angeles, CA: SAGE, Inc. Creswell, J.W. (2013). Qualitative inquiry and research design: Choosing among five Creswell, J. W. (2014). Research Design: Qualitative, Quantitative, and Mixed Methods Approaches (4th Edition). Thousand Oaks, CA: SAGE Publications, Inc. Dean, D., Pepper, J. V., Schmidt, R. M., & Stern, S. (2014). State vocational rehabilitation Del Valle, R. D., Leahy, M. J., Sherman, S., Anderson, C. A., Tansey, T. N., & Schoen, B. programs and federal disability insurance: An analysis of Virginia’s vocational rehabilitation program. IZA Journal of Labor Policy, 3(1), 1– 19. (2014). Promising best practices that lead to employment in vocational rehabilitation: Findings from a four-state multiple case study. Journal of Vocational Rehabilitation, 41, 99–113. Denzin, N. K., & Lincoln, Y. S. (2005). Introduction: The discipline and practice of qualitative research. In N. K. Denzin & Y. S. Lincoln (Eds.), The Sage Handbook of Qualitative Research Third Edition (pp. 1-32). Thousand Oaks, CA: Sage Publications. Dunst, C. J., Trivette, C. M., & Cutspec, P. A. (2002). Toward an operational definition of evidence-based practices. Centerscope, 1(1), 1-10. Dutta, A., Gervey, R., Chan, F., Chou, C. C., & Ditchman, N. (2008). Vocational rehabilitation 141 services and employment outcomes for people with disabilities: A United States study. Journal of occupational rehabilitation, 18(4), 326-334. Englander, M., Faculty of Health and Society, & Malmö University. (2012). The interview: Data collection in descriptive phenomenological human scientific research. Journal of Phenomenological Psychology, 43(1), 13-35. doi:10.1163/156916212X632943 Esterberg, K.G. (2002). Qualitative methods in social science research. Boston, MA: McGraw-Hill. Livingstone. Evidence-based medicine: How to practice and teach EBM (2nd ed.). New York, NY: Churchill Fitzgerald, S., Leahy, M. J., Kang, H. J., Chan, F., & Bezyak, J. (2016). Perceived Preparedness to Implement Evidence-Based Practice by Certified Rehabilitation Counselors A Qualitative Content Analysis. Rehabilitation Counseling Bulletin. Fleming, A.R., Fairweather, J.S., & Leahy, M.J. (2013). Quality of life as a potential Fylan, F. (2005). Semi-structured interviewing. In J. Miles & P. Gilbert (Eds.), A handbook of rehabilitation service outcome: the relationship between employment, quality of life, and other life areas. Rehabilitation Counseling Bulletin, 57(1), 9-22. research methods for clinical and health psychology, (pp. 65-77). New York, NY: Oxford University Press. Families in Society: The Journal of Contemporary Human Services, 80, 341. Gambrill, E. (1999). Evidence-based practice: An alternative to authority-based practice. Giorgi, A. (1994). A phenomenological perspective on certain qualitative research methods. Giorgi A. (2009). The descriptive phenomenological method in psychology. Pittsburgh, PA: Glaser, B. G. (2005). The grounded theory perspective: Theoretical coding. Mill Valley, CA: Journal of Phenomenological Psychology, 25(2), 190-220. doi:10.1163/156916294X00034 Duquesne University Press Sociology Press. Glaser, B. G., & Strauss, A. L. (1967). The discovery of grounded theory: Strategies for qualitative research. Piscataway, NJ: Aldine Transaction. qualitative research. Glaser, B. (2009). Anselm L. Strauss, 1967. The discovery of grounded theory: strategies for Glesne, C. (2016). Becoming qualitative researchers: An introduction (5" ed.). University of Vermont: Longman. 142 Goldblatt, H., Karnieli-Miller, O., & Neumann, M. (2011). Sharing qualitative research findings with participants: Study experiences of methodological and ethical dilemmas. Patient Education and Counselling, 82, 389-395. Goodheart, C. D., Kazdin, A. E., & Sternberg. (2006). Evidenced-based psychotherapy: Where practice and research meet. Washington, DC: American Psychological Association. Graham, C., Inge, K., Wehman, P., Murphy, K., Revell, W.G., & West, M. (2013). Moving employment research into practice: Knowledge and application of evidence-based practices by state vocational rehabilitation agency staff. Journal of Vocational Rehabilitation, 39, 75-81. doi: 10.3233/JVR-130643. Qualitative Methods, 3(1). 1-23 Graham, I., Logan, J., Harrison, M., Straus, S., Tetroe, J., Caswell, W., & Robinson, N. (2006). Lost in knowledge translation: Time for a map? Journal of Continuing Education in the Health Professions, 26( 1), 13– 24. Groenewald, T. (2004). A phenomenological research design illustrated. International Journal of Guest, G., Bunce, A., & Johnson, L. (2006). How many interviews are enough? An experiment Hendricks, D. (2010). Employment and adults with autism spectrum disorders: challenges and Holm M. (2000) Our mandate for the new millennium: evidence-based practice. A Journal of strategies for success. Journal of Vocational Rehabilitation, 32(2), 125-134. with data saturation and variability. Field methods, 18(1), 59-82. Occupational Therapy; 54(6), 575-585. Howard, M. O., Allen-Meares, P., & Ruffolo, M. C. (2007). Teaching evidence-based practice: Strategic and pedagogical recommendations for schools of social work. Research on Social Work Practice, 17(5), 561-568. Bryman & R.G. Burgess (Eds.), Qualitative research (3) 143-164. London: Sage Hycner, R.H. (1999). Some guidelines for the phenomenological analysis of interview data. In A. Ingersoll, G.L. (2000). Evidence-based nursing: What it is and what it isn’t. Nursing Outlook, 48, 151-152. Johnston, M. V., Stineman, M., & Velozo, C. A. (1997). Outcome research in medical rehabilitation. Foundations from the past and directions for the future. In M.J. Fuhrer (Ed.). Assessing medical rehabilitation practices. The promise of outcomes research. Baltimore: Paul H. Brookes. Kosciulek, J.F. (2004). Research applications in the longitudinal study of the vocational rehabilitation services program. Rehabilitation Counseling Bulletin, 47, 173. 143 imperative. Rehabilitation Education, 24(3-4), 205-212. research interviewing. Thousand Oaks, CA: Sage Publications. Patelarou, E. (2016). Educational strategies for teaching evidence-based practice to undergraduate health students: systematic review. Journal of educational evaluation for health professions, 13. Kosciulek, J. F. (2010). Evidence-based rehabilitation counseling practice: A pedagogical Kvale, S. & Brinkmann, S. (2009). Interviews (2nd Edition): Learning the craft of qualitative Kyriakoulis, K., Patelarou, A., Laliotis, A., Wan, A. C., Matalliotakis, M., Tsiou, C., & Larsson, J., & Holmström, I. (2007). Phenomenographic or phenomenological analysis: Does it Law, M. (2002). Evidence-based rehabilitation: A guide to practice. Thorofare, NJ: SLACK. Lawrence, J., & Tar, U. (2013). The use of grounded theory technique as a practical tool for matter? Examples from a study on anaesthesiologists’ work. International Journal of Qualitative Studies on Health and Well-being, 2(1), 55-64. qualitative data colletion and analysis. Electronic Journal of Business Research Methods, 11(1), 29-40. Leahy, M.J. & Arokiasamy, C.V. (2010). Prologue: Evidence-based practice research and knowledge translation in rehabilitation counseling. Rehabilitation Education, 24(3&4), 173-176. Leahy, M. J., Chan, F., Lui, J., Rosenthal, D., Tansey, T. N., Wehman, P., . . . Menz, F. E. (2014). An analysis of evidence-based best practices in the public vocational rehabilitation program: Gaps, future directions, and recommended steps to move forward. Journal of Vocational Rehabilitation, 41, 147–163. Leahy, M. J., Chan, F., Sung, C., & Kim, M. (2013). Empirically derived test specifications for the certified rehabilitation counselor examination. Rehabilitation Counseling Bulletin, 56, 199–214. doi:10.1177/0034355212469839 Leahy, M.J., Muenzen, P., Saunders, J.L., & Strauser, D. (2009). Essential knowledge domains underlying effective rehabilitation counseling practice. Rehabilitation Counseling Bulletin, 52(2), 95-106. R.G. Burgess (Eds.), Qualitative Research (Vol. 3). London:Sage. Lofland, J., & Lofland, L.H. (1999). Data logging in observation: Fieldnotes. In A Bryman & Martin, R., West-Evans, K., & Connelly, J. (2010). Vocational rehabilitation: Celebrating 90 years of careers and independence. American Rehabilitation, Special Edition, 15– 18. Mawhood, L., & Howlin, P. (1999). The outcome of supported employment scheme for high functioning adults with autism or Asperger syndrome. Autism, 3(3). 229-254. 144 Maxwell, J. A. (2013). Qualitative Research Design: An Interactive Approach 3rd Edition. Thousand Oaks, CA: Sage Publications, Inc. California: Sage Publications. Maxwell, J. A. (1996). Qualitative Research Design: An Interactive Approach. Thousand Oaks, McCluskey, A., & Cusick, A. (2002). Strategies for introducing evidence-based practice and changing clinician behavior: A manager’s toolbox. Australian Occupational Therapy Journal, 49, 63-70. McMahon, B. T. (2009). One veteran counselor's take on the future of rehabilitation counseling. Rehabilitation Counseling Bulletin, 52(2), 120–123. Mills, J., Bonner, A., & Francis, K. (2006b). Adopting a constructivist approach to grounded theory: Implications for research design. International Journal of Nursing Practice, 12(1), 8-13. Moustakas, C. (1994). Phenomenological research methods. Thousand Oaks: SAGE Publications Inc. Oxford University Press. Nathan, P., & Gorman, J. (Eds.). (1998). A guide to treatments that work. New York, NY: Nelson, T. D. & Steele, R. G. (2007). Predictors of practitioner self-reported use of evidence based practices: Practitioner fraining, clinical setting, and attitudes toward research. Administration and Policy in Mental Health and Mental Health Services Research, 54,319-330. Neulicht, A.T., McQuade, L.J., & Chapman, C.A. (Eds.) (2010). The CRCC Desk Reference on Professional Ethics: A Guide for Rehabilitation Counselors. Athens: GA: Elliot & Fitzpatrick. O’Donnell, C.A. (2004). Attitudes and knowledge of primary care professionals towards evidence-based practice: A postal survey. Journal of Evaluation in Clinical Practice, 10, 197-205. Palys, T. (2008). Purposive sampling. In L. M. Given (Ed.), The Sage Encyclopedia of Qualitative Research Methods (Vol. 2) (pp. 698-698). Los Angeles, CA: Sage. Paul, L., (1967). Strategy of outcome research in psychotherapy. Journal of Counseling Psychology, (31), 109-118. Pfaller, J. S., Tu, W. M., Morrison, B., Chan, F., Owens, L., Anderson, C. A., ... & Menz, F. E. (2015). Social-Cognitive Predictors of Readiness to Use Evidence-Based Practice A Survey of Community-Based Rehabilitation Practitioners. Rehabilitation Counseling 145 Bulletin, 0034355215591779. Pruett, S.R., Rosenthal, D.A., Swett, E. A., Lee, G.K., & Chan, F. (2008). Empirical evidence supporting the effectiveness of vocational rehabilitation. Journal of Rehabilitation, 74, 56-63. Rehabilitation Services Administration. (2005). RSA program data and statistics: Fiscal year 2005 data. Washington, DC: Author. Richie, B. S., Fassinger, R. E., Linn, S., G., Johnson, J., Prosser, J. & Robinson, S. (1997). Persistence, connection, and passion: a qualitative study of the career development of highly achieving African American-Black and White women. Journal of Counseling Psychology, 44, 133-148. Rubin, S.E., Chan, R., & Thomas, D. (2003). Assessing changes in life skills and quality of life resulting from rehabilitation services. Journal of Rehabilitation, 69, 4-9. Sackett, D.L., Rosenberg, W.M.C., Gray, J.A.M., Hayes, R.B., & Richardson, W.S. (1996). Evidence-based medicine: What it is and what it isn’t. British Medical Journal, 312, 170- 180. Sackett, D. L., Straus, S. E., Richardson, W. S., Rosenberg, W., & Haynes, R. B. (2000). Schlosser, L. Z., Dewey, J. H.,&Hill, C. E. (2012). Auditing. In C. E. Hill (Ed.), Consensual qualitative research: A practical resource for investigating social science phenomena (pp. 135–144). Washington, DC: American Psychological Association. Schultz, J. C., Koch, L. C., & Kontosh, L. G. (2007). Establishing rehabilitation research priorities for the national council on rehabilitation education. Rehabilitation Education, 21(3), 149-158. preparation, and professionalism. ERIC Digest. Sexton, T. L. (1999). Evidence-based counseling: Implications for counseling practice, Shaw, L. R., Leahy, M. J., Chan, F., & Catalano, D. (2006). Contemporary issues facing rehabilitation counseling: A delphi study of the perspectives of leaders of the discipline. Rehabilitation Education, 20(3), 163-178. Sherman, S., Leahy, M. J., Valle, R. D., Anderson, C. A., Tansey, T. N., & Lui, K. (2014). Organizational and cultural factors that promote creative best practices in the public rehabilitation program: Findings from a four-state multiple case study. Journal of Vocational Rehabilitation, 41, 115–125. Solomon, P., Cavanaugh, M. M., & Draine, J. (2009). Randomized control trials: Design and implementation for community-based psychosocial interventions. New York: Oxford University Press. 146 Stodden, R. A., & Mruzek, D. W. (2010). An introduction to postsecondary education and employment of persons with autism and developmental disabilities. Focus on Autism and Other Developmental Disabilities, 25(3), 131-133. Straus, S. E., Glasziou, P., Richardson, W. S., & Haynes, R. B. (2011). Evidence-based medicine: How to practice and teach it (4th Ed.). Edinburgh, UK: Churchhill Livingstone. for developing grounded theory (Vol. 15). Newbury Park, CA: Sage. to rehabilitation counseling research. Rehabilitation Education, 24(3&4), 213-224. Strauser, D.R., & Wong, A.W.K. (2010). Impact not efficacy: Applying the RE-AIM framework Strauss, A. L., & Corbin, J. (1990). Basics of qualitative research: Techniques and procedures Strauss, A. L., & Corbin, J. (1998). Basics of qualitative research: Grounded theory procedures Tanner, C. A. (2008). Clinical judgment and evidence-based practice: Toward pedagogies of Tansey, T. N., Bezyak, J., Chan, F., Leahy, M. J., & Lui, J. (2014). Social-cognitive predictors of and techniques (2nd ed.). Thousand Oaks, CA: Sage Publications. integration. Journal of Nursing Education, 47(8), 335-336. readiness to use evidence-based practice: A survey of state vocational rehabilitation counselors. Journal of Vocational Rehabilitation, 41, 127–136. Tarvydas, V., Addy, A., & Fleming, A. (2010). Reconciling evidence-based research practice with rehabilitation philosophy, ethics and practice: From dichotomy to dialectic. Rehabilitation Education, 24(3 & 4), 191-204. Thirty-Third Institute for Rehabilitation Issues. (2008). Evidence based practices: Improving employment outcomes for people with significant disabilities. Hot Springs, AR: Rehabilitation Continuing Education Center. Thomason, T. (2010). The trend toward evidence-based practice and the future of psychotherapy. American Journal of Psychotherapy, 64(1), 29-38. www.dol.gov/ odep/ topics/ disabilityemploymentstatistics.htm U.S. Department of Labor. (2016). Disability employment statistics. Retrieved from https:// U.S. Government Accountability Office. (2005). Vocational rehabilitation—Better measures and monitoring could improve the performance of the VR Program (GAO-05–865). Washington, DC: Author. U.S. Government Accountability Office. (2007). Vocational rehabilitation—Improved information and practices may enhance state agency earnings outcomes for SSA beneficiaries (GAO-07–521). Washington, DC: Author. 147 Walker, M. (2001). E-mail surveys: Non-response Figures Reflected. In Reips, U.D., Bosnjak, M. (Eds.), Dimensions of Internet science (pp. 231-237). Lengerich, Germany: Pabst Science Publishers. headaches: An evidence-based practice approach. Journal of Clinical Psychology, 62( 11), 1367– 1378. Walker, B. B., Seay, S. J., Solomon, A. C., & Spring, B. (2006). Treating chronic migraine Wampold, B. E. (2001). The great psychotherapy debates. Mahwah, NJ: Lawrence Erlbaum. Wampold, B. E., & Bhati, K. S. (2004). Attending to the omissions: A historical examination of evidence-based practice movements. Professional Psychology: Research and Practice, 35(6), 563-570. Winch, S., Henderson, A., & Creedy, D. (2005). Read, think, do!: A method for fitting research evidence into practice. Journal of Advanced Nursing Practice, 50, 20-26. Yaeda, J., Iwanaga, K., Fujikawa, M., Chan, F., & Bezyak, J. (2015). The use of evidence-based practice among Japanese vocational rehabilitation professionals. Rehabilitation Counseling Bulletin, 58, 70–79. Zhang, Y., & Wildemuth, B. (2009). Qualitative analysis of content. In B. Wildemuth (Ed.), Applications of coail research methods to questions in information and library science (pp. 308-319). Santa Barbara, CA: Greenwood Press. 148