REDUCING MISSED BEHAVIORAL HEALTH APPOINTMENTS 1 Reducing Missed Behavioral Health Appointments in a Federally Qualified Center: A Quality Improvement Project TaSharra L. Stephen-Green Michigan State University REDUCING MISSED BEHAVIORAL HEALTH APPOINTMENTS 2 Table of Contents Contents Abstract ........................................................................................................................................... 4 Problem Description ....................................................................................................................... 6 Available Knowledge ...................................................................................................................... 7 Evidence-Based Quality Improvement Model: PDSA ................................................................... 9 Review of the Literature ............................................................................................................... 10 Synthesis of the Literature .............................................................................................................11 Predictive Modeling .................................................................................................................. 11 Patient Reminders ..................................................................................................................... 12 Double Booking ........................................................................................................................ 12 Lead Time ................................................................................................................................. 12 Psychoeducation and Motivational Interviewing ...................................................................... 13 Specific Aims ................................................................................................................................ 13 Methods......................................................................................................................................... 14 Context ...................................................................................................................................... 14 Values and Beliefs ................................................................................................................. 15 Accessibility .......................................................................................................................... 15 Contributors ........................................................................................................................... 16 Services .................................................................................................................................. 16 Interventions .............................................................................................................................. 17 Measures.................................................................................................................................... 18 Process Measures ................................................................................................................... 18 Outcome Measures ................................................................................................................ 18 Data Collection ...................................................................................................................... 19 Timeline ................................................................................................................................. 20 Ethical Considerations ........................................................................................................... 20 Results ........................................................................................................................................... 21 Discussion ..................................................................................................................................... 24 Limitations .................................................................................................................................... 26 References ..................................................................................................................................... 28 REDUCING MISSED BEHAVIORAL HEALTH APPOINTMENTS 3 Appendix A ................................................................................................................................... 33 Appendix B ................................................................................................................................... 34 Appendix C ................................................................................................................................... 41 Appendix D ................................................................................................................................... 43 Appendix D ................................................................................................................................... 44 Appendix E ................................................................................................................................... 45 Appendix F.................................................................................................................................... 46 Appendix G ................................................................................................................................... 47 Appendix H ................................................................................................................................... 48 Appendix I .................................................................................................................................... 49 Appendix J .................................................................................................................................... 50 Appendix K ................................................................................................................................... 51 Appendix L ................................................................................................................................... 52 REDUCING MISSED BEHAVIORAL HEALTH APPOINTMENTS 4 Abstract Background: The COVID-19 pandemic has highlighted the importance of mental health in public health services, leading to an increased reliance on telehealth for behavioral health care (Koonin et al., 2020). However, the shift to telehealth has been associated with higher rates of missed appointments in behavioral health settings (Koonin et al., 2020). Problem: Since January 2023, a Federally Qualified Health Center (FQHC) in Southeastern Michigan has experienced a significant increase in no-show rates compared to the periods before and during the pandemic. Purpose: This quality improvement (QI) project aimed to reduce the no-show rate by 5% by identifying barriers to attendance and implementing targeted interventions, including 24-hour appointment reminders and gratitude calls. Methods: An initial 5-question survey was used to gather data on potential barriers. Appointment data were collected from the Electronic Health Record (EHR) system. The interventions were executed using the Plan-Do-Study-Act (PDSA) model and evaluated with statistical analysis in Excel. At the conclusion of the QI project, results were analyzed to determine whether the FQHC should adopt, adapt, or abandon the interventions. Results: The FQHC no-show rate was reduced by 5% during the eight-week intervention period from 27% to 22%. Patients reported transportation and forgetfulness as contributing factors to no-shows. Transportation issues were reported by 29% of the patients in the survey. REDUCING MISSED BEHAVIORAL HEALTH APPOINTMENTS 5 Reducing Missed Behavioral Health Appointments in a Federally Qualified Health Center: A Quality Improvement Project According to the National Institute of Mental Health (NIMH, 2023), approximately 57.8 million adults in the United States were affected by mental illness in 2023, representing over 20% of the adult population. Mental illnesses are broadly categorized into Any Mental Illness (AMI) and Serious Mental Illness (SMI). The National Alliance on Mental Illness (NAMI, 2023) reported that more than 1.4 million adults in Michigan experience mental health issues, with an estimated prevalence of SMI at 1 in 20 Americans (NAMI, 2023). In Michigan, approximately 355,000 individuals are affected by SMI (NAMI, 2023). The rising prevalence of mental health symptoms has led to an increased demand for mental health services. Given the limited availability of mental health professionals and the growing number of individuals seeking care, many mental health providers are operating at or near full capacity (American Psychological Association, 2022). In Wayne County, Michigan, the patient-to-provider ratio is 300:1 (Population Health Institute, 2023). This imbalance contributes to longer waiting periods, with patients often placed on waitlists for care (Population Health Institute, 2023). Despite the widespread need for mental health services and the high patient-to- provider ratio, there has been an increase in the number of missed in-person appointments in behavioral health (Population Health Institute). Missed appointments are a persistent issue in outpatient behavioral health settings. Milicevic et al. (2020) note that missed appointment rates in behavioral health can exceed 50%, compared to about 10% in primary care. Another study found that no-show rates can vary from 12% to 80% across different healthcare settings (Marbouh et al., 2020). High no-show rates negatively impact access to quality care and can reduce available resources at healthcare centers REDUCING MISSED BEHAVIORAL HEALTH APPOINTMENTS 6 (Marbouh et al., 2020). Moreover, longer wait times for potential patients may lead them to seek treatment elsewhere (Marbouh et al., 2020). These challenges underscore the need for strategies to address the increasing prevalence of missed appointments in mental health care. Problem Description There has been a noticeable increase in missed behavioral health appointments at a Federally Qualified Health Center (FQHC) in Southeastern Michigan (Central City Health, 2020). From January to August 2023, the no-show rate at this FQHC was significantly higher than pre-pandemic and pandemic levels (Central City Health, 2020). Missed appointments not only resulted in unused time slots but also led to decreased operational revenue for the behavioral health center (Marbouh et al., 2020). Data collected from January to August 2023 showed that the FQHC served 2,891 behavioral health patients, with a total of 38,502 scheduled appointments. Out of the total scheduled appointments, 10, 222 appointments were no-shows, resulting in a no-show rate of 26.5%. This high rate has potentially led to a 25% decrease in profits, impacting the delivery of care (Mohammadi et al., 2018). Longer wait times for appointments may also drive potential patients to seek services elsewhere (Marbouh et al., 2020). Identifying factors contributing to the high no-show rate and implementing effective solutions were ongoing challenges at the FQHC. According to the clinical director (personal communication, November 22, 2023), the FQHC struggled to pinpoint the specific factors driving these no-show rates. Potential contributing factors included transportation issues, socioeconomic status, co-occurring disorders, housing insecurity, and forgetfulness. Additionally, the reliance on telehealth services due to COVID-19 may have influenced no-show rates (D. Austin, personal communication, November 22, 2023). REDUCING MISSED BEHAVIORAL HEALTH APPOINTMENTS 7 Despite using automated phone reminders through the Electronic Health Record (EHR) system, there was not an observed improvement in missed appointment rates (D. Austin, personal communication, November 22, 2023). Additionally, if a patient did not arrive within 15 minutes of the scheduled time, a follow-up call was made to reschedule or offer a telehealth option. However, this step had been deprioritized due to staffing issues (D. Austin, personal communication, November 22, 2023). The FQHC contacted patients 72, 48, and 24 hours before their appointments via the EHR, but no further interventions were implemented to address no-shows (D. Austin, personal communication, November 22, 2023). The no-show rate at this Detroit-based FQHC is higher than the national average of 18% to 21.9% for behavioral health clinics (Teo et al., 2023). From January to August 2023, the no-show rate was 26.5%, surpassing the national mean of 14.9% from October 2020 to September 2021 (Teo et al., 2023). Available Knowledge The rate of mental illness has increased since the onset of the COVID-19 pandemic (Javed et al., 2020). Beginning in early 2020, the COVID-19 pandemic challenged the structure and resilience of the American healthcare system and significantly impacted the nation's mental health. Globally, the incidence and prevalence of mental health disorders, particularly anxiety and depression, rose by 25% within the first year of the pandemic (World Health Organization, 2022). A survey conducted by the Centers for Disease Control and Prevention (CDC) in June 2020, which included 5,412 participants in the United States, found that nearly half (40.9%) experienced worsened mental health since the pandemic began (Czeisler et al., 2020). The pandemic's impact left many individuals with behavioral health issues stemming from prolonged REDUCING MISSED BEHAVIORAL HEALTH APPOINTMENTS 8 isolation, personal illness, grief, the deaths of close contacts, and fear of the unknown (Monaghesh & Hajizadeh, 2020). During the pandemic, the use of telehealth services increased, allowing patients to receive treatment via video without needing to visit a clinic in person. Telehealth became accessible and convenient for many patients, which helped reduce no-show rates in outpatient settings (Monaghesh & Hajizadeh, 2020). Patients were more likely to attend their telehealth appointments via video or phone call. However, as the pandemic subsided, most providers encouraged patients to return to in-person visits at outpatient clinics, though this transition has been gradual (Koonin et al., 2020). Behavioral health outpatient clinics continue to face challenges with patients attending their appointments. A "no-show" refers to a scheduled appointment that is not canceled within a specified timeframe, typically 24 hours before the appointment (Oikonomidi et al., 2022). High no-show rates can result in financial losses for clinics and negatively impact patient health (Oikonomidi et al., 2022). A longstanding concern in healthcare is that patients who miss appointments may not receive adequate care, increasing their risk of hospitalization (Marbouh et al., 2020). Additionally, higher no-show rates can delay diagnoses and treatments for both existing and new patients. For example, new patients may face delays in receiving services due to scheduling constraints (Marbouh et al., 2020). Thus, no-show rates affect both current patients and potential new patients. Community factors and environmental context also impact no-show rates. These contexts include geographical location, staffing levels, available services, and the time required to complete tasks (Marbouh et al., 2020). The population served by a clinic can also significantly influence no-show rates. Several barriers contribute to no-shows, such as extended wait times, REDUCING MISSED BEHAVIORAL HEALTH APPOINTMENTS 9 individual health beliefs, financial constraints, and associated costs (National Library of Medicine, 2021). No-show rates are also correlated with factors like patient forgetfulness, demographic characteristics, transportation availability, psychiatric conditions, past attendance history, and the interval between appointments (Marbouh et al., 2020). Additional variables related to no-shows include age, gender, and socioeconomic status, which are frequently cited as contributing factors (Marbouh et al., 2020). Evidence-Based Quality Improvement Model: PDSA Implementing a model of improvement to reduce no-show rates in behavioral health settings can help decrease the number of missed appointments. The Plan-Do-Study-Act (PDSA) model was designed to implement interventions that facilitate change and assess their effectiveness. According to the Institute for Healthcare Improvement (IHI, 2023a), the PDSA model supports multiple cycles of testing, where each cycle is analyzed to determine whether the change is effective, if adjustments are needed, and whether the intervention should be adopted, adapted, or abandoned. These cycles can be repeated continuously until the desired outcome is achieved. At the health center, stakeholders use the PDSA cycle to drive quality improvement (IHI, 2023b). Consequently, the PDSA model was selected to develop strategies to reduce no- show rates in behavioral health, as it is a highly effective tool for quality improvement (QI) projects. The PDSA cycle was specifically chosen to identify methods for improving no-show rates due to its ability to provide rapid testing to determine if a proposed change leads to improvement. It involves a four-step process: developing a plan, implementing the plan on a small scale, analyzing the resulting data, and reviewing the lessons learned. If the desired outcome is not achieved, modifications are made, and a new test is planned. This model guided REDUCING MISSED BEHAVIORAL HEALTH APPOINTMENTS 10 the QI project by allowing for iterative testing and determining whether the implemented changes should be adopted, adapted, or abandoned (IHI, 2023b). Throughout the implementation of the PDSA model, there were opportunities for improvement at each stage of the process (see Appendix A ). Review of the Literature A comprehensive literature search was conducted to evaluate studies relevant to reducing no-show rates in behavioral health settings. The search focused on identifying effective strategies for decreasing missed appointments. The search terms used included "no show" OR "missed appointments" OR "non-attendance" AND "behavioral health" AND "mental health". The search was performed using the Cumulative Index to Nursing and Allied Health Literature (CINAHL), the PubMed database, and the Google search engine. The search was limited to peer-reviewed articles published within the last five years, from 2018 to 2023. The studies identified included a mix of randomized controlled trials, systematic reviews, and predictive modeling studies aimed at identifying ways to reduce no-show rates. The search yielded 115 articles from CINAHL, 15 articles from PubMed, and 2 articles from Google. Each article was reviewed for relevance to the Doctor of Nursing Practice (DNP) project by examining the titles and abstracts. After removing duplicates and articles not related to reducing no-show rates, 13 articles remained for a full-text review. Inclusion criteria for the review were studies involving adults aged 18 and older, with English as the preferred language, and a focus on behavioral health, unless the article specifically addressed no-show rates in other healthcare settings. Articles that determined or evaluated interventions aimed at reducing no-show rates were also included. After reviewing the articles and eliminating duplicates, 13 articles were selected for review, of which 11 met the inclusion REDUCING MISSED BEHAVIORAL HEALTH APPOINTMENTS 11 criteria. Exclusion criteria included articles written in languages other than English, studies involving individuals under 18 years of age, and literature unrelated to the project or published more than five years ago (see Appendix B). Two of the 13 articles were excluded because they met one or more of the exclusion criteria. Synthesis of the Literature Extensive research on the causes and contributing factors of no-shows provides a critical foundation for interventions aimed at reducing no-show rates in behavioral health settings. The key themes that emerged from the literature synthesis included predictive modeling, patient reminders, lead time, psychoeducation and motivational interviewing, double booking, and strategically scheduling appointments later in the week to decrease no-show rates. The studies reviewed focused on analyzing and improving no-show rates by identifying areas with potential for improvement (see Appendix C). Predictive Modeling Predictive modeling has proven to be a valuable tool for distinguishing between patients who are likely to miss an appointment and those who are likely to attend (Milicevic et al., 2020; Mohammadi et al., 2018; Oikonomidi et al., 2022; Valero-Bover et al., 2022). This approach involves extracting critical information from a system to predict an individual's likelihood of attending their appointment, making it a beneficial asset in healthcare settings (Jung et al., 2020). Predictive modeling uses data derived from patients' historical attendance records to provide calculated approximations of attendance probabilities. REDUCING MISSED BEHAVIORAL HEALTH APPOINTMENTS 12 Patient Reminders Patient forgetfulness is a common reason for missed appointments. In addition to predictive modeling, patient reminders have been shown to positively influence attendance and decrease no-show rates. Reminders, such as text messages, phone calls, emails, and assistance from patient navigators, serve as a courtesy to patients and help ensure continuity of care. These reminders, when incorporated into predictive models, have demonstrated high-certainty evidence in reducing no-show rates (Oikonomidi et al., 2022; Opon et al., 2020; Ulloa-Perez et al., 2022). Allowing patients to choose their preferred method of receiving reminders can further enhance patient satisfaction. Double Booking The practice of double booking has also demonstrated a positive impact on reducing missed appointments. Double booking involves scheduling patients who frequently miss appointments alongside those who are more likely to attend, thereby optimizing the appointment schedule and improving operational efficiency at behavioral health centers (Marbouh et al., 2020). This strategy can help make new appointments available more quickly and reduce lead times. However, double booking carries the risk of both patients arriving simultaneously, potentially causing delays in provider availability. Lead Time Lead time, or the interval between scheduling and the actual appointment, is a critical factor influencing the likelihood of appointment attendance (Milicevic et al., 2020; Mohammadi et al., 2018; Teo et al., 2023; Valero-Bover et al., 2022). Research has identified that certain days of the week have higher no-show rates. For instance, appointments scheduled early in the week, REDUCING MISSED BEHAVIORAL HEALTH APPOINTMENTS 13 such as on Mondays, are more likely to be missed than those later in the week, like Thursdays and Fridays (Ellis et al., 2022; Mohammadi et al., 2018). Understanding the relationship between lead time and no-show rates can help healthcare providers create schedules that minimize the likelihood of missed appointments. Psychoeducation and Motivational Interviewing Studies by Naar and Safren (2017) demonstrated a 50% reduction in missed appointments following the implementation of interventions such as psychoeducation and motivational interviewing. Psychoeducation involves providing educational information about mental health and well-being to help individuals manage their illnesses and enhance their treatment adherence. Motivational interviewing is a patient-centered technique designed to increase motivation for change by addressing and resolving ambivalence (Naar & Safren, 2017). These strategies have been effective in modifying patient behavior regarding missed appointments (Boyle & Schwinck, 2022). Integrating behavioral adjustments, including psychoeducation, can provide patients with the support and knowledge needed to better understand their treatment. Furthermore, using psychoeducation alongside motivational interviewing can help patients recognize the importance of taking responsibility for their actions. Specific Aims The purpose of this Quality Improvement (QI) project was to reduce the no-show rate at a behavioral health center in Southeast Michigan by 5% by April 5, 2024. The specific aims of this project were to (1) identify, collect, and monitor the key metrics necessary to evaluate outcomes and (2) establish a sustainable process that includes a plan for implementing changes to the procedures for reminding clients of upcoming appointments. The QI process focused on REDUCING MISSED BEHAVIORAL HEALTH APPOINTMENTS 14 deploying tools to ensure patients attend their scheduled appointments. Research indicates that reminders are the most effective method for reducing missed appointments (Opon et al., 2020). Currently, the Federally Qualified Health Center (FQHC) utilizes automated appointment reminders. This QI project aimed to build upon these efforts to further decrease the number of no-show appointments. By reducing no-show rates, the project also sought to enhance the overall quality of patient care, improve customer service, and shorten wait times. The implementation of this QI project is crucial as it seeks to modify and customize existing processes to better meet client needs, thereby improving attendance rates and reducing the occurrence of no-shows. Methods Context This Quality Improvement (QI) project was conducted at a Federally Qualified Health Center (FQHC) in Southeastern Michigan. Originally established in the 1960s under the name "Model City Neighborhood," the center transitioned to a 501(c)(3) private nonprofit organization and has since become an essential resource for the communities it serves (Central City Health, 2018). The FQHC is known for providing various services to populations with limited access to care. Although it operates on a fee-based system, individuals are not denied care based on their ability to pay (Central City Health, 2020). Services offered include behavioral health, primary care, dental care, housing assistance, outreach programs, and Assertive Community Treatment (ACT). The community served by the FQHC faces challenges such as homelessness and substance abuse. Individuals seeking medical assistance at the center, referred to as "patients," have access to the full range of services provided. Most patients are covered by Medicaid or REDUCING MISSED BEHAVIORAL HEALTH APPOINTMENTS 15 Medicare, with a small percentage having private insurance (Central City Health, 2020). The area is characterized by a growing, diverse population of low to middle-class residents from various racial backgrounds. FQHCs are eligible for funding under Section 330 of the Public Health Service Act and receive enhanced Medicare and Medicaid benefits (FQHC Associates, n.d.). The FQHC provides healthcare to underserved populations without discrimination based on race, age, gender, or religion (Central City Health, 2023). Due to Medicaid billing restrictions, consumers cannot be billed for services not rendered, meaning providers cannot charge for missed appointments (Michigan Department of Health & Human Services, 2023). FQHCs are required to maintain a quality assurance program and have a governing board of directors to oversee operations (FQHC Associates, n.d.). The integration of multiple services within a single location enables the provision of comprehensive care. Values and Beliefs The FQHC is committed to delivering high-quality care that is widely recognized and valued within the community. The core principles that support the center's mission and vision include creating a welcoming environment that fosters a sense of comfort and security among patients. The center's care is evidence-based, promoting the well-being and recovery of its clients. Accessibility The FQHC is strategically located for easy access by various communities through public transportation. It also offers low-cost housing attached to the center for disadvantaged individuals and those with disabilities, helping to address the issue of homelessness. Patients can REDUCING MISSED BEHAVIORAL HEALTH APPOINTMENTS 16 access additional services, such as peer support, substance use treatment, community resources, and insurance assistance (Central City Health, 2018). Contributors Several stakeholders are involved in ensuring the smooth operation and delivery of quality services at the FQHC. These stakeholders include leadership, the board of directors, senior staff, and the QI team. The FQHC also collaborates with external partners to provide supportive services and funding. Services The FQHC offers behavioral health services to patients with mental health disorders and co-occurring conditions. The behavioral health team comprises psychiatrists, psychiatric mental health nurse practitioners, peer support specialists, case managers, and receptionists (Central City Health, 2020). A SWOT analysis (Strengths, Weaknesses, Opportunities, Threats) was conducted to evaluate internal and external factors and any associated risks (Teoli et al., 2022). This analysis revealed that the FQHC provides a comprehensive range of health services aimed at consistent and comprehensive care tailored to individual needs. However, challenges such as the patient-to-staff ratio impact the staff's ability to deliver high-quality care. Additionally, the high rate of no-shows poses a risk of reduced federal funding if allocations are based on a per-patient basis. There is an opportunity to reduce no-show rates by establishing a streamlined procedure for sending appointment reminders. The full SWOT analysis is included in Appendix D. Despite the many strengths of the FQHC, significant rates of missed appointments remain a concern. REDUCING MISSED BEHAVIORAL HEALTH APPOINTMENTS 17 Interventions This quality improvement (QI) project utilized the Plan-Do-Study-Act (PDSA) methodology to implement changes aimed at reducing no-show rates at a Federally Qualified Health Center (FQHC) in Southeast Michigan. The planning phase began with a literature review to understand the factors contributing to no-show rates in clinical settings and to identify effective strategies for reducing these rates. A "no-show" was defined as a patient who failed to attend a scheduled appointment without prior notice within 24 hours. The objective of this project was to reduce the no-show rate by 5% at the FQHC by implementing targeted interventions. Project interventions (Appendix E) included: obtaining patients' preferred mode of communication in the EHR; live communication with 10 patients daily, 24 hours before their scheduled appointments; voluntary patient questionnaires regarding possible barriers that may contribute to missed appointments; contacting the patient after the visit to convey appreciation; and collaborating with the project mentor and IT department to integrate automated post-appointment messaging for patients who attended appointments or missed appointments. The PDSA cycle involved close collaboration between the project leader and the support specialist, with weekly monitoring of interventions. The project obtained approval from the Institutional Review Board (IRB) on February 8, 2024. During the implementation phase, the support professional contacted patients via phone 24 hours before their scheduled appointments to remind them of their upcoming visits. The project leader intermittently monitored the implementation process and collaborated with the support specialist to gather qualitative feedback on patient responses to the reminders. Data on appointment attendance and patient feedback were collected weekly. REDUCING MISSED BEHAVIORAL HEALTH APPOINTMENTS 18 While the FQHC already used automated appointment reminders, the QI project expanded these efforts by incorporating live calls. Research indicates that live reminders significantly reduce no-show rates compared to automated messages or voicemails (Teo et al., 2017). The project aimed to build on this evidence, integrating additional strategies to further decrease missed appointments. The collected data were analyzed to compare the actual outcomes with the predicted outcomes established during the planning phase. The project team reviewed the data weekly, discussing the effectiveness of the interventions and identifying any areas for improvement. Measures This project employed both process and outcome measures to gauge the success of QI interventions. These process and outcome measures are crucial for testing and evaluating the effectiveness of the implemented changes. Process Measures The primary process measure is the number of live calls made to remind patients of their upcoming appointments. A detailed log was maintained to track calls made, calls answered, instances of no answer, voicemails left, or phones disconnected. Weekly data were collected and visualized using run charts to monitor trends and effectiveness over time. Outcome Measures The key outcome measures included the number of no-show appointments and patient feedback. The number of no-show appointments was tracked through the Electronic Health Record (EHR) system. Patient feedback was gathered using a five-question survey designed to REDUCING MISSED BEHAVIORAL HEALTH APPOINTMENTS 19 capture their voices and opinions about barriers related to appointments. Additional outcome measures included the number of follow-up calls made to reschedule missed appointments. Data Collection Following the implementation of the QI project, the team closely monitored progress throughout the first PDSA cycle. Data collection occurred weekly over four weeks, targeting a total of 40 patients scheduled for future appointments. The dataset included 10 patients per day over a four-day workweek, with automated reminders and rescheduling messages for missed visits. Data collection across four weeks provided a summary of the percentages of patients contacted and those who attended their appointments. A run chart was used to compare weekly data. A second PDSA cycle, also four weeks long, involved two additional interventions: support specialists making reminder calls and automated gratitude messages for attended appointments. Data from this second cycle were compared to the first to determine if these additional interventions further reduced no-show rates. After analyzing the results, the FQHC would decide whether to adopt, adapt, or abandon the additional interventions. To assess whether factors such as transportation, scheduling ease, and lead time impacted no- show rates, the team provided a voluntary survey to patients with the following questions: Have you missed any behavioral health appointments in the last six months?; Do you have trouble reaching a representative when calling the health center?; Are you able to schedule an appointment within 0–2 weeks of calling?; Was transportation a problem for your behavioral health appointment in the past six months?; Have you missed doctor's appointments because of financial reasons? Survey questions are detailed in Appendix E. REDUCING MISSED BEHAVIORAL HEALTH APPOINTMENTS 20 Timeline The project's timeline included several key stages: planning, implementation, data analysis, and dissemination. Planning included obtaining approval from the College of Nursing, the university Institutional Review Board (IRB), and the FQHC. Following IRB approval, a pre- implementation stage was conducted, which involved a meeting with the project mentor and support staff to review the planned interventions. The project consisted of two Plan-Do-Study-Act (PDSA) cycles. The initial PDSA cycle began with a pre-implementation phase focused on developing an anonymous, voluntary questionnaire, which was administered during this period. Phase two of the first cycle involved a four-week implementation period, where patients were called 24 hours before their appointments as a reminder. Data was collected at the end of each week, and brief meetings with support personnel and the project mentor were conducted either in person, over the phone, or via email to review progress. The initial results were analyzed during the first week of the second PDSA cycle, and improvements were made based on the findings. The second PDSA cycle comprised four weeks of implementing patient reminders and addressing transportation needs, followed by data collection, evaluation, analysis, and drawing conclusions. A project timeline is provided in Appendix F. Ethical Considerations This Quality Improvement (QI) project posed no risk of harm to human subjects and was not classified as research. Instead, it was considered a QI initiative aimed at improving operational processes and patient outcomes. Before the project's implementation, several steps were required for approval. Initially, the Michigan State University College of Nursing approved REDUCING MISSED BEHAVIORAL HEALTH APPOINTMENTS 21 the project, after which it was submitted to the Institutional Review Board (IRB) for further review. The project mentor, who was the Executive Director of Nursing Services and Compliance/Risk Manager at the FQHC, played a key role in ensuring the project's alignment with organizational standards. Although the FQHC did not have its own IRB, the project manager independently determined whether the QI project was accepted or denied at the facility. The project mentor worked closely with the project team to ensure compliance with organizational guidelines and standards. The results of this project directly impacted the institution's operations by identifying areas for enhancement and implementing necessary remedial measures. The project provided an opportunity to improve patient outcomes. To ensure all ethical standards were met, the facility included an agreement form for project approval (see Appendix G for agency approval letter). A primary focus of this Doctor of Nursing Practice (DNP) project was to ensure the protection of health information. The project leader collaborated closely with the project mentor, data analyst, and IT department to ensure compliance with all organizational standards and objectives. The project strictly adhered to acquiring only the information necessary to achieve the aims of the DNP project, thereby safeguarding patient confidentiality and data security. Results The study reviewed data on missed appointments for the eight weeks preceding the start of telephone call reminders. The FQHC recorded a no-show rate of 27% during this period (see Appendix H for pre-intervention no-show rates). Early in the project, it was noted that the FQHC was also conducting a survey focused on member satisfaction, which could potentially interfere with the DNP project survey, as patients would be asked to complete two questionnaires at their REDUCING MISSED BEHAVIORAL HEALTH APPOINTMENTS 22 appointment. To address this, the project mentor advised distributing the DNP project questionnaires only to patients of one psychiatrist at the clinic rather than all patients. Additionally, the FQHC expanded during the proposal process, resulting in most providers relocating to a new facility with their patients. While this led to a limited number of patients at the original clinic, it did not impact the project, as it focused on a specific group of patients. The DNP project involved distributing 50 anonymous surveys over one week to assess the prevalence of no-show appointments and factors that might prevent patients from attending scheduled appointments. Out of the 50 surveys distributed, 29 were returned, yielding a 58% response rate (see Appendix I). The survey results (see Appendix J) revealed that 41% of respondents had missed a behavioral health appointment in the past six months, 41% had trouble reaching a representative when calling the health center, 69% were able to schedule an appointment within two weeks of calling, 21% reported missing an appointment due to transportation issues, and 17% missed an appointment due to the cost of transportation. The interventions included 24-hour appointment reminders and gratitude calls for attending scheduled appointments. Initially, the project aimed to collect data from eight patients daily. However, it was decided to increase the number of patients to 10 per day to increase the amount of data collected. After the first week, it became clear that 10 patients per day did not provide sufficient data, so the number was increased to 20 patients per day. This adjustment provided a larger data pool, enhancing the potential for more robust data analysis. Due to limited staff at the FQHC, the project leader assisted the support specialist with making patient reminders throughout the week. REDUCING MISSED BEHAVIORAL HEALTH APPOINTMENTS 23 The reminder calls were partially effective. Some patients were receptive to the calls and expressed gratitude. However, some patients, despite answering the phone and confirming their appointments, still did not attend. During the first week of interventions, data collection covered 40 booked appointments. The no-show rate for this period was 10%, 13% of appointments were canceled, and 5% of behavioral health appointments were rescheduled. The number of reminder calls per day was increased to 20 for the following seven weeks to improve data collection and analysis. The average no-show rate for the initial four weeks of interventions was 22%, while 7% of patients canceled and rescheduled their visits. During the last four weeks of the PDSA cycle, the project leader instructed the support specialist to ask patients about their transportation needs during phone calls and provide transportation assistance to those who required it. The FQHC offered transportation support to patients who requested it when booking their follow-up visits. In the final four weeks of interventions, the no-show rate remained at 22%, and approximately 10% of patients chose to cancel and reschedule appointments. There was no significant difference in the no-show rates after transportation support was included compared to the first four weeks of implementation. Additionally, less than 10% of patients received reminders via voicemail, and there were also fewer than 10% of non-functioning phone numbers. The overall no-show rate for the eight-week project period was 22% (see Appendix K). The DNP project resulted in an increase in the number of patients attending their behavioral health appointments over the eight-week period. The no-show rate decreased by 5% during the implementation of 24-hour reminder calls. Pre-intervention data indicated a higher percentage of cancellations or rescheduling of appointments compared to the post-intervention REDUCING MISSED BEHAVIORAL HEALTH APPOINTMENTS 24 phase (see Appendix L). The project achieved its target of reducing the no-show rate as outlined in its aims. Continued implementation of additional interventions and ongoing modifications could further decrease the number of no-shows. Due to limited staff and resources, the effectiveness of gratitude calls could not be conclusively determined, making this intervention challenging and unsustainable for the FQHC. In the future, the FQHC could work with the IT department to implement automated gratitude messages. Other challenges that contributed to no-shows, such as forgetfulness and lack of transportation, were not fully addressed during the project interventions. Discussion The interventions implemented during the DNP project were designed to reduce missed appointments by sending reminders through verbal calls made by the support professional. Although reminder calls were made, several patients who confirmed their appointments still did not attend. The support professional provided feedback to the project leader and mentor about patient responses during these calls. Some patients expressed gratitude for the reminder, mentioning that they had forgotten about their appointments. However, other patients were frustrated because they had already received a reminder through the automated call system. The project observed that patients who received support from friends or family members were more likely to attend their appointments. The strategies used to reduce no-show rates benefited from initiating a conversation with the patient. During the reminder call, patients could address issues related to their upcoming visit, including transportation. The 24-hour reminder calls provided throughout the eight-week cycle successfully reduced no-show rates by five percent during the post-intervention period. The interventions chosen for the project were those that could be easily measured over a short period. REDUCING MISSED BEHAVIORAL HEALTH APPOINTMENTS 25 Additionally, the survey and phone calls facilitated the identification of specific barriers contributing to no-show rates, such as transportation and forgetfulness. These strategies were expected to be effective because multiple studies had successfully integrated 24-hour reminder calls into their routines and demonstrated their efficacy in reducing no-show rates. Two primary issues emerged: forgetfulness and transportation difficulties. The next intervention planned to incorporate gratitude phone calls after patients attended their appointments. However, the support specialist reported that conducting gratitude calls was not feasible due to the time required to complete reminder calls and other responsibilities. The project mentor agreed, citing low staffing levels at the FQHC. Since transportation was a significant issue identified during the calls, the support specialist inquired about transportation needs. This intervention helped patients secure the assistance they needed to attend their appointments. The findings of the DNP project at the FQHC were more closely aligned with issues observed in behavioral health settings than those described in many articles about no-shows in general healthcare. One of the primary factors contributing to high no-show rates, as identified in peer-reviewed literature, is patient forgetfulness. Several studies highlighted different types of reminders to help reduce no-show rates. The overall outcomes of the DNP project were consistent with the conclusions drawn from the literature review. Proactively contacting patients before their appointments helped address obstacles such as difficulty reaching the FQHC to reschedule or cancel appointments, as well as arranging transportation for upcoming visits. The project mentor acknowledged that achieving a sustained reduction in no-show rates would require ongoing adjustments and continuous evaluation to prevent rates from increasing again. Verbal reminders may also improve communication regarding the ability to fill in for REDUCING MISSED BEHAVIORAL HEALTH APPOINTMENTS 26 missed or rescheduled appointments, freeing up time for patients awaiting provider availability or urgent appointments. Reducing no-show rates can help the FQHC increase profitability, reduce hospitalizations, and enhance patient outcomes. Limitations The project had several limitations. The limited size of the initial project sample resulted in insufficient data collection during the first week of interventions. Additionally, relocating other providers to a new facility restricted the selection of patients to those of a single provider, which may have influenced the outcomes. On some days, the provider's sick leave further hindered data collection, leading to gaps in the data. Moreover, no demographic information was collected about the patients, limiting the ability to analyze how different patient characteristics may have impacted no-show rates or the effectiveness of the interventions. Another limitation was the policy regarding patient lateness. Some patients arrived more than 15 minutes late for their appointments. While most healthcare facilities have a 10 to 15- minute grace period before considering a patient a no-show, the FQHC maintains a policy of not turning away patients who arrive late. This policy could have affected the project's ability to measure no-show rates accurately. Conclusion The increasing incidence of missed appointments in mental health settings has been shown to negatively impact both patients and healthcare facilities. Research indicates that many individuals with mental illnesses also suffer from one or more comorbid conditions, which can contribute to higher no-show rates. Missing mental health appointments can lead to increased hospitalization rates, added stress on families, reduced resources due to restricted funding, and REDUCING MISSED BEHAVIORAL HEALTH APPOINTMENTS 27 challenges in managing illnesses effectively. Implementing targeted interventions can help mitigate these issues by reducing the frequency of missed visits. The findings of this DNP study demonstrate that patient reminders given the day before a scheduled appointment can effectively reduce no-show rates. Future strategies could include implementing a reward system for patients who attend their appointments, as well as other techniques to enhance patient engagement in their care. Further research could explore risk factors related to demographics to identify which populations are more likely to be no-shows and tailor interventions accordingly. Funding The budgetary implications for the FQHC in Southeast Detroit remained minimal, as the project leveraged existing resources. These resources included an established Electronic Health Record (EHR) system, which provided crucial data for the project and facilitated automatic appointment reminders. Additionally, the FQHC utilized RingCentral, a Voice over Internet Protocol (VoIP) system, enabling staff to contact patients with appointment reminders. The project mentor volunteered their time to oversee the initiative, further minimizing costs. If the interventions successfully reduce missed appointments, the FQHC could see increased revenue over time due to better patient attendance and more efficient resource utilization. REDUCING MISSED BEHAVIORAL HEALTH APPOINTMENTS 28 References American Psychological Association. (2022, November 15). Increased need for mental health care strains capacity. [Press release]. https://www.apa.org/news/press/releases/2022/11/mental-health-care-strains Boyle, M. K., & Schwinck, J. (2022). Reducing missed psychotherapy appointments: An advanced practice nurse-initiated telephone orientation protocol. Perspectives in Psychiatric Care, 58(4), 2756-2763. https://doi.org/10.1111/ppc.13116 Centers for Medicare & Medicaid Services. (2023, September 6). Electronic health records. https://www.cms.gov/priorities/key-initiatives/e-health/records Central City Health. (2018, September 6). Annual Report. https://www.centralcityhealth.com/about Central City Health. (2020). About. https://www.centralcityhealth.com/about Czeisler, M. É., Lane, R. I., Petrosky, E., Wiley, J. F., Christensen, A., Njai, R., Weaver, M. D., Robbins, R., Facer-Childs, E. R., Barger, L. K., Czeisler, C. A., Howard, M. E., & Rajaratnam, S. M. (2020). Mental health, substance use, and suicidal ideation during the COVID-19 pandemic — United States, June 24–30, 2020. MMWR. Morbidity and Mortality Weekly Report, 69(32), 1049-1057. https://doi.org/10.15585/mmwr.mm6932a1 Ellis, D. A., Sanders, J. G., Jenkins, R., & McAuslan, L. (2022). A weekday intervention to reduce missed appointments. PLOS ONE, 17(9), e0274670. https://doi.org/10.1371/journal.pone.0274670 FQHC Associates. (n.d.). What is a Federally qualified health center (FQHC)? — FQHC associates. https://www.fqhc.org/what-is-an-fqhc/ REDUCING MISSED BEHAVIORAL HEALTH APPOINTMENTS 29 Institute for Healthcare Improvement. (2023a). How to improve: Model for improvement: Testing changes. https://www.ihi.org/resources/how-to-improve/model-for- improvement-testing-changes Institute for Healthcare Improvement. (2023b). How to improve: Model for improvement. https://www.ihi.org/resources/how-to-improve Javed, B., Sarwer, A., Soto, E. B., & Mashwani, Z. (2020). The coronavirus (COVID-19) pandemic's impact on mental health. The International Journal of Health Planning and Management, 35(5), 993-996. https://doi.org/10.1002/hpm.3008 Jung, K., Kashyap, S., Avati, A., Harman, S., Shaw, H., Li, R., Smith, M., Shum, K., Javitz, J., Vetteth, Y., Seto, T., Bagley, S. C., & Shah, N. H. (2020). A framework for making predictive models useful in practice. Journal of the American Medical Informatics Association, 28(6), 1149-1158. https://doi.org/10.1093/jamia/ocaa318 Koonin, L. M., Hoots, B., Tsang, C. A., Leroy, Z., Farris, K., Tilman Jolly, B., Antall, P., McCabe, B., Zellis, C., Tong, I., & Harris, A. M. (2020). Erratum: Vol. 69, No. 43 trends in the use of Telehealth during the emergence of the COVID-19 pandemic — United States, January–March 2020. MMWR. Morbidity and Mortality Weekly Report, 69(45), 1711. https://doi.org/10.15585/mmwr.mm6945a9 Marbouh, D., Khaleel, I., Al Shanqiti, K., Al Tamimi, M., Simsekler, M. C., Ellahham, S., Alibazoglu, D., & Alibazoglu, H. (2020). Evaluating the impact of patient no-shows on service Quality. Risk Management and Healthcare Policy, 13, 509-517. https://doi.org/10.2147/rmhp.s232114 Michigan Department of Health & Human Services. (2023). Billing and reimbursement. REDUCING MISSED BEHAVIORAL HEALTH APPOINTMENTS 30 https://www.michigan.gov/mdhhs/assistance-programs/medicaid/portalhome/medicaid- providers/billing-and-reimbursement Milicevic, A. S., Mitsantisuk, K., Tjader, A., Vargas, D. L., Hubert, T. L., & Scott, B. (2020). Modeling patient no-show history and predicting future appointment behavior at the veterans administration’s outpatient mental health clinics: NIRMO-2. Military Medicine, 185(7-8), e988-e994. https://doi.org/10.1093/milmed/usaa095 Mohammadi, I., Wu, H., Turkcan, A., Toscos, T., & Doebbeling, B. N. (2018). Data analytics and modeling for appointment no-show in community health centers. Journal of Primary Care & Community Health, 9, 215013271881169. https://doi.org/10.1177/2150132718811692 Monaghesh, E., & Hajizadeh, A. (2020). The role of Telehealth during COVID-19 outbreak: A systematic review based on current evidence. BMC Public Health, 20(1). https://doi.org/10.1186/s12889-020-09301-4 Naar, S., & Safren, S. A. (2017). Motivational interviewing and CBT: Combining strategies for maximum effectiveness. Guilford Publications. https://psycnet.apa.org/record/201726282-000 National Alliance on Mental Illness. (2021). Mental Health in Michigan. https://nami.org/NAMI/media/NAMI- Media/StateFactSheets/MichiganStateFactSheet.pdf National Institute of Mental Health. (2023). Mental illness. https://www.nimh.nih.gov/health/statistics/mental-illness National Library of Medicine. (2021). Access to healthcare and disparities in access – 2021 REDUCING MISSED BEHAVIORAL HEALTH APPOINTMENTS 31 national healthcare quality and disparities report - NCBI bookshelf. https://www.ncbi.nlm.nih.gov/books/NBK578537/ Oikonomidi, T., Norman, G., McGarrigle, L., Stokes, J., Van der Veer, S. N., & Dowding, D. (2022). Predictive model-based interventions to reduce outpatient no-shows: A rapid systematic review. Journal of the American Medical Informatics Association, 30(3), 559-569. https://doi.org/10.1093/jamia/ocac242 Opon, S. O., Tenambergen, W. M., & Njoroge, K. M. (2020). The effect of patient reminders in reducing missed appointment in medical settings: A systematic review. PAMJ - One Health, 2. https://doi.org/10.11604/pamj-oh.2020.2.9.21839 Perla, R. J., Provost, L. P., & Murray, S. K. (2011). The run chart: A simple analytical tool for learning from variation in healthcare processes. BMJ Quality & Safety, 20(1), 46-51. https://doi.org/10.1136/bmjqs.2009.037895 Population Health Institute. (2023). Wayne, Michigan. County Health Rankings & Roadmaps. https://www.countyhealthrankings.org/explore-health- rankings/michigan/wayne?year=2023 Ramlucken, L., & Sibiya, M. N. (2018). Frequency and reasons for missed appointments of outpatient mental health care users in the uMgungundlovu district. Curationis, 41(1). https://doi.org/10.4102/curationis.v41i1.1835 Teo, A. R., Forsberg, C. W., Marsh, H. E., Saha, S., & Dobscha, S. K. (2017). No-show rates when phone appointment reminders are not directly delivered. Psychiatric Services, 68(11), 1098-1100. https://doi.org/10.1176/appi.ps.201700128 Teo, A. R., Niederhausen, M., Handley, R., Metcalf, E. E., Call, A. A., Jacob, R. L., Zikmund REDUCING MISSED BEHAVIORAL HEALTH APPOINTMENTS 32 Fisher, B. J., Dobscha, S. K., & Kaboli, P. J. (2023). Using nudges to reduce missed appointments in primary care and mental health: A pragmatic trial. Journal of General Internal Medicine, 38(S3), 894-904. https://doi.org/10.1007/s11606-023-08131-5 Teoli, D., Sanvictores, T., & An, J. (2022). SWOT Analysis. In StatPearls. Ulloa-Pérez, E., Blasi, P. R., Westbrook, E. O., Lozano, P., Coleman, K. F., & Coley, R. Y. (2022). Pragmatic randomized study of targeted text message reminders to reduce missed clinic visits. The Permanente Journal, 26(1), 64-72. https://doi.org/10.7812/tpp/21.078 Valero-Bover, D., González, P., Carot-Sans, G., Cano, I., Saura, P., Otermin, P., Garcia, C., Gálvez, M., Lupiáñez-Villanueva, F., & Piera-Jiménez, J. (2022). Reducing non attendance in outpatient appointments: Predictive model development, validation, and clinical assessment. BMC Health Services Research, 22(1). https://doi.org/10.1186/s12913-022-07865-y World Health Organization. (2022, March 2). COVID-19 pandemic triggers 25% increase in prevalence of anxiety and depression worldwide. https://www.who.int/news/item/02-03- 2022-covid-19-pandemic-triggers-25-increase-in-prevalence-of-anxiety-and-depression- worldwide REDUCING MISSED BEHAVIORAL HEALTH APPOINTMENTS 33 Appendix A PDSA Model Plan: Literature Review Develop interventions Develop Survey Questions Act: Adopt Adapt Abandon Do: Collect Data for appointments Call members 24 hours before appointment Hand out surveys Make Gratitude call 1 day after appointment Study: Review data Discuss what is learned Reflect on the barriers that exist doing implementation REDUCING MISSED BEHAVIORAL HEALTH APPOINTMENTS 34 Appendix B Review of Literature Citation Study description/aim Method Data source Sample Measurement Outcomes Strength/ Weakness Boyle & Schwinck, 2022 Quasi experimental/ Reduced missed appointment rates by improving client attendance behavior early in the treatment process. Ellis et al., 2022 RCT/ Simple intervention to schedule appointments later in the week to reduce Did Not Attend rates IBM SPSS software was used for data analysis. Independent sample t-tests and point biserial were performed to determine statistical significance. UK mental health hospital TOP intervention combined attendance policy review, psychoeducatio n, motivational interviewing techniques, and missed appointment rates were compared before and after TOP was initiated Data to compare attendance rates for 12- months before and 12-months after the intervention began. 9 5 individuals total 45 participants in each of the intervention and comparison group. Missed appointment rates were calculated by the number of missed appointments divided by the total number of scheduled appointments by each participant. 916 appointments in total Comparing attendance rates before and after the intervention. Completing TOP reduced missed appointments rate at 4,8, and 12 weeks (about 3 months) by 50%. Strengths: Level 2 evidence, low cost and moderate to large sample size. Weaknesses: non-random sampling. Bias limiting generalizability. Intervention and data collection was by the same researcher. Appointments at the beginning of the week were more likely to be missed than appointments at the end of the week. Strengths: Level 1 evidence, DNA (Did Not Attend) rate can be reduced by loading onto high attendance days. Weaknesses: Practical constraints from moving Monday appointments to Fridays. REDUCING MISSED BEHAVIORAL HEALTH APPOINTMENTS 35 Marbough et al., 2020 Retrospective study/ Understanding appointments rules, classification of patients Study that reviews current literature and investigates tools and methods to identify no- shows and causes Not reported Case study De-identifiable data from hospital setting. Milicevic et al., 2020 Study to model history of no-show and predict future appointment behavior De-identified administrative data from the Veterans Administration Corporate Data Warehouse 13 Veterans Administration Medical Centers 1,206,271 appointment records scheduled between January 2013 and February 28, 2017. serving inpatient and outpatient clinics retrospective data was analyzed. Predictive models to track, understand, and analyze factors to predict missed opportunities and to develop and implement strategies to reduce no- show rates. Strengths: Level 3 evidence, List of recommendations that can help to lower the rate of no-shows. Weaknesses: Lack of structed qualitative and quantitative data showing the factors that cause no-show appointments. No show rates are high due to multiple factors, including patient behavior, patients’ financial situation, scheduling, and environmental factors. Strengths: Findings on how the effects of varied factors showed increases in no-show rates, used data to predict the probability of no-show. Weaknesses: Lacks vital patient information. The type of study has not been identified. Level of evidence not identified. The best indicator for if someone is going to miss an appointment is their historical attendance behavior. Other factors were no-show rates over 2 years before current appointment and the probability of no show derived from the empirical up to the 10th order of the Markov model REDUCING MISSED BEHAVIORAL HEALTH APPOINTMENTS 36 Mohammadi et al., 2018 Using predictive modeling techniques to develop and compare appointment no-show prediction models Electronic health record data and appointment data including patient, provider, and clinical visit characteristics over a 3-year period. Urban system of community health centers with 10 facilities. 599,636 appointments by 76,453 unique patients Predictors of missed appointments versus attended appointments. Strengths: The study developed statistical model and machine learning models that can be used to predict patient’s chance of no-showing to next appointment Weaknesses: Study included patients from 1 CHC system with multiple geographic sites and study did not have information on clinical, physical, and functional status of patients The methodologies and findings of this study can be used to redesign scheduling systems in CHCs to reduce the rate of no-show appointments and no-show predictions models can be implemented in EHR (Electronic Health Records) systems that can help identify people with substantial risk of no-show appointment. REDUCING MISSED BEHAVIORAL HEALTH APPOINTMENTS 37 Oikonomidi et al., 2022 Rapid systematic Rapid systematic review of randomized control trials (RCTs) and non-RCTs. Medline, Cochrane CENTRAL, Embase, IEEE Xplore, and Clinical Trial Registries review/ Reviewing the effectiveness of predictive model- based interventions on outpatient no-shows, intervention costs, acceptability, and equity. Complete appraisal of the evidence using ROB 2, ROBINS-I, and GRADE tools. 7 RCTs and 1 non-RCT in dermatology, outpatient primary care, endoscopy, oncology, mental health, pneumology, and magnetic resonance imaging clinic. The rapid review found that predictive model-based text message reminders are effective at reducing outpatient no- shows, and phone call reminders, patient navigator calls are probable effective at reducing no- shows. Strengths: Level 1 evidence, the identification of effective predictive model-based interventions and evidence gaps can be used to guide future research and support current implementation of predictive modeling. Weaknesses: Studies may not generalize to low-income countries and pediatrics. More high-quality studies are needed for effectiveness of predictive model-based interventions. Opon et al., 2020 Systematic review/ review the effect of patient reminders in reducing missed appointment rates Google Scholar and PubMed PRISMA guidelines for systematic reviews using an extensive literature search/ Patient reminders/SMS reminders/ phone call reminders 20 articles met inclusion criteria Risk difference between missed appointments compared to those who attended their appointments. Strengths: Level I evidence, large sets of data from different studies. Weaknesses: Different study approaches may affect interpretation. Patient reminders reduced missed appointment rates and improved clinic attendance rates. REDUCING MISSED BEHAVIORAL HEALTH APPOINTMENTS 38 Ramlucken & Sibiya, 2018 Quantitative study/Aim of the study is to determine the frequency and reasoning for missed appointments of outpatient mental health patients. Quantitative surveys were used, and a non-probability convenient sampling method was used. A pre-testing survey was conducted on 8 participants who met inclusion criteria. SPSS version 23 for descriptive statistics. Reasons and frequency of missed appointments, 182 patients within four outpatient clinics in uMgungundlov u. Strengths: Level 3 evidence. Weaknesses: Sample size was small and reasoning for missed appointment varied between locations. A degree of bias due to the researcher being familiar with the outpatient clinics. Appointments were missed due to patients forgetting, no transportation, work commitment. 42% of respondents had to travel a distance to appointment, which could indicate financial implications. Findings also indicated age may not be a barrier to technology. REDUCING MISSED BEHAVIORAL HEALTH APPOINTMENTS 39 Teo et al., 2023 Cluster randomized control pragmatic trial to determine the effect of incorporating nudges into appointment reminder letters on measures of appointment attendance. VA medical center and its satellite clinics that were eligible for analysis. Pragmatic trial with broad eligibility criteria intervention implementation integrated with usual care, usual care as the comparison condition, and outcome assessment using the electronic health record data. 36,960 patients Logistic regression models adjusting for demographic Appointment reminder letters including behavioral nudges were ineffective in no-shows. Strengths: Level 1 evidence, large trial. The trial made physicians call patients if they did not show up for appointment time. Weaknesses: Patients being in multiple studies. There was also limited subjection to the study. Limitations on additional ways to send reminders. Ulloa-Pérez et al., 2022 Pragmatic Randomized study /To use risk prediction models to target reminders for missed visits. Randomized quality improvement project sending text message reminders 125,076 primary care visits and 33,593 mental health visits. Kaiser Permanente Washington with primary care patients and mental health visits. Comparing the results of one text reminder versus two text reminders on no-shows and same day cancellations. Findings indicate utilizing a predictive model to target reminders can reduce no- shows. Strengths: Level 1 evidence. The randomized study allowed for greater diversity in the sample. Weaknesses: Study was performed in a single health care system, may not be generalizable to other populations. The different barriers were not addressed to show why appointments were missed. REDUCING MISSED BEHAVIORAL HEALTH APPOINTMENTS 40 Valero-Bover et al., 2021 Reducing non- attendance in outpatient appointments/To develop a predictive model for assessing non-attendance and assessing the effectiveness of an intervention to reducing missed appointments based on the model. Three stage modeling that included model development, prospective validation of the model with new data, clinical assessment with a pilot study 33,329 appointments in dermatology and 21,050 in pneumology service. Scheduled appointments between January 1, 2015, to April 19, 2019, in the dermatology and pneumonology outpatient services at Hospital Municipal de Badalona (Spain) Bivariate analyses to identify relationships between the available variables and non-attendance and correlations between covariates to rule out strong interaction Strengths: Level 2 evidence, using decision trees to build predictive models, incorporated determinants to identify high- risk individuals. Weaknesses: The actions resulted in an excessive workload for staff, highlighting the necessity for organizational modifications. Interventions in the pilot study resulted in a significant decrease in non-attendance rate for dermatology and pneumology services. Attendance was lower in the intervention group. REDUCING MISSED BEHAVIORAL HEALTH APPOINTMENTS 41 Appendix C Synthesis Table EVIDENCE-BASED/NO-SHOW INTERVENTIONS TYPE OF REMINDERS Citation Predictive Modeling Lead Time Double Booking Behavior Modification Patient Navigator Email Text Message Call • • • • Boyle & Schwinck, 2022 Ellis et al., 2022 Marbouh et al., 2020 Milicevic et al., 2020 Mohammadi et al., 2018 Oikonomidi et al., 2022 Opon et al., 2020 Ramlucken & Sibiya, 2018 Teo et al., 2023 • • • • • • • • • • • • • • • • • • • • • REDUCING MISSED BEHAVIORAL HEALTH APPOINTMENTS Ulloa-Perez et al., 2022 Valero-Bover et al., 2022 • • • • 42 • REDUCING MISSED BEHAVIORAL HEALTH APPOINTMENTS 43 Appendix D SWOT Analysis Strengths Weaknesses ▪ QI Department and Supportive agency ▪ Variety services and patient to staff ratio ▪ Variety of health services ▪ Key indicators have not been measured ▪ Transportation ▪ Community and insurance resource assistance ▪ Strong community bond related to no-show ▪ The staff patients are involved with various tasks and responsibilities Opportunities Threats ▪ Geographic expansion ▪ Uncertainty in funding ▪ Expansion of telehealth ▪ Changes in policy ▪ Enhanced patient notification system ▪ Competition with local healthcare ▪ Public health exhibitions facilities REDUCING MISSED BEHAVIORAL HEALTH APPOINTMENTS 44 Appendix D Project Interventions 1. Obtaining patients' preferred mode of communication in the EHR 2. The project leader and support professionals will collaborate to communicate with 10 patients daily, 24 hours before their scheduled appointments. 3. Offer voluntary questionnaires to patients regarding possible barriers that may contribute to missed appointments. 4. Contact the patient following the visit and convey appreciation for attending the appointment. 5. Engage in collaboration with the project mentor and IT department to assess the viability of integrating an automatic expression of thanks for appointment attendance as well as an automated message for missed appointments and rescheduling into the EHR system, if compatibility exists. REDUCING MISSED BEHAVIORAL HEALTH APPOINTMENTS 45 Appendix E Survey Questions Survey Questions The following survey is conducted by the psychiatric mental health nurse practitioner student. The goal of the survey is to identify barriers that could impact missed appointments and help customer service. The survey is voluntarily at your choice. There are 5 questions in the survey. Please Circle One 1. Have you missed any behavioral health appointments in the last 6 months? Yes or No 2. Do you have trouble reaching a representative when calling the health center? Yes or No 3. Are you able to call and schedule an appointment within 0–2 weeks of calling? Yes or No 4. Did transportation affect your behavioral health appointment in the past six months? Yes or No 5. Have you missed doctor's appointments because of transportation costs? Yes or No Thank you for your time! REDUCING MISSED BEHAVIORAL HEALTH APPOINTMENTS 46 Appendix F Project Timeline DNP Timeline:Gantt ChartTaskActivityProgressStartDaysTask 1: Meeting with project mentor/QI team100%6/1/202320Task 2: DNP proposal plan100%7/1/202331Task 3: Completetion Of IHI QI classes100%6/4/202322Task 4: Squire 2.0 draft template100%6/1/202390Task 5: Intial Draft Revisions100%9/1/2023175Task 6:College of Nursing approval100%2/12/202414Task 7: IRB approval/Meeting with QI team100%2/24/202410Pre-implementation Task 1: Development of survey questions100%3/4/20242Task 2: Meeting with QI team100%3/5/20241Task 3: Survey questions100%3/7/20245Task 4: Implement PDSA cycle 1100%3/18/202320Task 5: Implement PDSA cycle 2100%4/22/202420Post-implementation/Task 1: Meet with QI team/Advisor100%5/17/20245Task 2: Review and organize data100%5/20/202413Task 3: Meet with statistician100%4/22/202416Task 4: Analyze data and interpret results100%5/25/202417Task 5:Meet with statistician to finalize 100%7/3/202473Task 6: Final revision/Powerpoint 100%8/16/202429Task 7: Presentation to the MSU CON100%9/16/20241 REDUCING MISSED BEHAVIORAL HEALTH APPOINTMENTS 47 Appendix G Letter of Approval REDUCING MISSED BEHAVIORAL HEALTH APPOINTMENTS 48 Appendix H Pre-intervention Appointment and No-show Pre Intervention Appointment and No-show s t n e i t a P 60 50 40 30 20 10 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 Day Appointment NoShow REDUCING MISSED BEHAVIORAL HEALTH APPOINTMENTS 49 Appendix I Behavioral Health Five-Questions Survey REDUCING MISSED BEHAVIORAL HEALTH APPOINTMENTS 50 Appendix J Survey Responses REDUCING MISSED BEHAVIORAL HEALTH APPOINTMENTS 51 Appendix K Post-Intervention Appointment and No-Show Rates Post-Intervention Appointment and No show s t n e i t a P 60 50 40 30 20 10 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 Day Appointment NoShow REDUCING MISSED BEHAVIORAL HEALTH APPOINTMENTS 52 Appendix L FQHC Comparison of Pre- and Post-intervention No-show Rates Comparison of Pre and Post-Intervention No-Show Rate Pre intervention No Show rate Post No show rate E T A R W O H S - O N 0.5 0.45 0.4 0.35 0.3 0.25 0.2 0.15 0.1 0.05 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 DAY