1 Running head: POST-COVID-19 PATIENTS A Quality Improvement Project for Post-COVID-19 Patients through Reminder Phone Calls Danielle Sweeney and Abrielle Wildman Michigan State University College of Nursing April 21, 2022 2 POST-COVID-19 PATIENTS Acknowledgements We would like to acknowledge the individuals who have assisted us during the creation and development of this project. This includes Michigan State University faculty Dr. Patrick Crane, our faculty advisor as well as our project course professors, Dr. Mary Smania and Dr. Kara Schrader. We would also like to thank the large Midwest healthcare organization staff, our community partner and lead physician of the COVID-19 recovery care program. We would like to acknowledge the work of Michigan State University doctoral nursing students, Mohamad Hussein and Mallory Radzik for their contributions to the preliminary stages of this project proposal. Lastly, we would like to thank our family and friends for their unwavering support throughout our doctoral program. 3 POST-COVID-19 PATIENTS Table of Contents Abstract............................................................................................................................................6 Introduction......................................................................................................................................7 Background and Significance..........................................................................................................7 Problem Statement/Clinical Question............................................................................................10 Organizational Assessment “Gap” Analysis” of Project Site........................................................11 Purpose of the Project....................................................................................................................11 Evidence Based Practice Model/QI Model....................................................................................12 Review of the Literature................................................................................................................13 Goals, Objectives, and Expected Outcomes..................................................................................19 Methods.........................................................................................................................................20 Project Site and Population................................................................................................20 Ethical Considerations/Protection of Human Subjects......................................................22 Setting Facilitators and Barriers........................................................................................24 The Intervention and Data Collection Procedure..............................................................26 Measurement Instruments/Tools.......................................................................................30 Analysis.........................................................................................................................................31 Sustainability Plan.........................................................................................................................33 Discussion/Implications for Nursing.............................................................................................34 Cost-Benefit Analysis/Budget.......................................................................................................35 4 POST-COVID-19 PATIENTS Conclusion.....................................................................................................................................36 References......................................................................................................................................39 Appendix A (PROMIS-29 Form)..................................................................................................43 Appendix B (Fishbone Diagram)...................................................................................................46 Appendix C (PDSA Diagram).......................................................................................................47 Appendix D (PRISMA Diagram)..................................................................................................48 Appendix E (Literature Table).......................................................................................................49 Appendix F (Themes Table)..........................................................................................................56 Appendix G (SWOT Analysis)......................................................................................................57 Appendix H (Pre-Intervention Survey).........................................................................................58 Appendix I (Post-Intervention Survey).........................................................................................60 Appendix J (Voicemail Script)......................................................................................................62 Appendix K (Non-Patient Script)..................................................................................................63 Appendix L (Patient Script)...........................................................................................................64 Appendix M (Data Excel Spreadsheet).........................................................................................65 Appendix N (Agency Letter).........................................................................................................66 Appendix O (GANTT Chart).........................................................................................................67 Appendix P (MSU IRB Letter)......................................................................................................71 Appendix Q (Large Midwest Healthcare Organization IBR Letter).............................................72 Appendix R (Completed Data Excel Spreadsheet)........................................................................73 5 POST-COVID-19 PATIENTS Appendix S (Results Bar Chart)....................................................................................................74 Appendix T (Project Budget Analysis)..........................................................................................75 6 POST-COVID-19 PATIENTS Abstract Background and Review of Literature: COVID-19 is a devastating illness that has detrimental health, financial and emotional consequences. Healthcare providers are utilizing questionnaires such as the PROMIS-29 form to help guide management of long term symptoms. Calling patients prior to their appointment as a reminder to complete their forms has proven to increase patient compliance with form completion. Purpose: The purpose of this quality improvement project was to increase the PROMIS-29 form response rate prior to the COVID-19 recovery care appointment, in hopes of improving care for patients living with post-COVID-19 conditions. Methods: This project was implemented at a large Midwest healthcare organization’s primary care clinics. The Doctor of Nursing Practice (DNP) students called patients within 72 hours prior to their appointment, to remind them to complete the PROMIS-29 form on their MyChart portal. Implementation Plan/Procedure: The DNP students completed HIPAA training and utilized a script provided by the large Midwest healthcare organization when calling patients. A weekly report was provided by the community partner that includes a list of patients who needed to be called, according to the “post COVID follow up” visit type. The DNP students traveled to the large Midwest healthcare organization’s administration building to conduct phone calls. Pre- intervention and post-intervention surveys were sent to the providers to determine the success of the intervention. Implications/Conclusion: Various project limitations impeded the ability to determine the success of the intervention however, the project proved helpful in determining that other reminder modalities may be of benefit to explore in the future. Keywords: Patient, Remind, Phone, Text, Email, Call, Form, Paperwork 7 POST-COVID-19 PATIENTS A Quality Improvement Project for Post-COVID-19 Patients through Reminder Phone Calls Coronavirus 2019 (COVID-19) is a devastating disease that was first discovered in December 2019, and continues to affect millions of people worldwide (Centers for Disease Control and Prevention [CDC], 2021a; CDC, 2022). The people of the United States (U.S.) have been severely burdened by this disease with over 80 million diagnosed, with Michigan ranking tenth highest in the country (CDC, 2022). The metro Detroit area accounts for the majority of Michigan’s COVID-19 cases (Suleyman et al., 2020). There is an increase in patients experiencing symptoms of COVID-19 for months after initial diagnosis (Greenhalgh, Knight, A’Court, Buxton, & Husain, 2020). These are considered long term effects of COVID-19 (Greenhalgh et al., 2020). Treatment for these patients can be difficult for providers to manage due to extensive and diverse symptoms (Greenhalgh et al., 2020). Healthcare providers are utilizing symptom assessment forms to help navigate and provide care for patients experiencing the long term effects of COVID-19. Increasing the utilization of these symptom assessment forms is beneficial for improving the care of patients suffering from post-COVID-19 symptoms. Background and Significance COVID-19 is a contagious respiratory illness caused by the virus, SAR-CoV-2 (CDC, 2021a). The first case emerged in China, December 2019 (CDC, 2021a). Not long after, the first U.S. case was discovered in the State of Washington in January 2020 (CDC, 2020). At the time of this writing, there have been 492,189,439 total confirmed cases worldwide with 6,159,474 deaths (World Health Organization [WHO], 2022). There have been 80,066,744 confirmed cases of COVID-19 in the U.S. and 980,220 confirmed deaths (CDC, 2022). Examining the COVID-19 pandemic on a local or state-wide scale, Michigan had the highest case fatality rate in the U.S. (Suleyman et al., 2020). More specifically, the city of Detroit 8 POST-COVID-19 PATIENTS and surrounding counties accounted for 80% of COVID-19 cases and 86% of deaths due to COVID-19 in the State of Michigan in 2020 (Suleyman et al., 2020). A cross sectional study from Angulo, Finelli, and Swerdlow (2021) suggested a gross underreporting of cases, increasing the potential long term effects of COVID-19 illness. COVID- 19 not only negatively impacts a patient’s health in the acute phase of the disease but has the potential to cause long term effects (Greenhalgh et al., 2020). About 10% of COVID-19 patients experience symptoms such as cough, fatigue, shortness of breath and brain fog beyond three weeks and some even suffer for months (Greenhalgh et al., 2020). Health Disparities As the rate of infection continues to rise in certain areas of the world and decrease in others, it is certain that COVID-19 has undoubtedly contributed to health inequities on both a global and local scale (Shadmi et al., 2020). This pandemic has disproportionately impacted individuals of lower socioeconomic, migrant and minority status (Shadmi et al., 2020). This population often has a history of chronic conditions, live in densely populated areas, experience food insecurities, and have lower paying jobs (Shadmi et al., 2020). Additionally, the areas in which these individuals work are often in the public sector, working in transportation or in grocery stores (Shadmi et al., 2020). Those who have migrated to the U.S. may face cultural and language barriers that pose a hurdle to understanding the preventative measures related to COVID-19 (Shadmi et al., 2020). All of these factors place vulnerable populations at an increased risk of contracting COVID-19 infection (Shadmi et al., 2020). One study conducted at the a hospital in Detroit, examined the characteristics and comorbidities of COVID-19 patients from March 9, 2020 until March 27, 2020 and found that 72.1% of COVID-19 patients were African American (Suleyman et al., 2020). These results are 9 POST-COVID-19 PATIENTS particularly concerning considering African Americans make up 14% of Michigan’s population, but still accounted for 32% of COVID-19 cases and 41% of deaths due to COVID-19 as of May 2020 (Suleyman et al., 2020). Social determinants of health that impact this population in Detroit include poverty, lack of health insurance coverage, and overcrowded housing which makes social distancing near impossible (Suleyman et al., 2020). Historically, a patient’s comorbidities or genetics have been used to explain health inequities in minority populations (Burton, Bennett, & Burton, 2020). However, it is imperative we examine how systemic racism and social determinants of health lead to chronic stress and ultimately comorbid conditions that place patients at an increased risk of COVID-19 complications (Burton et al., 2020). Long-Term Side Effects of COVID-19 Studies have shown that only 65% of COVID-19 patients have returned to their baseline health status 14-21 days after the diagnosis was confirmed (Greenhalgh et al., 2020). Various theories regarding why certain individuals are impacted more than others point to a weak antibody response, autoimmune conditions, and poor conditioning (Greenhalgh et al., 2020). It is equally concerning that providers may not yet know how best to manage this population or how to properly refer patients experiencing these symptoms (Greenhalgh et al., 2020). Unfortunately, evidence is limited and there is still much to learn regarding the long-term effects of COVID-19 (Greenhalgh et al., 2020). Until long term studies are completed and more is known about the long-term effects of COVID-19, this condition will continuously be evaluated in the primary care setting (Greenhalgh et al., 2020). This will require interdisciplinary support, reliance on patient provider communication, managing comorbidities and the treatment of symptoms (Greenhalgh et al., 2020). With there being such a large variation in symptoms experienced by these patients post- 10 POST-COVID-19 PATIENTS acute illness, it is clear that there is an increased need to assess their symptoms more frequently, to create an effective treatment plan (CDC, 2021b). Symptom Assessment of Post-Acute COVID-19 Symptoms As more patients are experiencing post-COVID-19 symptoms, providers have relied on tools such as the Patient Reported Outcomes Measurement Information System (PROMIS-29) form in order to assess patient symptoms and severity (see Appendix A for PROMIS-29 form). This form provides primary care providers information regarding the symptoms patients may be experiencing, mental or physical, while also providing information regarding the effect these symptoms have on patients’ everyday lives (Hays, Spritzer, Schalet, & Cella, 2018). Problem Statement/Clinical Question With the discovery of individuals experiencing post-COVID-19 conditions, there is an increased need for care of this population in the primary care setting. Patients are experiencing a multitude of symptoms with varying severity (CDC, 2021c). One form of assessment that can be better utilized by patients and providers for the evaluation of symptoms, treatment, and the progression of illness is the PROMIS-29 form (Hays et al., 2018). Using the PROMIS-29 form to assess the impact of patient’s symptoms is essential for providers in order to create a successful treatment plan and evaluate its impact (CDC, 2021b). The objective of this project is to focus on increasing patient provider communication, correctly assessing and managing post-COVID-19 symptoms, and ultimately improving quality of life. Increasing PROMIS-29 form adherence and utilization will help bridge the gap between an individuals’ manifestations of post-COVID-19 symptoms and communication needed for providers to create a mutually agreed upon care plan, in the hopes of recovery. 11 POST-COVID-19 PATIENTS Organizational Assessment “Gap” Analysis” of Project Site The large Midwest healthcare organization launched their COVID-19 recovery care program in April, 2021 (R. Eis, personal communication, June 13, 2021). When the program was launched the PROMIS-29 form was selected as a way to accurately assess symptoms while encouraging patient participation for better symptom management. One problem that has been a barrier for the providers and patients is the inconsistent utilization of the PROMIS-29 form. A fishbone gap analysis showed that the main issue is a low response rate of form completion prior to the patient appointment. The gap analysis also showed a lack of knowledge regarding the form. Finally, the gap analysis showed that there is no standardized policy regarding reminding patients of completing this form or education on its importance (see Appendix B for fishbone diagram). Improved response rates of completion of the PROMIS-29 form could likely benefit the COVID-19 recovery care program by improving patient-provider communications and care for patients suffering with long term symptoms of COVID-19. A standardized approach regarding the implementation of interventions to ensure patient completion and auditing of the PROMIS-29 form would improve response rates at the large Midwest healthcare organization. Purpose of the Project The purpose of this quality improvement project was to increase the response rate at which adult patients complete the PROMIS-29 form prior to their large Midwest healthcare organization COVID-19 recovery care appointment. This was accomplished through an evidenced based reminder modality such as calling patients prior to their appointment. Pre- intervention and post-intervention provider surveys responses were reviewed after the implementation phase. This intervention would allow providers time to review the patient’s 12 POST-COVID-19 PATIENTS results prior to their appointment, thus creating an effective treatment plan and reducing time associated costs. Ultimately, the aim of this project was to improve care for patients suffering from post-COVID-19 conditions in the primary care setting. Evidence Based Practice Model/QI Model Plan-Do-Study-Act The Plan-Do-Study-Act (PDSA) model created by the Institute for Healthcare Improvement (IHI) is a tool that can be utilized to test an intervention for change (Institute for Healthcare Improvement [IHI], 2021). The PDSA model is a vital component of quality improvement projects, as it can be used to determine how much change can be expected from the intervention, if the intervention will work in the environment it is designed for, and to evaluate the impact of the change (IHI, 2021). PDSA cycles should be continuously monitored and adjusted as needed, until the change is implemented on a larger scale (IHI, 2021). The PDSA model was utilized to guide this DNP project (see Appendix C for PDSA diagram). The components of the PDSA cycle provided the DNP students with the steps required to plan, implement and evaluate an intervention for post-COVID-19 patients. For the purposes of this project, the PDSA steps were followed with the goal of increasing the percentage of patients who fill out the PROMIS-29 form prior to their post-COVID-19 appointment. During the first planning phase, DNP students met with the community partner, physicians, various stakeholders, conducted a literature review and gathered demographic information about the patient population. During the second step of the PDSA cycle, DNP students called patients to remind them to fill out the PROMIS-29 form. The third step of this process involved assessing the effectiveness of the intervention and evaluating the pre and post-intervention provider survey responses. Finally, the fourth step involved discussing modifications to the intervention and 13 POST-COVID-19 PATIENTS developing a sustainability plan for the future. The methods section provides a comprehensive discussion regarding how the PDSA cycle was utilized to guide this project. Review of the Literature Search Strategy A comprehensive literature search was conducted on June 24th, 2021 using the Cumulative Index to Nursing and Allied Health Literature (CINAHL) database and the PubMed database (see Appendix D for PRISMA diagram). A variety of search terms and combinations were attempted before deciding on the most comprehensive search strategy. Search terms used included “(patient) AND (remind*) AND (phone OR text OR email OR call) AND (form OR paperwork)”. This combination of search terms provided 68 results from CINAHL and 137 results from PubMed. After filtering the search for articles written between 2016 and 2021, there were 37 from CINAHL and 77 from PubMed. After including only those articles written in English, 76 articles remained from PubMed and 37 remained from CINAHL. Of the combined 113 articles from both databases, 18 duplicates were removed, with 95 articles remaining. Of those 95 articles, 32 remained after reading through the title and abstract for appropriateness. Each article was read in its entirety to determine whether it would be applicable to the project. Eight articles then remained to be included and considered for the development of the intervention (see Appendix E for literature table). Inclusion criteria included those articles written between the years 2016 and 2021, written in English. Studies performed in all countries regardless of origin were also included. The search strategy included all levels of evidence including single qualitative studies up to systematic reviews and meta-analysis. Exclusion criteria included pediatric studies or those that focused primarily on female or male populations only. 14 POST-COVID-19 PATIENTS There was extremely limited literature regarding patient reminders and the PROMIS-29 form. Therefore, the literature search was focused on the impact of various patient reminder modalities to improve patient adherence to treatment plans, medication regimens and form completion. It is reasonable to infer that if a reminder strategy improved patient compliance with treatment adherence, then it can be used to increase the completion of the PROMIS-29 form by post-COVID-19 patients prior to their appointment. Following a full text review of the final eight articles, various themes were identified and synthesized into a table (see Appendix F for themes table). Those themes include the use of text message, email, phone call, and mailed letters as a reminder for patients, either alone or in combination with one another. Interventions Text message. Four of the articles (Amankwaa, Boateng, Quansah, Akuoko, & Evans, 2018; Brandzel et al., 2017; Coronado et al., 2018; Robotham, Satkunanathan, Reynolds, Stahl, & Wykes, 2016) utilized text messaging as an intervention to improve patient adherence and participation in their plan of care. Amankwaa et al. (2018) and Robotham et al. (2016) both evaluated the use of text messages as the sole intervention, while Coronada et al. (2018) and Brandzel et al. (2017) used text messages in combination with other interventions. Amankwaa et al. (2018) evaluated the use of mobile phone technologies in order to improve antiretroviral therapy (ART) adherence. The results of the study concluded scheduled text messages improved ART compliance among 18 years or older individuals who are human immunodeficiency virus (HIV) positive. The study concluded that mobile voice calls and triggered text messages did not represent a significant increase in adherence (Amankwaa et al., 2018). Similarly, Robotham et al. (2016) evaluated the use of text message reminders and patient appointment attendance. The study concluded adult 15 POST-COVID-19 PATIENTS patients who received reminder text messages are 23% more likely to attend their clinic appointments compared to patients who did not and were 25% less likely to miss appointments (Robotham et al., 2016). In contrast to the other studies, Coronado et al. (2018) evaluated the use of text messages in addition to email, phone call and mailed letters as interventions to improve patience compliance with fecal immunochemical testing (FIT). The study concluded 10% of patients who received a mailed letter returned their FIT sample within three weeks, 25.5% of the remaining patients returned the FIT after receiving a reminder. The reminder type varied and included a patient portal email reminder, automated phone call, live phone call, text message or a combination of a reminder letter with live call, automated and live call, and text message with live call (Coronado et al., 2018). Results revealed that live calls had better results when compared with written communication such as a letter or text (Coronado et al., 2018). There are several gaps in the literature identified within these articles. Although it was concluded that text messaging can be successful in improving patient adherence and participation, in articles Robotham et al. (2016) and Amankwaa et al. (2018), the frequency and quantity of messages is still uncertain. Robotham et al. (2016) concluded multiple text notifications increased the rate of patients attending appointments but did not decrease the number of missed appointments. Amankwaa et al. (2018) had differing results regarding text messaging frequency and quantity. Lastly, Coronado et al. (2018) identified a barrier to the text message intervention being that there is no way to ensure patients are receiving their reminders in text message form. 16 POST-COVID-19 PATIENTS Email. The use of email as a method of patient reminder was assessed by three of the articles (Pugh et al., 2021; Brandzel et al., 2017; Triplet et al., 2017). Pugh et al. (2021) and Triplet et al. (2017) both examined email exclusively while Brandzel et al. (2017) used email in conjunction with other interventions. Pugh et al. (2021) and Triplet et al. (2017) both evaluated the use of automated email reminders and its effect on completion and submission of patient-reported outcome measures (PROMs) forms and patient survey questionnaires. Both studies found that submissions of these forms and questionnaires were improved with automated email reminders when compared to those without. For example, Pugh et al. (2021) concluded that the PROMs form completion rate was 59.2% in one month after receiving email reminders compared to a 31.3% completion rate for those without reminders. Similarly, Triplet et al. (2017) found a 25.8% increase in survey completion rates for those patients who received email reminders. Each study discussed that this form of intervention allowed flexibility for patients to complete the forms on their own time, outside of the clinic, which is predicted to have a positive influence on the results. Phone call. Using telephone calls as a patient reminder modality was emphasized in five of the articles (Amankwaa et al., 2018; Brandzel et al., 2017; Coronado et al., 2018; Ivers et al., 2020; Zangalli et al., 2016). An identified gap in the literature exists in the fact that the phone call intervention was never studied independently. In each of the five articles, phone calls were used in combination with other reminders such as text message, email, or a mailed letter (Amankwaa et al., 2018; Brandzel et al., 2017; Coronado et al., 2018; Ivers et al., 2020; Zangalli et al., 2016). Four of the five studies were quantitative in nature and focused on a combination reminder 17 POST-COVID-19 PATIENTS intervention (Amankwaa et al., 2018; Coronado et al., 2018; Ivers et al., 2020; Zangalli et al., 2016). In contrast to the other four studies mentioned, Brandzel et al. (2017) conducted a qualitative focus group study which examined patient preference regarding cancer screening reminders. Patient responses were coded to determine which reminder modality was preferred (Brandzel et al., 2017). Most younger patients preferred text message or email reminders whereas older adults preferred a phone call to allow time for questions and to schedule an appointment (Brandzel et al., 2017). While this study may be a lower level of evidence, it is important to consider the utilization of phone call reminders in the older adult population (Brandzel et al., 2017). Brandzel et al. (2017) also found that patients preferred personalized reminders less than three months before the screening date. While Brandzel et al. (2017) recognized the impact age has on patient preference, the effect that this may have on the results is unknown, thus creating a gap in the literature. Mailed letter. Patient reminders in the form of mailed letters were studied in three articles (Coronado et al., 2018; Ivers et al., 2020; Zangalli et al., 2016). None of the studies included in this literature search examined using mailed letters as the only intervention, indicating a gap in the literature. More specifically, mailed letters were always used in combination with other reminders such as text message or phone call, making it difficult to discern if this intervention would be helpful independently. For example, Ivers et al. (2020) and Zangalli et al. (2016) both examined how the combination of phone calls and mailed letters as reminders, impacts adherence to treatment plans. Ivers et al. (2020) concluded that 37% of patients in the intervention group completed cardiac rehabilitation compared to the control group, of which only 27% completed the 18 POST-COVID-19 PATIENTS rehabilitation. These findings reflect those found in the Zangalli et al. (2016) article, which discovered the combination of reminder phone calls and mailed letters increased diabetic eye exam appointment attendance by 58%. In contrast to these two studies, Coronado et al. (2018) examined the use of mailed letters, phone calls and email reminders to improve FIT compliance. Summary Overall, this literature review illustrated that various reminder modalities can be effective at increasing patient compliance (Amankwaa et al., 2018; Brandzel et al., 2017; Coronado et al., 2018; Ivers et al., 2020; Pugh et al., 2021; Robotham et al., 2016; Triplet et al., 2017; Zangalli et al., 2016). Although various interventions were studied, either alone or in combination with others, it is evident that the utilization of phone calls and text messages are the most effective interventions (Amankwaa et al., 2018; Brandzel et al., 2017; Coronado et al., 2018; Ivers et al., 2020; Robotham et al., 2016; Zangalli et al., 2016). Robotham et al. (2016) and Amankwaa et al. (2018) were the only two level I studies included in this literature review and both demonstrated that text messaging was beneficial with improving patient adherence. Phone calls were also utilized in the majority of the studies and demonstrated a positive impact as reminder modalities (Amankwaa et al., 2018; Brandzel et al., 2017; Coronado et al., 2018; Ivers et al., 2020; Zangalli et al., 2016). However, unlike text messaging which proved to be successful as a single intervention by Robotham et al. (2016), phone calls were often studied as combination reminder strategies (Amankwaa et al., 2018; Brandzel et al., 2017; Coronado et al., 2018; Ivers et al., 2020; Zangalli et al., 2016). For these reasons, it is evident that current evidence supports reminder strategies such as text messages or phone calls as effective patient reminders (Amankwaa et al., 2018; Brandzel et al., 2017; Coronado et al., 2018; Ivers et al., 2020; Robotham et al., 2016; Zangalli et al., 2016). 19 POST-COVID-19 PATIENTS Due to the lack of evidence specifically regarding the use of reminders and completion of the PROMIS-29 form, the literature search was expanded to include the use of reminders for various patient situations. Both text message and phone call reminders have proven to be effective across different patient populations and outcome measures (Amankwaa et al., 2018; Brandzel et al., 2017; Coronado et al., 2018; Ivers et al., 2020; Robotham et al., 2016; Zangalli et al., 2016). Therefore, it is reasonable to conclude that phone call reminders could also improve the PROMIS-29 form response rate in post-COVID-19 patients. Goals, Objectives, and Expected Outcomes It was evident after discussing with the community partner, that a need exists for providing care to post-COVID-19 patients. One way to accomplish this is through the use of the PROMIS-29 form. If this form could be completed prior to the patient’s visit, it could be reviewed by the provider before the appointment and improve visit times. Through anecdotal data, via discussions with the lead physician of the COVID-19 recovery care program it was evident that essentially no patients were completing the PROMIS-29 form before their appointment. For this reason, the primary outcome of this project was to increase the patient's compliance with completing the form prior to their appointment. A secondary outcome was to improve provider efficiency when caring for these patients. In order to determine if the outcomes were met, data was collected using pre-intervention and post-intervention surveys. The following Specific, Measurable, Assignable, Realistic and Time specific (SMART) goals were developed in order to achieve the expected outcomes. 1. Increase the percentage of patients who fill out the PROMIS-29 form prior to their appointment from 0% to 30% by the end of December 2021. 20 POST-COVID-19 PATIENTS 2. DNP students called patients within 72 hours prior to their appointment as a reminder between November 2021-December 2021. 3. Improve provider efficiency as evidenced by comparing the provider’s pre-intervention survey responses to post-intervention survey responses by the end of December 2021. 4. Present the large Midwest healthcare organization with the quality improvement (QI) project results and sustainability plan by May 2022, with the goal of using this information to guide the large Midwest healthcare organization in developing a reminder policy in the future. Methods Project Site and Population The large Midwest healthcare organization’s COVID-19 recovery care program launched in April 2021. (R. Eis, personal communication, June 13, 2021) This program implemented in the primary care setting, focuses on the physical and emotional health of patients experiencing the long term effects of COVID-19 infection (R. Eis, personal communication, June 13, 2021). Its foundation aligns with the large Midwest healthcare organization’s mission statement of improving lives through science and the art of healing (Henry Ford Health System, 2021). The population being served by this program are adult patients, 18 years and older who have been diagnosed with COVID-19 and are suffering from long term effects of the virus (R. Eis, personal communication, June 13, 2021). Long term effects are considered continued symptoms including but not limited to dyspnea, brain fog, and fatigue for more than 12 weeks from the initial COVID-19 diagnosis, that cannot be attributed to an alternative diagnosis (R. Eis, personal communication, June 13, 2021). 21 POST-COVID-19 PATIENTS The purpose of the COVID-19 recovery care program is to assist primary care providers (PCP) with the management of long term COVID-19 symptoms (R. Eis, personal communication, June 13, 2021). The large Midwest healthcare organization has a variety of specialists available to help assist with managing patient care and are listed below. ● Physical therapy ● Occupational therapy ● Rehabilitation services ● Behavioral health ● Cardiology ● Neurology ● Neuropsychology ● Pulmonary ● Sleep disorders ● Social work ● Speech therapy This program is being developed alongside our community partner who is a registered nurse, as well as a project manager for the large Midwest healthcare organization (R. Eis, personal communication, June 13, 2021). The program is also being managed by a lead physician for care design and innovation of the project (R. Eis, personal communication, June 13, 2021). PROMIS-29 Form. The PROMIS-29 forms are patient self-reporting measures, that are used to assess the various aspects of a patient’s health including their mental health, physical health, and social determinants (Hays et al., 2018). The PROMIS-29 form is divided up into seven categories 22 POST-COVID-19 PATIENTS which include physical function, anxiety, depression, fatigue, sleep disturbance, the ability to participate in social roles and activities, and pain (Hays et al., 2018; PROMIS Health Organization [PHO], 2018). There are four questions related to each category (Hays et al., 2018). Lastly, the patient is able to rate the intensity of their pain (Hays et al., 2018). According to Hays et al. (2018), the PROMIS-29 form can be used to evaluate the impact of treatment and track the progression of illness over time (see Appendix A for PROMIS-29 form). The majority of patients are not completing the PROMIS-29 form prior to their visit, causing delays in care. This occurs because the provider must take time to review the 29 question form during the visit, thus taking up visit time that could be spent discussing the patient’s symptoms and developing a plan of care. These validated forms can be used by the PCP to objectively measure the patient’s symptoms and determine if they are progressing in their recovery. This was identified by our community partners as an area of opportunity to improve patient care outcomes (see Appendix G for SWOT analysis). Additionally, this is a new service that is still being fully developed and therefore, not all the Midwest healthcare organization’s PCPs are aware of the appropriate visit type to use (see Appendix G for SWOT analysis). For this reason, there are some patients experiencing long term symptoms of COVID-19 and are not benefiting from this service. Ethical Considerations/Protection of Human Subjects Michigan State University (MSU) Internal Review Board (IRB) and the large Midwest healthcare organization’s IRB approval was obtained prior to initiating the DNP project. The official IRB determination form was submitted upon proposal approval. The pre-intervention data regarding how many patients were using the COVID-19 recovery care services was de- identified, aggregated and not accessible to the DNP students. This information was only 23 POST-COVID-19 PATIENTS accessible to the community partner. Informed consent was obtained from providers completing the pre-intervention and post-intervention surveys (see Appendix H for pre-intervention survey, see Appendix I for post-interventions survey). This QI project was discussed with the large Midwest healthcare organization’s Nursing Professional Development and Education Administrator who submitted a request for EPIC and email access. The DNP students met with the healthcare organization’s Director, Deputy Information Privacy Officer who approved the project and allowed students to contact patients through phone calls. The scripts used to call patients were developed by and approved by the healthcare organization’s Director, Deputy Information Privacy Officer (see Appendix J for voicemail script, see Appendix K for non-patient script, see Appendix L for patient script). This script included the sentence the DNP students read to obtain verbal consent from the patient at the beginning of the phone call. The DNP students completed the Health Insurance Portability and Accountability Act (HIPAA) training and followed the large Midwest healthcare organization’s privacy policy. Access to large Midwest healthcare organization’s electronic medical record (EPIC) was given to students. The community partner obtained a weekly report of all patients under the specific “post COVID follow up” visit type from the large Midwest healthcare organization’s data analytics team. This report included patient’s names, medical record numbers, the date of their visit, location of their visit and provider. This report was sent via the large Midwest healthcare organization’ secure email to the DNP students to be utilized to call patients. The patient information was accessed and phone calls were made at the large Midwest healthcare organization’s administration building. The DNP students were trained on how to call patients, what information can be left on a voicemail and how to determine if the patient 24 POST-COVID-19 PATIENTS authorized another person to discuss their protected health information. Patient information was not accessed using the DNP student’s personal laptops and patient phone calls were not made from their personal phones. The excel spreadsheet used to gather data (see Appendix M for data excel spreadsheet) does not contain any patient identifying information and is stored on the student’s personal laptop. The excel spreadsheet was reviewed with the large Midwest healthcare organization’s Director Deputy Information Privacy Officer, who approved the use of the spreadsheet on the student’s personal laptops. The DNP students followed the large Midwest healthcare organization’s policies and upheld the standards of care. Setting Facilitators and Barriers This project’s setting was within the large Midwest healthcare organization’s outpatient primary care clinics. The population included adult patients, 18 years of age or older, with a history of COVID-19, who were experiencing long term effects of the virus and who were evaluated through the COVID-19 recovery service. The stakeholders involved in the implementation of the project included the lead physician of the COVID-19 recovery care program, the lead nurse and project manager of the COVID-19 recovery care program, information technology staff, the large Midwest healthcare organization’s compliance officer, Nursing Professional Development and Education Administrator, Director of Access Technology and Experience, the Senior Marketing Specialist and Department of Public Health Sciences assistant scientist. Each stakeholder provided their own expertise and perspectives to the development of this project. They were all instrumental in supporting the DNP students throughout each stage of the project. In order to better assess the strengths, weaknesses, opportunities, and threats (SWOT) of the project site, further analysis was conducted (see Appendix G for SWOT analysis). Barriers 25 POST-COVID-19 PATIENTS that posed a threat to the success of the project were the utilization of the new COVID-19 recovery care service by providers, lack of staff, lack of physician survey responses, time constraints of intervention and continuous evolution of the service. The implementation of the COVID-19 recovery care program is very new and many physicians were not aware of the service. Consequently, providers are still learning the workflow of the service, and how to use the proper visit type, which resulted in a small patient population to sample. Approximately 100 post-COVID-19 patients have been seen between April 2021 and August 2021 (R. Eis, personal communication, August 9, 2021). The lead physician and project manager are working to increase providers’ knowledge of the service, with the aim of gradually increasing the number of patients under this visit type. A staffing shortage resulted in an increased workload of the medical assistants. For this reason, the DNP students volunteered to implement the intervention themselves to avoid increasing the staff’s workload. The providers selected to complete the pre-intervention and post- intervention surveys were those that were using the “post COVID follow-up” visit type at the time of sending out the surveys. The physicians were given two weeks to respond with reminder emails sent out due to lack of response. After two weeks, the DNP students received one pre- intervention survey response. The lack of responses posed a barrier when the DNP students were evaluating the effectiveness of the intervention, which is described further in the analysis section. The DNP students conducted patient phone call reminders once a week on Tuesdays, within 72 hours of their scheduled appointment. Therefore, the patients whose appointments were outside of the 72 hour window were missed. For example, all patients with appointments scheduled on Mondays did not receive reminder phone calls. 26 POST-COVID-19 PATIENTS Lastly, because the COVID-19 recovery care program is new to the large Midwest healthcare organization, it is continuously changing and evolving with time. For this reason, the services' needs were continuously changing. The DNP students created their project centered around communication with the lead physician and project manager allowing for mutual consensus on the intervention to be implemented. Despite these barriers, the DNP students were supported by the community partners within the large Midwest healthcare organization as evidenced by the agency agreement letter (see Appendix N for agency letter). The Intervention and Data Collection Procedure The following paragraphs explain the PDSA cycle in detail and provide an explanation of the project’s timeline. A visual representation of the timeline is available in Appendix O. Plan. In preparation for implementing the intervention, the DNP students met with the community partner and lead physician of the COVID-19 recovery care program over zoom. During several meetings, the DNP students gathered information regarding the workflow process of the COVID-19 recovery care program and which patient population it serves. DNP students conducted a literature review and discussed those findings with the community partner, lead physician and MSU clinical faculty. Over the course of approximately four months, the DNP students met virtually with various stakeholders as discussed in the setting facilitators and barriers section. After meeting with the Director of Access Technology and their staff, it was evident that using text messaging as a reminder modality would not be possible due to the launch of a new text messaging service. It was difficult for DNP students to use this option as the timeline for implementation of the service was not guaranteed. After meeting with the Senior Marketing 27 POST-COVID-19 PATIENTS Specialist, it was clear that emailing specific patients as a reminder method would not be possible as the large Midwest healthcare organization emails patients in large groups and not based on a specific visit code. Due to the increased workload of the medical assistants and staffing issues, the lead physician of the program determined it would not be feasible to have the medical assistants call patients. All of these options including having the DNP students call patients, was discussed in detail over the course of multiple meetings with the Director Deputy Information Privacy Officer. The community partner gathered data regarding the number of patients that were evaluated in the clinic based on a specific “post COVID follow up” visit code. After this information was gathered, it was determined that the DNP students would call patients to remind them to fill out the PROMIS-29 form prior to their appointment. The large Midwest healthcare organization EPIC and email access was obtained for the DNP students to access patient information and email providers surveys by the large Midwest healthcare organization’s Nursing Professional Development and Education Administrator. The DNP students were provided with an office, computers and phone, located at the large Midwest healthcare organization administration building. The DNP students obtained a script to use when calling patients by the large Midwest healthcare organization’s Director Deputy Information Privacy Officer (see Appendix J for voicemail script, see Appendix K for non-patient script, see Appendix L for patient script). The DNP students completed iComply training that was required by the large Midwest healthcare organization, including HIPAA training. Additionally, the DNP students received education from the Director Deputy Information Privacy Officer in order to be compliant when calling patients. This education included information on the phone call script, voicemail protocols and privacy considerations. 28 POST-COVID-19 PATIENTS The DNP students met with the community partner, lead physician and MSU clinical faculty to determine what completion percentage would indicate a successful intervention. The DNP students developed a pre-intervention and post-intervention survey for providers to complete regarding their attitudes towards the PROMIS-29 form response rate and usefulness in guiding care (see Appendix H for pre-intervention survey, see Appendix I for post-intervention survey). Finally, the DNP students developed an excel spreadsheet that was used to keep track of various data. This data included whether the patient answered, if someone other than the patient answered, if a voicemail was left, and if they have portal access (see Appendix M for data excel spreadsheet). Do. The DNP students submitted the project proposal to MSU and the large Midwest healthcare organization’s IRBs where the project was determined to be non-human subjects research and exempted from IRB oversight (see Appendix P for MSU IRB letter, see Appendix Q for large Midwest healthcare organization’s IRB letter) After consulting with the large Midwest healthcare organization’s Director Deputy Information Privacy Officer and research administration department staff, it was determined that a data use agreement was not required and approval to begin the project was granted. This approval was granted by the large Midwest healthcare organization’s Director Deputy Information Privacy Officer, Nursing Professional Development and Education Administrator, research administration department staff and MSU clinical faculty. Prior to calling patients, the DNP students input both surveys into Qualtrics, a survey generating website, in order to email it to providers. The DNP students met with MSU’s Qualtrics team on numerous occasions in order to incorporate a random identification (ID) 29 POST-COVID-19 PATIENTS feature that would allow them to match pre-intervention survey responses to post-intervention survey responses. The community partner then obtained a weekly report of all patients under the specific “post COVID follow up” visit type. Starting in November 2021, the report was sent via the large Midwest healthcare organization’s secure email to the DNP students to be utilized to call patients. The report indicated if the patient had MyChart portal access. In the event the patient did not have MyChart access, the DNP students did not contact the patient. This report also included the PCPs that use the specific “post COVID follow up” visit type. The DNP students traveled to 1 Ford Place and emailed the pre-intervention survey on November 2nd, 2021 to 14 PCPs. The providers were given two weeks to complete the pre-intervention survey and a reminder email was sent one week after the survey was originally emailed as a reminder. The DNP students called patients, from the large Midwest healthcare organization’s administration building, within 72 hours of their appointment. The DNP students called patients for a total of six weeks, starting on November 16th, 2021 through December 21st, 2021. The process of calling patients began with using EPIC to view the media tab and determine if the patient had an authorization or representative form. These forms indicate whether patient information can be discussed with someone other than the patient, that the patient appointed as their representative. After consulting EPIC, the DNP students called patients and kept track of the phone call results on the provided spreadsheet (see Appendix M for data excel spreadsheet). After implementing the intervention, the post-intervention survey was emailed to providers on December 28th, 2021. No responses were received within the two weeks following the email and therefore, a reminder email was sent on January 11th, 2022. Due to continued lack of responses, a second reminder and final email was sent on January 20th, 2022, which was successful. Study. 30 POST-COVID-19 PATIENTS The DNP students collected the provider surveys and compared results from the pre- intervention survey to the post-intervention survey. The DNP students reviewed the data collected from the excel spreadsheet (see Appendix R for completed data excel spreadsheet). This information was used to compare the frequency of those patients who answered and those who were left a voicemail. Unfortunately, due to a lack of survey responses from providers, it was difficult for the DNP students to determine the effectiveness of the intervention. The results of the project and a discussion surrounding the barriers to this stage in the PDSA cycle are discussed further in the analysis section. Act. After implementing the intervention for six weeks, the DNP students analyzed the survey results and identified areas for improvement. Various solutions were discussed in order to address some of the barriers faced by the DNP students. These included provider education, training additional staff and implementing a texting reminder option. Ultimately, increasing awareness of the program would be very valuable as this would hopefully increase provider participation. As the long term effects of the pandemic continue to impact the world, programs such as the large Midwest healthcare organization’s COVID-19 recover care program will be required by patients. To support the longevity of this program, various modifications will need to be made. These modifications are discussed further in the sustainability plan section. Measurement Instruments/Tools In order to measure the outcomes of this DNP project, pre and post-intervention surveys were utilized (see Appendix H for pre-intervention survey, see Appendix I for post-intervention survey). The DNP students designed the surveys to gather meaningful qualitative data while being concise in order to improve response rates of the survey. Both surveys were converted into 31 POST-COVID-19 PATIENTS an electronic form via Qualtrics and the survey link was sent to the providers via the large Midwest healthcare organization’s email. The pre-intervention survey was sent prior to the implementation of the intervention to assess provider’s attitudes regarding the helpfulness of the PROMIS-29 form in managing patients with post-COVID-19 illness. The DNP students inquired about the provider’s perspective on how often the PROMIS-29 form was being completed prior to the visit. The pre-intervention and post-intervention surveys consisted of the same three questions and providers could choose from a list of available answers. None of the questions were required, if the provider chose not to answer a question, they were able to do so. In addition to the pre-intervention survey and post-intervention survey responses, the DNP students also gathered additional information through analysis of the excel spreadsheet data that was collected during the six week intervention process (see Appendix M for data excel spreadsheet). This data was used to determine how many patients answered the phone, if someone other than the patient answered and if a voicemail was left. This information was helpful in order to supplement the survey responses and guide the development of a sustainability plan. Analysis The goal of the DNP project was to increase the completion of the PROMIS-29 form prior to patient appointments using patient reminder phone calls. Over the course of six weeks, the DNP students called patients and tracked this information in an excel spreadsheet (see Appendix R for completed data excel spreadsheet). In the six weeks that the students made phone calls, a total of six patients were called. Of these six patients, one patient answered, four patients were left voicemails, and one patient’s voicemail box was full (see Appendix S for results bar chart). The DNP students exclusively called patients who were seeing providers that 32 POST-COVID-19 PATIENTS received a pre-intervention survey, for this reason there were two weeks in which no patients were called. It is important to note that the DNP students noticed, as the project evolved, more providers were using the “post COVID follow up” visit type. Unfortunately, these provider’s patients were not called as they did not receive the pre-intervention survey. To evaluate the success of the project, pre and post-intervention surveys were conducted. Only one provider completed both surveys. The pre-intervention survey results revealed that the provider believed patients were completing the PROMIS-29 form prior to their appointments 82% of the time. The post-intervention survey results revealed that the provider believed 76% of the time patients were completing the PROMIS-29 form prior to their appointments. In the pre and post-intervention survey, the provider found it very helpful to review the patient’s responses on the PROMIS-29 form prior to their appointment. Lastly, they strongly agreed that completion of the PROMIS-29 form prior to the patient visit improved care for patients with post-COVID-19 complications. When comparing these results, the provider’s perception was that less patients were completing the form despite receiving the reminder phone call. This does not indicate that the intervention was a success nor a failure. Due to the lack of survey response data, a statistical analysis was not possible and therefore a definite conclusion regarding the results could not be made. The surveys were designed to be anonymous but inadvertently one provider informed us that they completed the survey, making it no longer anonymous. For this reason, the DNP students were able to match which patient was called based on the provider who completed the survey. Therefore, the DNP students discovered that only one of the provider’s patients was called and a voicemail was left. This is important because there is no way of ensuring that the patient received the message, which may also have affected the results of the provider’s post- 33 POST-COVID-19 PATIENTS intervention survey responses. Additionally, it is important to consider that patients may not answer calls from unfamiliar numbers or check their voicemail prior to their appointment. These barriers made it difficult for the DNP students to be certain that patients received the message. For these reasons, texting should be considered as an alternate intervention as it eliminates some of the barriers that phone calls pose. More information on texting will be provided in the sustainability plan section. Due to the limited timeline of this project, the DNP students were not able to conduct patient reminder phone calls for a period greater than six weeks nor include more providers in the pre and post-intervention surveys. If this project were implemented in 2022 and beyond, there would have been more providers using the “post COVID follow up” visit type and thus more providers to survey. Additionally, it is recommended that patient reminder phone calls be conducted for longer than six weeks to reach more patients. As this is a new program, the DNP students believe the project may have had increased success if providers were more aware of the program and the purpose of the DNP project. Thereby, hopefully increasing provider participation in survey responses and thus improved data analysis. More information regarding these recommendations is available in the sustainability plan section. Sustainability Plan In the future, it would be beneficial to educate providers prior to implementing the intervention in order to increase awareness of the project and the COVID-19 recovery care program. This educational program could also assist with increasing the number of providers who use the “post COVID follow up” visit type thus increasing the number of patients available to call. The future implementation of this project into the large Midwest healthcare organization’s workflow is dependent on whether there is available staff to be hired and trained 34 POST-COVID-19 PATIENTS on the process of calling patients. Allocating staff to educate providers on the COVID-19 recovery care program would also be beneficial to increase provider participation. The DNP students also recommend that staff call patients two days per week in order to contact all possible patients. Additionally, it may be beneficial to discuss the possibility of texting patients after the Information Technology (IT) department’s new texting program is launched. This would eliminate the need to train and hire additional staff for making patient phone calls and eliminate the issues the DNP students faced with using phone calls as their selected reminder approach. Lastly, with future implementation, conducting the intervention for a longer period of time may be beneficial to the success of the project. Discussion/Implications for Nursing Unfortunately, the results of the project were not conclusive and thereby making it challenging to defer how practice would be impacted. With this being said, the COVID-19 recovery care program is newer to the large Midwest healthcare organization and therefore, the processes for which the program operates are also new. This project was beneficial to the providers and patients of the COVID-19 recovery care program in the sense that it illustrated that calling patients may not be the most effective way to remind patients to complete their PROMIS- 29 forms. Prior to the implementation of this DNP project, there was a gap in knowledge regarding how to increase patient participation in completing the form prior to their appointment. Using the results from the project, the large Midwest healthcare organization is able to infer that using other methods such as text messaging or email as reminders may be worth investigating or utilizing in the future. This would require additional costs as staff would need to be allocated to continue the project. However, if a reminder modality was found to be successful, this would 35 POST-COVID-19 PATIENTS prove to be cost effective to the large Midwest healthcare organization in the future, as less provider time would be spent reviewing patient’s answers during their appointment. Ultimately, this project illustrates the importance of discovering effective patient reminder modalities in order to improve patient outcomes. When a patient's PROMIS-29 form can be reviewed ahead of their appointment, providers are able to develop a more comprehensive plan of care. Additionally, more time could be spent during the visit answering the patient’s questions, completing a thorough assessment and discussing the treatment plan. As the pandemic continues on, it is assumed that more patients will be in need of the COVID-19 recovery care program services and therefore, it is essential to streamline processes that are currently in place. Recognizing the large Midwest healthcare organization does not currently have standard work or a guideline outlining how to remind patients to fill out their PROMIS-29 form prior to their appointment, this project serves as the first step to improving workflow. The large Midwest healthcare organization’s staff can utilize results of this project moving forward and potentially decide to implement a different reminder strategy to improve outcomes. Although the results of the DNP project were not conclusive, the implications for the future of nursing practice are still valuable. Cost-Benefit Analysis/Budget Cost efficiency was a priority during the development of this project. This QI project offered many cost-efficient benefits and limited excessive implementation costs. It was the DNP students’ goal that as more patients were completing the PROMIS-29 form prior to their appointment, providers would be able to better utilize their appointment time and focus on symptom management. This would be both beneficial to the patient and provider. This process appeared more time efficient by potentially allowing providers to see more patients in a day and 36 POST-COVID-19 PATIENTS reducing time spent at the clinic. From a financial standpoint, this would possibly allow providers to see more billable patients in a given day. It is also reasonable to consider that as more post-COVID-19 patients are managed in the outpatient setting, emergency room associated healthcare costs may possibly be reduced. The intervention was implemented by the DNP students using the large Midwest healthcare organization’s current resources, creating no additional costs to the large Midwest healthcare organization. According to the U.S. Bureau of Labor Statistics (2021), the mean hourly wage for registered nurses in Michigan is $38.47 per hour. The time taken to conduct patient phone call reminders averaged about 30 to 60 minutes depending on the number of patients scheduled. The DNP students called patients for a total of six weeks and sent surveys and survey reminder emails for four additional weeks. The reports given to the DNP students by the community partner are generated by the large Midwest healthcare organization’s data analytics department. The estimated time taken to generate a report is about 30 minutes to 60 minutes per report, for a total of approximately eight hours. The mean hourly wage for a data analyst at the large Midwest healthcare organization is estimated at $45.38 (Indeed, 2021). The DNP students traveled approximately 18.5 miles per trip to 1 Ford Place and the cost of transportation for DNP students was also included in the budget. The DNP students spent a total of 240 hours on this project. The total cost of this project was $18,828.64. For the purpose of the project, the DNP students donated their time. For more information, a detailed budget can be located in Appendix T. Conclusion As the pandemic continues to impact every aspect of our lives, there are many individuals who are still managing the long-term health effects of COVID-19. Greenhalgh et al. (2020) explains that patients may experience various respiratory, cardiac, gastrointestinal and neurologic 37 POST-COVID-19 PATIENTS symptoms months beyond the acute phase of the infection. Since COVID-19 is a novel virus, there is limited evidence regarding its lasting effects and how to manage this population (Greenhalgh et al., 2020). Despite these obstacles, the large Midwest healthcare organization has developed a COVID-19 recovery care program with the goal of improving care for patients experiencing long term effects of COVID-19. In order to effectively manage this population, the large Midwest healthcare organization is using the PROMIS-29 form to evaluate patient’s symptoms and treatment response. Following discussions with the community partner and stakeholders, it was determined that having the patients complete this form prior to their appointment would be beneficial. After a thorough literature review of the current evidence, it was evident that calling patients would be an effective reminder modality (Amankwaa et al., 2018; Brandzel et al., 2017; Coronado et al., 2018; Ivers et al., 2020; Zangalli et al., 2016). Through the use of the PDSA cycle, the DNP students developed an evidenced based DNP project for the large Midwest healthcare organization’s COVID-19 recovery care program. Over the course of six weeks, the DNP students called patients prior to their post-COVID-19 follow up appointment to remind them to complete the PROMIS-29 form. In order to determine the effectiveness of the intervention a pre and post-intervention survey was emailed to large Midwest healthcare organization’s providers. Unfortunately, due to the lack of survey responses received, there was not enough data to confidently determine whether or not the intervention was successful. Although it was not feasible to conduct a statistical analysis on the limited data available, there was a level of clinical significance in our findings which suggested that exploring other reminder modalities may be beneficial. In the future the large Midwest healthcare organization could consider texting patients to remind them to fill out the PROMIS-29 form prior to their appointment. In order to learn more about the long term trajectory of COVID- 38 POST-COVID-19 PATIENTS 19, more research and quality improvement projects will need to be conducted to effectively manage this patient population. 39 POST-COVID-19 PATIENTS References Amankwaa, I., Boateng, D., Quansah, D. Y., Akuoko, C. P., & Evans, C. (2018). Effectiveness of short message services and voice call interventions for antiretroviral therapy adherence and other outcomes: A systematic review and meta-analysis. PloS One, 13(9), e0204091, 1-20. doi:10.1371/journal.pone.0204091 Angulo, F. J., Finelli, L., & Swerdlow, D. L. (2021). Estimation of US SARS-CoV-2 infections, symptomatic infections, hospitalizations, and deaths using seroprevalence surveys. JAMA Network Open, 4(1), e2033706-e2033706. doi:10.1001/jamanetworkopen.2020.33706 Brandzel, S. D., Bowles, E., Wieneke, A., Bradford, S. C., Kimbel, K., Gao, H., & Buist, D. S. (2017). Cancer screening reminders: Addressing the spectrum of patient preferences. The Permanente Journal, 21, 17-051. doi:10.7812/TPP/17-051 Burton, E. C., Bennett, D., & Burton, L. M. (2020). COVID-19: Health disparities and social determinants of health. International Social Work, 63(6), 771-776. doi:10.1177/0020872820944985 Centers for Disease Control and Prevention. (2020). First travel-related case of 2019 novel Coronavirus detected in the United States. Retrieved from https://www.cdc.gov/media/releases/2020/p0121-novel-coronavirus-travel-case.html Centers for Disease Control and Prevention. (2021a). Basics of COVID-19. Retrieved from https://www.cdc.gov/coronavirus/2019-ncov/your-health/about-covid-19/basics-covid- 19.html 40 POST-COVID-19 PATIENTS Centers for Disease Control and Prevention (2021b). Evaluating and caring for patients with post-COVID conditions: Interim guidance. Retrieved from https://www.cdc.gov/coronavirus/2019-ncov/hcp/clinical-care/post-covid-index.html Centers for Disease Control and Prevention. (2021c). Post-COVID conditions: Information for healthcare providers. Retrieved from https://www.cdc.gov/coronavirus/2019-ncov/hcp/clinical-care/post-covid- conditions.html?CDC_AA_refVal=https%3A%2F%2Fwww.cdc.gov%2Fcoronavirus %2F2019-ncov%2Fhcp%2Fclinical-care%2Flate-sequelae.html Centers for Disease Control and Prevention (2022). COVID data tracker. Retrieved April 7, 2022, from https://covid.cdc.gov/covid-data-tracker/#datatracker-home Coronado, G. D., Rivelli, J. S., Fuoco, M. J., Vollmer, W. M., Petrik, A. F., Keast, E., . . . Jimenez, R. (2018). Effect of reminding patients to complete fecal immunochemical testing: A comparative effectiveness study of automated and live approaches. Journal of General Internal Medicine, 33(1), 72-78. doi:10.1007/s11606-017-4184-x Greenhalgh, T., Knight, M., A’Court, C., Buxton, M., & Husain, L. (2020). Management of post- acute covid-19 in primary care. British Medical Journal, 370(8258), 1-8. doi:10.1136/bmj.m3026 Hays, R. D., Spritzer, K. L., Schalet, B. D., & Cella, D. (2018). PROMIS®-29 v2.0 profile physical and mental health summary scores. Quality of Life Research: An International Journal of Quality of Life Aspects of Treatment, Care and Rehabilitation, 27(7), 1885- 1891. doi:10.1007/s11136-018-1842-3 Henry Ford Health System. (2021). Our mission, vision and values. Retrieved from https://www.henryford.com/about/culture/mission-vision-values 41 POST-COVID-19 PATIENTS Indeed. (2021). Henry Ford Health System data analyst yearly salaries in the United States. Retrieved from https://www.indeed.com/cmp/Henry-Ford-Health-System/salaries/Data- Analyst. Institute for Healthcare Improvement. (2021). How to improve. Retrieved July 14, 2021, from http://www.ihi.org/resources/Pages/HowtoImprove/ScienceofImprovementTestingChang es.aspx Ivers, N. M., Schwalm, J. D., Bouck, Z., McCready, T., Taljaard, M., Grace, S. L., . . . Grimshaw, J. M. (2020). Interventions supporting long term adherence and decreasing cardiovascular events after myocardial infarction (ISLAND): Pragmatic randomised controlled trial. BMJ (Clinical Research ed.), 369, m1731, 1-12. doi:10.1136/bmj.m1731 PROMIS Health Organization. (2018). Search & view measures. Retrieved from https://www.healthmeasures.net/search-view-measures?task=Search.search Pugh, S. L., Rodgers, J. P., Moughan, J., Bonanni, R., Boparai, J., Chen, R. C., . . . Bruner, D. W. (2021). Do reminder emails and past due notifications improve patient completion and institutional data submission for patient-reported outcome measures? Quality of Life Research: An International Journal of Quality of Life Aspects of Treatment, Care and Rehabilitation, 30(1), 81-89. doi:10.1007/s11136-020-02613-3 Robotham, D., Satkunanathan, S., Reynolds, J., Stahl, D., & Wykes, T. (2016). Using digital notifications to improve attendance in clinic: Systematic review and meta-analysis. BMJ Open, 6(10), e012116. doi:10.1136/bmjopen-2016-012116 Shadmi, E., Chen, Y., Dourado, I., Faran-Perach, I., Furler, J., Hangoma, P., . . . Willems, S. (2020). Healthy equity and COVID-19: Global perspectives. International Journal for Equity in Health, 19(104), 1-16. doi:10.1186/s12939-020-01218-z 42 POST-COVID-19 PATIENTS Suleyman, G., Fadel, R. A., Malette, K. M., Hammond, C., Abdulla, H., Entz, A., . . . Brar, I. (2020). Clinical characteristics and morbidity associated with coronavirus disease 2019 in a series of patients in metropolitan Detroit. JAMA Network Open, 3(6), 1-12. doi:10.1001/jamanetworkopen.2020.12270 Triplet, J. J., Momoh, E., Kurowicki, J., Villarroel, L. D., Law, T. Y., & Levy, J. C. (2017). E- mail reminders improve completion rates of patient-reported outcome measures. JSES open access, 1(1), 25-28. doi:10.1016/j.jses.2017.03.002 U.S. Bureau of Labor Statistics. (2021). Occupational employment and wage statistics. Retrieved from https://www.bls.gov/oes/current/oes291141.htm World Health Organization. (2022). World Health Organization (Covid) dashboard. Retrieved April 7, 2022, from https://covid19.who.int Zangalli, C. S., Murchison, A. P., Hale, N., Hark, L. A., Pizzi, L. T., Dai, Y., . . . Haller, J. A. (2016). An education- and telephone-based intervention to improve follow-up to vision care in patients with diabetes: A prospective, single-blinded, randomized trial. American Journal of Medical Quality: The Official Journal of the American College of Medical Quality, 31(2), 156-161. doi:10.1177/1062860614552670 43 POST-COVID-19 PATIENTS Appendix A PROMIS-29 Form 44 POST-COVID-19 PATIENTS 45 POST-COVID-19 PATIENTS From “PROMIS-29 Profile v2.1,” by PROMIS Health Organization, 2018 (https://www.healthmeasures.net/search-view-measures?task=Search.search). Copyright 2008- 2018 by PROMIS Health Organization. Reprinted with permission. 46 POST-COVID-19 PATIENTS Appendix B Fishbone Diagram 47 POST-COVID-19 PATIENTS Appendix C PDSA Cycle Plan - Meet with community stakeholders - Gather demographic data - Conduct literature search/review - Create surveys/excel spreadsheet for data collection Do - Obtain email/EPIC Act access - Identify areas of - Privacy education improvement - Travel to - Adjust intervention adminstrative building as needed to call patients - Develop - Email providers pre sustainability plan and post-intervention surveys Study - Collect survey responses - Analyze pre and post-interveniton survey responses - Determine effectiveness of intervention 48 POST-COVID-19 PATIENTS Appendix D PRISMA Diagram Identification Articles removed before screening: Articles identified from Duplicate articles removed (n = CINAHL(n = 68) 18) PubMed (n = 137) Articles removed for other reasons (n = 92)* Screening Records excluded based on Articles screened title/abstract (n = 95) (n = 63) Articles read for full review Articles excluded after full review (n = 32) (n = 24) Articles assessed for eligibility Articles excluded (n = 8) (n = 0) Included Qualitative articles included (n = 1) Total articles included in review (n = 8) Quantitative articles included (n = 7) * Reasons: must be written in English between 2016 and 2021 49 POST-COVID-19 PATIENTS Appendix E Literature Table Author/Title Level of Purpose of the Patient Results How does this Implications Evidence project/research population relate to your for Practice & project? Framewo rk Title: Do Level IV- The purpose of this Adult oncology Submission of the PROM form The results of this study Automatic email reminder Case study was to patients who are was significantly timelier in support the use of reminders can be emails and control determine if enrolled in an clinical trials with automatic automatic email utilized to past due automatic email NRG clinical email reminders as compared to reminders to improve improve patient notifications Framework reminders increased trial those trials without. For prostate submission of forms. compliance with improve - none the patient's cancer trials with automatic email However, this study did submitting patient completion, reminders, timely submission was not prove that forms necessary forms. completion submission and 79.7% in one month compared to were completed in their and timeliness of the 75.7% in trials without reminders. entirety due to the institutional patient-reported For breast cancer trials with email reminders. data outcome measures automatic email reminders, timely submission for (PROMs) form. This submission was 59.2% in one patient- form is completed by month compared 31.3% in trials reported a patient regarding without reminders. outcome their functional, measures? psychological and physical health which Author: is helpful for ongoing Stephanie L. oncology research. Pugh, Joseph P. Rodgers, Jennifer Moughan, Roseann Bonanni, Jaskaran Boparai, Ronald C. 50 POST-COVID-19 PATIENTS Chen, James J. Dignam, Deborah W. Bruner Title: Cancer Level VI- The purpose of this Women 21-75 Patient qualitative responses from One can infer that Investing in a Screening Qualitative study was to utilize years old and the focus groups were coded for giving patients the system that can Reminders: study patient focus groups men 50-75 years analysis. Patients preferred choice as to how they give patients the Addressing the to explore and old in personalized reminders less than are reminded of cancer opportunity to Spectrum of Framework understand patient’s Washington 3 months before the screening screening appointments choose their Patient - none perspectives on state. date. Most patients, especially can translate into preferred Preferences cancer screenings. younger patients preferred text patients submitting the reminder modality This study also message and email reminders. PROMIS tool prior to may increase Author: Susan sought to understand Older adults preferred phone call their appointment. It is patient D Brandzel, patient preferences versus mailed letter to allow time assumed that patients participation in Erin J Aiello regarding screening to ask questions and schedule prefer text or email care. Bowles, Arika reminders. their appointment. reminders. Wieneke; Susan Carol Bradford, Kilian Kimbel, Hongyuan Gao, Diana SM Buist Title: Using Level I: The purpose of this 21 studies were Patients who received text One can infer that if Utilizing text digital Systematic study was to included in the messages were 23% more likely text message reminders messages can notifications to review and determine the effect systematic to attend their clinic appointment improve patient improve patient improve meta- of text messaging review for a compared to patients who did not compliance with going adherence to attendance in analysis reminders on total of 8345 receive a text. Patients with text to their appointment clinic clinic: improving patient’s patients reminders were 25% less likely to then text messages can appointments and systematic Framework attendance at clinic receiving text not show for an appointment. be used to improve prevent no shows. review and - appointments. messages and Multiple notifications improved patient compliance with meta-analysis Cochrane’s 7731 patients adherence rates. completing the framework receiving no text PROMIS tool, Author: Dan message. especially if multiple Robotham, Patients were reminders are sent. Safarina primarily adults Satkunanathan and studies were , John conducted in Reynolds, Europe, Asia, Daniel Stahl, Africa, Australia 51 POST-COVID-19 PATIENTS Til Wykes and America. Title: Effect of Level II: The purpose of this 2772 adults Patients were all initially mailed a One can infer that if Utilizing patient Reminding Randomize study was to examine between the letter and FIT, of those patients patient reminders reminders such as Patients to d control the effectiveness of ages of 50–75 only 10% returned their FIT increase compliance live phone calls Complete trial different reminder that are not up to sample within 3 weeks of with FIT, then can increase Fecal modalities (text, date with receiving the letter. Of those who reminders will increase patient Immunochemi Framework email, phone call, colorectal cancer didn’t return the FIT, 25.5% compliance with compliance for cal Testing: A - none mailed letter) on screenings and returned the FIT after reminders completing the completing FIT. Comparative patient compliance located in the were sent. Return rates were PROMIS tool. It Effectiveness with fecal State of higher for those receiving a live appears reminders that Study of immunochemical Washington. phone call versus text message or include a live phone Automated testing (FIT). letter. call are more effective and Live than text messages Approaches. alone. Author: Gloria D. Coronado,, Jennifer S. Rivelli, Morgan J. Fuoco, William M. Vollmer, Amanda F. Petrik, Erin Keast, Sara Barker, Emily Topalanchik, Ricardo Jimenez Title: Level II: The purpose of the 9 of the 18 The patients were randomized The results of this study The use of mail- Interventions Randomize study was to evaluate cardiac centers into intervention groups. The support the out education supporting d control interventions in Ontario interventions included mail plus implementation of an brochures with the long term trial designed to improve participated in phone calls, mail only, and usual interventions of mail- use of phone calls adherence and long term adherence the study. The care which included outs in the addition of a can increase the decreasing Framework to the recommended population 2,632 communication which varied phone calls can adherence and cardiovascular - secondary prevention patients which between the hospital team, improve patient completion of events after Behavioral treatment after a included all primary care provider and adherence for cardiac rehab post myocardial theory myocardial infarction adults with a outpatient specialist when needed. completion of cardiac MI infarction valid provincial 27% of patients in the usual care rehab. One can infer 52 POST-COVID-19 PATIENTS (ISLAND): health card group, 32% of patients in the that implementation of pragmatic number, had a mailed letter only group and 37% these interventions can randomised coronary of patients in the mailed letter increase patient controlled trial angiogram post plus phone call group completed adherence to other STEMI or cardiac rehabilitation. aspects of their Author: Noah NSTEMI with healthcare M Ivers, Jon- evidence of David CAD and were Schwalm, discharged from Zachary the center after Bouck, Tara the procedure. McCready, Monica Talijaard, Sherry L Grace, Jennifer Cunningham, Beth Bosiak, Justin Presseau, Holly O Witteman, Neville Suskin, Harindra C Wijeysundera, Clare Atzema, R Sacha Bhatia, Madhu Natarajan, Jeremy M Grimshaw Title: Level I: The purpose of the The review 7 of the RCTs measure the effect The results of the study The use of Effectiveness Systematic review was to included 13 of SMS intervention and 4 showed that scheduled scheduled mobile of short review and determine when RCTs and 2 measures the effect of voice calls. mobile phone phone messaging message meta- including all mobile cohort and Mobile SMS intervention had a messaging improved can be utilized to services and analysis of phone-based quasi- significant effect on adherence to patient compliance with increase the voice call randomized interventions, whether experimental ART however, only scheduled ART. One can infer that adherence to ART interventions controlled the use of mobile studies in which SMS rather than triggered SMS. the use of mobile text 53 POST-COVID-19 PATIENTS for trials and phone technologies 11 RCTs were Mobile voice interventions had no messaging could antiretroviral quasi- can improve included in the significant association with increase patient therapy experiment antiretroviral therapy meta-analysis. increased adherence. compliance with filling adherence and al studies (ART) adherence The studies out the PROMIS tool. other included HIV outcomes: A Framework positive systematic - none individuals review and receiving ART meta-analysis majority being were >18 years Author: Isaac of age, gender, Amankwaa, or clinical state Daniel of HIV infection Boateng, Dan in a primary, Yedu outpatient, Quansah, community and Cynthia hospital care Pomaa setting. Akuoko, Catrin Evans, Javier R. Lama Title: E-mail Level IV: The purpose of the 186 patients There were two groups examined. The results of this study Implementation of reminders Retrospecti study was to evaluate who underwent The tablet group consisted of can be used to support email reminders improve ve Case whether email shoulder surgery patients who completed the the implementation of for post-operative completion Control reminders would during October tablet-based surveys in the office email reminders to shoulder surgery rates of study improve the 2012 to July during the planned follow-up visit increase the completion patients can patient- completion of the 2013 with and those patients who completed rates of the PROMIS increase the reported Framework Simple Shoulder Test preoperative the survey online but tool. completion rates outcome - none (SST) and 12-Item survey data subsequently scheduled their of patient- measures Short Form Health missed appointment. The reported outcome Survey questionnaires email group consisted of patients measure surveys Authors: Jacob after elective shoulder who completed surveys using the Triplet, Enesi surgery online portal following an email Momoh, reminder after a missed office Jennifer visit combined with the tablet- Kurowicki, only group. Results were Leonardo D. evaluated at 1 year and 2 year Villarroel, follow-up. There was a 25.8% Tsun Yee increase in survey completion rate Law, Jonathan for a complete data set which is 54 POST-COVID-19 PATIENTS C Levy preoperative, 1-year postoperative, and 2-year postoperative follow-up, for those in the email reminder group. Title: An Level II: The purpose of the 521 Diabetic Patients were randomized to the One can infer that if Personalized mail Education- and single- study was to evaluate adult patients 18 usual care group and an personalized mail and and phone Telephone- blinded, the effectiveness of years or older intervention group. The study phone communications communications Based randomized communication between April concluded that patients in the can improve DFE can improve the Intervention to controlled interventions on and October intervention group were more adherence than these adherence rates to Improve trial dilated fundus 2012, who were likely to schedule an appointment same interventions may DFE follow-up Follow-up to examination follow- previously seen than the usual care group. The improve adherence to rates Vision Care in Framework up adherence- in the general likelihood of attending completion of the Patients with - none specifically patients eye clinic and appointment was increased by PROMIS tool Diabetes: A with no, mild, or were due for a 58%. 24% of patients scheduled Prospective, moderate diabetic follow-up appointments after receiving the Single- retinopathy who are dilated fundus letter and brochure. 40% of the Blinded, less likely to adhere examine who patients on call list made Randomized had no, mild, or appointment after the first call. Trial moderate The study concluded that a diabetic combination of personalized mail Authors: retinopathy and phone communications are Camila S. effective in improving DFE Zangalli, Ann adherence than solely standard P. Murchison, mail reminders Nicole Hale, Lisa A. Hark, Laura T. Pizzi, Yang Dai, Benjamin E. Leiby, Julia A. Haller 55 POST-COVID-19 PATIENTS Appendix F Themes Table Article Intervention 1: Intervention 2: Intervention 3: Intervention 4: Text message Email Phone call Mailed Letter Article 1: Pugh X et al. (2021) Article 2: X X X Brandzel et al. (2017) Article 3: X Robotham et al. (2016) Article 4: X X X Coronado et al. (2018) Article 5: Ivers X X et al. (2020) Article 6: X X Amankwaa et al. (2018) Article 7: Triplet X et al. (2017) Article 8: X X Zangalli et al. (2016) 56 POST-COVID-19 PATIENTS Appendix G SWOT Analysis Weaknesses Strengths  Completing project within  Team is very responsive and allotted time timely with communicating  May struggle with lack of  Our group members are effective direction from team communicators  Coordinating scheduled  Group works well together, meetings with multiple completes tasks on time stakeholders  Willing to brainstorm different  Still learning the quality solutions to issues that arise improvement process  Strong stakeholder involvement  Determining how to measure the  The team is inviting, allowing us effectiveness of the intervention to be involved in may be challenging multidisciplinary meetings  Service is new and many  Resources (MSU faculty, Jessica providers are not aware of the Sender) are willing and available program yet to help guide our intervention Technological barriers Opportunities  Reduce health disparities related Threats to post COVID-19 long term  Potential lack of access to effects MyChart in order to complete  Improve the health outcomes of PROMIS-29 form citizens of Detroit and  Some patients may not have surrounding areas internet access  Decrease the number and  Patients may not answer their severity of post COVID-19 phone or return the phone call complications  Patient resistance towards filling  Improve the process for out another health form managing post COVID-19 patients 57 POST-COVID-19 PATIENTS Appendix H Pre-Intervention Survey This survey is being done to help us evaluate the effectiveness of completing the PROMIS-29 tool prior to the patient visit. We are asking you to complete a survey before we implement our intervention (calling patients to remind them to fill out the questionnaire). Our group would like to determine if we are increasing the rate of form completion prior to the patient appointment. Participation in this survey is voluntary and you may refuse to answer any question. You may withdraw or stop participating at any time without consequence. By completing the survey, you are indicating your voluntary agreement to participate. 1. What percentage of the time, is the PROMIS-29 tool completed by the patient, at home, prior to the patient visit? - Providers are able to indicate a percentage on a scale from 0-100% 2. In your opinion, how helpful is it to review the patient’s PROMIS-29 tool answers prior to the patient visit? a. Very helpful b. Helpful c. Somewhat helpful d. A little helpful e. Not helpful at all - If provider selects E, they will be asked to provide a response explaining their answer in a text box. 3. Indicate your opinion of the following statement: Completion of the PROMIS-29 tool prior to the patient visit improves care for those with post COVID-19 complications. a. Strongly agree b. Agree c. Neutral 58 POST-COVID-19 PATIENTS d. Disagree e. Strongly disagree - If provider selects E, they will be asked to provide a response explaining their answer in a text box. 59 POST-COVID-19 PATIENTS Appendix I Post-Intervention Survey This survey is being done to help us evaluate the effectiveness of completing the PROMIS-29 tool prior to the patient visit. We are asking you to complete a survey now that we have implemented our intervention (calling patients to remind them to fill out the questionnaire). Our group would like to determine if we increased the rate of form completion prior to the patient appointment. Participation in this survey is voluntary and you may refuse to answer any question. You may withdraw or stop participating at any time without consequence. By completing the survey, you are indicating your voluntary agreement to participate. 1. What percentage of the time, is the PROMIS-29 tool completed by the patient, at home, prior to the patient visit? - Providers are able to indicate a percentage on a scale from 0-100% 2. In your opinion, how helpful is it to review the patient’s PROMIS-29 tool answers prior to the patient visit? a. Very helpful b. Helpful c. Somewhat helpful d. A little helpful e. Not helpful at all - If provider selects E, they will be asked to provide a response explaining their answer in a text box. 3. Indicate your opinion of the following statement: Completion of the PROMIS-29 tool prior to the patient visit improves care for those with post COVID-19 complications. a. Strongly agree b. Agree c. Neutral 60 POST-COVID-19 PATIENTS d. Disagree e. Strongly disagree - If provider selects E, they will be asked to provide a response explaining their answer in a text box. 61 POST-COVID-19 PATIENTS Appendix J Voicemail Script “Hi, this is (insert name here) from (insert large Midwest healthcare organization’s name), please check your MyChart for a form that needs to be completed before your upcoming appointment.” 62 POST-COVID-19 PATIENTS Appendix K Non-Patient Script “Hi, this (insert name) from (insert large Midwest healthcare organization’s name), is (insert patient name) available to speak with me?” If a patient is not available: “Thank you, I will call them back at another time.” 63 POST-COVID-19 PATIENTS Appendix L Patient Script “Hi, this is (insert name) from (insert large Midwest healthcare organization’s name). I’m a nurse practitioner student from Michigan State University and I’m conducting a quality improvement project regarding completing forms prior to appointments. Is it okay if we discuss your upcoming appointment?” If patient answers yes: “Please complete the form in your MyChart portal titled PROMIS-29 prior your upcoming appointment on (insert date). It is helpful for your provider to review your answers prior to your appointment.” If patient answers no: “Thank you for your time.” 64 POST-COVID-19 PATIENTS Appendix M Data Excel Spreadsheet 65 POST-COVID-19 PATIENTS Appendix N Agency Letter 66 POST-COVID-19 PATIENTS Appendix O GANTT Chart 67 POST-COVID-19 PATIENTS 69 POST-COVID-19 PATIENTS 70 POST-COVID-19 PATIENTS Appendix P MSU IRB Determination Letter DETERMINED NOT “RESEARCH” Revised Common Rule September 14, 2021 To: Abby Wildman Re: MSU Study ID: STUDY00006639 Principal Investigator: Abby Wildman Determination Date: 9/14/2021 Title: DNP Project- A Quality Improvemen t Project for Post-COVID-19 Patien ts through Reminder Phone Calls The activity described in this submission was determined not to be “research” as defined by the Common Rule as codified in the U.S. Department of Health and Human Services (DHHS) regulations for the protection of huma n research subjects. Definition of Research For DHHS, “Research means a systematic investigation, including research development, testing, and evaluation, designed to develop or contribute to generalizable knowledge. Activities that meet this definition constitute research for purposes of this policy, whether or not they are conducted or supported under a program that is considered research for other purposes. For example, some demonstration and service programs may include research activities. For purposes of this part, the following activities are deemed not to be research: Office of (1) Scholarly and journalistic activities (e.g., oral history, journalism, biography, literary criticism, legal research, and historical scholarship), Regulatory Affairs including the collection and use of information, that focus directly on the Human Research specific individuals about whom the informa tion is collected. Protection Program (2) Public health surveillance activities, including the collection and testing of 4000 Collins Road information or biospecimens, conducted, supported, requested, ordered, Suite 136 Lansing, MI 48910 required, or authorized by a public health authority. Such activities are limited to those necessary to allow a public health authority to identify, monitor, 517-355-2180 assess, or investigate potential public health signals, onsets of disease Fax: 517-432-4503 outbreaks, or conditions of public health importance (including trends, signals, Email: irb@msu.edu www.hrpp.msu.edu risk factors, patterns in diseases, or increases in injuries from using consumer products). Such activities include those associated with providing timely situational awareness and priority setting during the course of an event or crisis that threatens pu blic health (including natural or man-made disasters). (3) Collection and analysis of information, biospecimens, or records by or for a criminal justice agency for activities authorized by law or court order solely for criminal justice or criminal investigative purposes. 71 POST-COVID-19 PATIENTS Appendix Q Large Midwest Healthcare Organization IRB Determination Letter Research Administration Henry Ford Health System 1 Ford Place – 2F Detroit, MI 48202-2689 (313) 874-4464 Office (313) 874-4288 NOT HUMAN SUBJECTSRESEARCH ACKNOWLEDGMENT To: Eunice Yu, MD Public Health Sciences From: Ellen Ryall Date: October 20, 2021 IRB No.: 15199 Title: A Quality Improvement Project for Post-COVID-19 Patients through Reminder Phone Calls The activities described by the Principal Investigator involve a quality improvement project focused on improving patient compliance with completing the PROMIS-29 form prior to their COVID-19 follow up appointment. As such, the activities do not meet the definition of human subjects research, as defined by the Common Rule and thereby this is determined not to be human subject research. The project also does not involve a test article (drug or device) and therefore, the FDA regulations do not apply. Therefore, this study does not require review by the Henry Ford Health Systems (HFHS) IRB. Please contact Pre-Award in the Research Administration Dept. to determine if any agreements will need to be executed for HFHS data to be shared with and/or sent to non-HFHS entities for this Quality Improvement Project. For assistance with agreements, Please contact: Crystal Moffett, Research Grant/Contract Analyst, (313) 874-4354, cmoffet2@hfhs.org NOTE: When publishing the results of a Program Evaluation/Quality Improvement/Quality Assurance project, the publication must clearly state that the activities involved were done for the purposes of Program Evaluation/Quality Improvement/Quality Assurance, not research. Please contact the IRB Administration Office at 313-874-4464 or ResearchAdmin@HFHS.org if you have any questions or concerns. 72 POST-COVID-19 PATIENTS Appendix R Completed Data Excel Spreadsheet 73 POST-COVID-19 PATIENTS Appendix S Results Bar Chart Completed Patient Reminder Phone Calls Answered Phone Calls 1 Unanswered Phone Calls 4 1 0 1 2 3 4 5 6 Voicemail Left Voicemail Unavailable Patient Answered 74 POST-COVID-19 PATIENTS Appendix T Project Budget Analysis May 2021 through April 2022 Expense Source Cost Total Personnel  DNP Student: Abby In-Kind 38.47/hour x 240 hours $9,232.80  DNP Student: Danielle In-Kind 38.47/hour x 240 hours $9,232.80  Data Analyst Direct 45.38/hour x 8 hours $363.04 Other Expenses  DNP Travel Costs In-Kind 370 mi/20mi. per gal. x 3.18/per gal. $58.83 Total $18,828.64