ANALYZING THE QUALITY OF PREHOSPITAL STROKE CARE IN MICHIGAN: LEVERAGING A STATE - WIDE STROKE REGISTRY TO QUANTIFY VARIATION IN EMS CARE AND IDENTIFY PRE HOSPITAL PERFORMANCE METRICS ASSOCIATED WITH OPTIMAL DOW N S T REAM CARE By John Adam Oostema A THESIS Submitted to Michigan State University in partial fulfillment of the requirements for the degree of Epidemiology Master of Science 2021 iv ABSTRACT ANALYZING THE QUALITY OF PREHOSPITAL STROKE CARE I N MICHIGAN: LEVERAGING A STATE - WIDE STROKE REGISTRY TO QUANTIFY VARIATION IN EMS CARE AND IDENTIFY PRE HOSPITAL PERFORMANCE METRICS ASSOCIATED WITH OPTIMAL DOWNSTREAM CARE By John Adam Oostema Introduction : Acute stroke is a debilitating condition responsible for substantial morbidity and mortality and for which the efficacy of acute treatment s is highly time dependent. As such, e mergency m edical s However, studies of EMS stroke care a re limited and suggest a high degree of variability in care. Methods : This analysis utilized linked data from a state - level EMS and the Michigan Acute Stroke Registry to audit EMS compliance with 6 performance measures derived from clinical guidelines , determine factors contributing to variability in compliance , and examine associations between EMS performance and stroke care in the Emergency Department (ED) . Results : Among 5 7 08 EMS - transported stroke cases transported between January 2018 and July 2019, compliance with EMS performance measures varied widely. EMS compliance was lower among p atients with subarachnoid hemorrhage, those with very low or high stroke severity, and those who presented later and there was substantial agency - level variability. In multivariable models, compliance with EMS performance measures was associated with earl ier CT acquisition in the ED . EMS stroke recognition and hospital prenotification were also associated with greater odds of receiving timely acute ischemic stroke tr eatment with alteplase. Conclusions : EMS compliance with recommended practices for stroke was variable in Michigan and is influenced by both patient - level and EMS agency - level factors. EMS compliance with performance measures was associated with more favo rable stroke care following hospital arrival. v Copyright by JOHN ADAM OOSTEMA 2021 iv ACKNOWLEDGEMENTS I would first like to thank the members of my thesis committee. Dr. Mathew Reeves has been a valued mentor and friend whose intelligence, good humor, and indefatigable pursuit of excellence are an inspiration. Dr. Michael Brown is perhaps the person most immediately responsible for my interest in research and in pursuing this degree. His humble leadership, expert instruction, and unwavering support of my academic career have been a true blessing. Dr. Zhehui Luo has been extremely generous in sharing her incredible s tatistical expertise and valuable time to help me attain greater analytic competence. Thank you all! I would also like to thank the faculty of Department of Epidemiology . I have benefitted greatly from the ir broad expertise and fantastic instruction ove r the course of my studies at MS U . Additionally, I wish to express my gratitude to the MSU CHM Department of Emergency Medicine and Emergency Care Specialists for their sustained support of my research career . Furthermore, I wish to acknowledge and tha nk Adrienne Nickles and the MOSAIC team for allowing me to assist them in their important work. Finally, to my wonderful family: thank you for your patience, love, and support. v TABLE OF CONTENTS LIST OF TABLES ................................ ................................ ................................ ....................... v i LIST OF FIGURES ................................ ................................ ................................ ................... v i i KEY TO ABBREVIATIONS ................................ ................................ ................................ ...... vi i i CHAPTER 1: INTRODUCTION ................................ ................................ ................................ ... 1 Scope of Problem ................................ ................................ ................................ ............ 1 Stroke Epidemiology ................................ ................................ ................................ ........ 1 New Opportunities for T reatment ................................ ................................ ..................... 2 The Role of EMS ................................ ................................ ................................ ............. 4 EMS Stroke Care: What Do We Know? ................................ ................................ ........... 5 Leveraging Existing Data System s ................................ ................................ .................. 9 Specific Aims ................................ ................................ ................................ ................. 10 CHAPTER 2: METHODS ................................ ................................ ................................ .......... 1 2 Study Design ................................ ................................ ................................ ................. 1 2 Data Sources and Analytic Population ................................ ................................ ........... 1 2 Statistical Methods ................................ ................................ ................................ ........ 14 Aim 1 : Patient - level EMS Performance Assessment ................................ .......... 14 Aim 2: Group Level Variation in EMS Compliance ................................ ............. 16 Aim 3: Associations Between Prehospital Performance and In - H ospital Stroke Care ................................ ................................ ................................ ................... 1 8 CHAPTER 3: RESULTS ................................ ................................ ................................ ........... 20 Analytic Population ................................ ................................ ................................ ........ 20 Aim 1 : Patient - level EMS Performance Assessment ................................ ...................... 2 3 Aim 2: Group Level Variation in EMS Compliance ................................ ......................... 2 7 Aim 3: Associations Between Prehospital Performance and In - H ospital Stroke Care .... 2 9 CHAPTER 4: DISCU S SION ................................ ................................ ................................ ...... 33 The Role of EMS Stroke Recognition ................................ ................................ ............ 41 Implications for EMS in Michigan ................................ ................................ ................... 42 Limitations ................................ ................................ ................................ ..................... 4 3 Conclusions ................................ ................................ ................................ ................... 4 5 APPEN D I CES ................................ ................................ ................................ .......................... 4 6 APPENDIX A: Data Elements and Sources ................................ ................................ ... 4 7 APPENDIX B: Sample Stata Programs ................................ ................................ ......... 4 8 APPENDIX C : Comparison of different random effects ................................ .................. 4 9 REFERENCES ................................ ................................ ................................ ......................... 50 vi LIST OF TABLES Table 1: Prehospital stroke performance measure definitions. For each measure, the compliance rate is calculated as the number of compliant cases divided by all EMS - transported stroke cases ................................ ................................ ................................ .............................. 14 Table 2: Characteristics of the 5708 EMS - transported Stroke Patients ................................ ..... 20 Table 3: Phi correlation coefficients for compliance between pairs of performance metrics across all 5708 EMS - transported stroke cases ................................ ................................ ......... 24 Table 4: Relationship between demographic and clinical characteristics and EMS performance measure compliance among 5708 EMS - transported stroke cases in unadjusted and adjusted ORs generated from multivariable crossed random effects logistic regression models .............. 25 Table 5: Group - level performance measure compliance across 292 unique agency - hospital pairs ................................ ................................ ................................ ................................ .......... 28 Table 6: Group - level performance measure compliance across 101 unique agency - hospital pairs with greater than 10 stroke transports ................................ ................................ .............. 29 Table 7 : Relationship between clinical and demographic variables and ED outcomes in unadjusted and adjusted (multivariable crossed random effects or single - level random effect) logistic regression models ................................ ................................ ................................ ......... 30 Table 8 : Associations between EMS quality meas ure compliance and ED outcomes derived from random effect logistic regression models ................................ ................................ .......... 32 Table 9: Comparison of logistic regression models for PSS compliance utilizing different random effects ................................ ................................ ................................ ................................ ....... 49 vii LIST OF FIGURES Figure 1: Locations of MOSAIC - participating hospitals in Michigan ................................ ........... 13 Figure 2: Bar graphs demonstrating the distribution of EMS - transported stroke cases across hospitals, EMS agencies, and unique EMS agency - destination hospital pairs .......................... 21 Figure 3: Percentage of all 5708 EMS - trans ported stroke cases with documented prehospital stroke performance metric compliance with 95% confidence intervals ................................ ...... 23 viii KEY TO ABBREVIATIONS CPSS Cincinnati Prehospital Stroke Scale CSC Comprehensive Stroke Center CT Computed Tomography DTCT Doo r - to - CT Time DTN Door - to - Needle Time ED Emergency Department EMR Electronic Medical Record EMS Emergency Medical Services EVT Endovascular Therapy FAST Face, Arm, Speech, Time ICC Intraclass Correlation Coefficient IQR Interquartile Range LAPSS Los Angeles Prehospital Stroke Screen LAMS Los Angeles Motor Scale LKW Last Known Well LVO Large Vessel Occlusion MI - EMSIS Michigan EMS Information System MOR Median Odds Ratio MOSAIC to Accelerate the Improvement of Care MT Mechanical Thrombectomy NEMSIS National EMS Information System NIHSS National Institutes of Health Stroke Severity OR Odds ratio OST On - Scene Time PCR Patient Care Report PSC Primary Stroke Center PSS Prehospital Stroke Screen ix SAH Subarachnoid Hemorrhage SD Standard Deviation TIA Transient Ischemic Attack 1 CHAPTER 1: INTRODUCTION Scope of Problem Acute stroke is a medical emergency that is responsible for significant morbidity and mortality worldwide . 1 Strokes are classified by etiology as either ischemic (approximately 85% of cases ) , which occur when arterial supply to cerebral tissue is interrupted by thrombus or embolus , or hemorrhagic ( approximately 15% of cases ) , which are caused by rupture of cerebral blood vessels. In the US , a stroke occurs about once every 40 seconds , accounting for 1 ou t of every 19 deaths (the 5 th leading cause) and leading to a substantial burden of disability and medical complications. 2 Direct and indirect costs attributable to stroke and its complications are estimated to exceed $45 billion annually. 2 Stroke Epidemiology Epidemiological studies have associated several risk factors with the incidence, morbidity, and mortality of stroke . While stroke may occur at any age, risk of stroke increases steadily for each decade of life. Individuals over the age of 85 represent 17% of all stroke s and older stroke patients have higher risk - adjusted mortality as well as greater disability following a stroke. 2 Females account for slightly more than half of all stroke cases in the US, however this is partially driven longer life expectancy of women as age - adjusted stroke risk is lower for females than males through middle age. 2 Differences in stroke incidence and outcomes have also been observed by race, with blacks experiencing higher age - adjusted stroke incidence and mortality than non - Hispanic whites, particularly among younger adults . 2 S troke incidence and belt , and c are delivered to stroke patients has been demonstrated to be highly variable by geographic region. 3, 4 2 New Opportunities for Treatment The most immediate goal in the tr eatment of acute stroke is correction of the underlying vascular defect ( occlusion or disruption ) . For both ischemic and hemorrhagic stroke, accurate diagnosis requires brain imaging as soon as possible. 5 Once identified, rapid reversal of therapeutic anticoagulation (e.g. warfarin) and aggressive blood pressure control may improve outcomes for hemorrhagic strokes . 6 For ischemic stroke, t reatment options have expanded significantly over the past three decades . The first breakthrough was in the utilization of recombinant tissue plasminogen activator ( alteplase ) to promote thrombolysis of clots in the cerebral vasculature, thus re - establishing perfusion of brain tissue and limiting permanent neuronal damage. T wo randomized trials published in a single manuscript in 1995 demonstrated reduced rates of disability at 90 - days following treatment of ischemic stroke patients who presented within 3 hours of the o nset of their symptoms with alteplase . 7 Based upon this data , the FDA approved alteplase for treatment of acute ischemic stroke in 1996. Over the ensuing decade, alteplase remained underutilized . 8 - 11 Barriers to treatment included skepticism regarding the risk/benefit ratio of the drug, 12 early reports of higher - than - expected rates of intracerebral hemorrhage, 13 and the complexity of identifying appropriate candidates and treating within the narrow therapeutic window. 14 Prompted in par t by the Institute of Medicine 15 report on suboptimal safety and quality provided by the US healthcare system , new systems for monitoring and improving performance were initiated t o address the gap between published recommendations and practice for ac ute stroke treatment . Stroke registries , including the Paul Coverdell National Acute Stroke Registry (PCNASR) and Ge t With the Guidelines - Stroke (GWTG - Stroke) , began to collect a n extensive set of variables describing the quality of care delivered to hosp italized stroke patients. 11, 16, 17 Quality measures were developed to encourage optimal stroke treatment , including a measure specific ally directed toward alteplase delivery . 18 N ational organizations such as the American Heart Association (AHA) began advocating for develo ping organized systems of stroke care, 3 including designation of specific hospitals as Stroke Centers , which would implement a systematic approach to improving stroke care across the continuum of care from pre - hospital to post - hospital settings . 19 To that end, the Joint Commission on the Accreditation of Healthcare organizations (JCAHO) began a formal certifi cation process for establishing Primary Stroke Centers in 2004 . Additionally, the AHA introduced Target: Stroke, a quality improvement network linked with its GWTG - Stroke database . The net effect of these efforts has been improvements in both the utilization of alteplase and in achieving faster treatment as measured by door - to - needle (DTN) times , especially in certified stroke centers ; nevertheless, reducing variability in quality of care continues to be a primary focus of stroke quality improveme nt programs . 20 - 23 Despite improvements in alteplase delivery , many ischemic stroke patients do not benefit from this therapy . Because alteplase treatment is not efficacious when administered beyond 4.5 hours from symptom onset , 24 a large proportion of ischemic stroke patients are excluded from this therapy due to delays in presentaiton . 9, 25 Even among candidates for alteplase , patient s with the most se vere ischemic strokes those with large cerebral vessel occlusion (LVO) benefit less from systemic thrombolysis compared to those with smaller strokes , and treatment carries higher risk of complications. 26 - 28 To address both shortcomings , there has long been interest in therapies that treat ischemic stroke directly at the site of vessel occlusion. Often referred to by the umbrella term endovascular treatment (EVT), su ch catheter - based therapies were already t he mainstay of acute myocardial infarction treatment . 29 Early studies of EVT modalities such as catheter - directed intra - arterial thrombolysis and mechanical thrombectomy (MT) were largely equivocal 30 - 32 until publication of the landmark MR CLEAN trial. 33 This trail, along with several others published at nearly the same time 34 demonstrated dramatic benefit for mechanical thrombectomy usi ng stent - retriever devices when candidates were selected on the basis of advanced brain imaging . In addition to achieving reductions in disability for qualifying patients, 4 these trials also substantially increased the window of time during which treatment can be administered, with later trials identifying some candidates benefitting from treatment as many as 24 hours form symptom onset. 35 While the benefits of MT are dramatic for those who qualify for treatment, both the imaging assessment required to determine candidacy and the specialists who provide the treatment are immediately accessible to a s little as one fifth of the US population . 36 Furthermore, although some LVO strokes may still receive benefit from MT despite delays in presentation, outcomes remain time - dependent 37, 38 and candidacy for MT falls steadily over time from symptom onset . 39 Furth ermore, it has been observed that when patients are initially treated in a facility without EVT capabilities, significant delays to MT occur due to the need to transfer patients to stroke centers capable of providing such care . 40, 41 This has amplified calls for stroke sys tems of care that seek to regionalize stroke treatment by directing patients to higher levels of care depending on the resources needed for treatment . 42 - 45 At the center of this - and - (Figure 1) of regionalized care delivery are Comprehensive Stroke Centers, which offer the broadest capabilities for diagnostic imaging and treatment. These hospitals are typically tertiary referral centers with around - the - clock capabilities for advanced stroke eva luations and treatment, including EVT, and are required to demonstrate a systematic approach to monitoring and wide variety of quality related performance measures. 46 The R ole of EMS As the first point of contact for more than half of patients with stroke, 47 EMS providers often have the earliest opportunity to activate a rapid stroke response. Compared to those that arrive by private vehicle, i schemic s troke patients are transported by EMS arrive in the ED earlier, 48, 49 undergo brain imaging faster, 48, 49 receive alteplase more frequently, 49 and achieve faster door - to - needle (DTN) times . 50 Observational studies have also suggested that hemorrhagic stroke patients transported by EMS receive definitive care earlier 50 and have lower 5 mortality. 51 Despite these benefits, only 59% of stroke patients activate EMS, and u tilization of EMS for stroke is flat over the past several years . 47 EMS systems have long been interested in optimizing prehospital care for many time - dependent emergencies, including stroke . 52 - 55 However , achieving this goal has been difficult. T he outlined three key barriers that hinder progress in delivery of high - quality prehospital care . 56 First, there is a historic paucity of dedicated EMS researchers and funding for EMS projects, resulting in a dearth of high - quality studies to inform care. 57 Second, EMS systems are highly fragmented, provided by a variety of entities (volunteer, municipal, private companies, hospital - owned) with diverse orga nizational structures that are often inadequately coordinated. 56 Fin ally, in order to assess the quality of EMS care, it is necessary to obtain hospital - based data to determine the final diagnosis and outcomes of patients who were transported by EMS. However, EMS and hospital records exist in silos with no consistent mech anism to allow for bilateral flow of information. 56 EMS Stroke Care: What Do We Know? T he literature describing the quality of EMS stroke care and its impact on stroke patients is limited compared to in - hospital care and consists primarily of observational analyses. This section summarizes studies that were identified by periodic searches of PubMed using the following MeSH terms: emergency medical services OR ambulances OR emergency medical technicians AND s troke . 53 However, relatively little has been published to characterize EMS provider stroke - specific knowledge. The largest single survey of prehospital knowledge was performed in 1999 by Crocco et al. Their findings indicated that while EMS providers were general ly familiar with symptoms of acute stroke, only about one third could correctly identify the time window for IV alteplase (3 hours at that time). 58 This is not surprising as IV alteplase was relati vely new (FDA 6 approved alteplase in 1996) and underutilized at that time, however a 2019 survey of paramedics in New York found that over 50% of respondents believed the treatment window for alteplase was longer than 4.5 hours. 59 Another recent st udy of EMS education prior to implementation of a state - wide LVO stroke protocol documented relatively low levels of knowledge regarding timing of interventions and the prehospital stroke severity scale (LAMS). 60 Beyond this, no studies directly addressed EMS knowledge regarding stroke protocols, quality meas ures, or knowledge regarding the potential impact of EMS on stroke outcomes. Stroke guidelines have outlined several best practices for EMS stroke care that represent potential quality metrics. 61 - 63 These generally arise from three over - arching goals of EMS care: early recognition of stroke, rapid transport to an appropriate stroke center, and prehospital activation of ED stroke response. Since hypoglycemia is a well - known stroke mimic that is treatable in the prehospital setting, obtaining a point - of - care glucose test is recommended. 64 To address difficulties in recognizing stroke due to the heterogeneous nature of stroke clinical presentations, 61 - 63 stroke screening tool s have been developed and validated to assist EMS providers in recognizing stroke symptoms. 62 Documentation of a validated stroke scale has therefore been a longstanding guideline recommendation to maximize EMS identification of stroke symptoms. 65 To enhance efficiency of transport, recommendations also encourage limiting on - scene time to 15 minutes. 45, 64, 65 Determination of the time at which p atients were last known to be well (LKW) is a recommendation intended to assist hospitals in making decisions regarding alteplase eligibility. Finally, hospital prenotification by EMS is recommended as it has been demonstrated to facilitate stroke treatme nt following ED in several studies. 66 - 72 For example, in an analysis of over 370,000 EMS transported stroke patients in the GWTG - Stroke database revealed that prenotification was associated with increased alteplase delivery, faster CT scan acquisition, and earlier alteplase administration. 67 Despite consistent endorsement of these practices in clinical guidelines, 73 - 75 the national EMS model protocol, 76 and incorporation into local EMS p rotocols, 77 - 79 the small number of 7 available published assessments of EMS performance suggest significant variability. A large study analyzing data from the national GWTG - Stroke registry found that about one third of EMS transported stroke cases did not receive hospital prenotification with substantial regional variation (0 - 100%). 80 On - scene times (OST) were also evaluated using the National EMS Information System database and concluded that less than half of stroke transports met the criterion of leaving the scene within 15 minutes of a rrival. 81 A study conducted in Rhode Isl and also noted modest stroke scale documentation compliance (56%) and less than 50% compliance with LKW documentation and prenotification prior to implementation of a feedback program for EMS. 82 Furthermore, no systematic analysis has been conducted to examine the degree of variability in stroke care performance between different EMS agencies. These findings demonstrate the need for a systematic approach to EMS stroke care monitoring and improvement, which could be facilitated by valid EMS performance measures. Tr anslation of clinical guidelines into functional performance measures has been a slow process . C urrently, documentation of a PSS is the only stroke - specific quality metric endorsed by the National EMS Quality Alliance, 83 a non - governmental organization tasked by the Na tional Highway Traffic and Safety Administration (NHTSA) with developing EMS quality measures. The lack of clearly defined, consistent EMS performance metrics is not confined to stroke care as is evidenced by the findings of three recent systematic revie ws of available prehospital quality indicators . 61 - 63 After reviewing the literature describing quality measures for EMS care, these analyses found that the majority of quality metrics are derived by consensus rather than derivation from performance data 61 and target efficiency - oriented process measures such as response time intervals 61 - 63 or deployment of appropriate resources. 62 While the largest number of quality indicators have been developed for out of hospital cardiac arrest, stroke was consistently included among measures that target specific diseas es , 61, 62 highlighting that EMS agencies recognize this as a high - priority condition for quality improvement efforts . 8 In pilot work conducted in West Michigan, we documented moderately high rates of compliance with stroke scale documentation (79%) and glucose check (86%), but lower rates of compliance with LKW documentation (68%), scene time goals (47% less than 15 minutes) and hospital prenotification (57%). 84 Not surprisingly, we found that stroke scale documentation was closely related to EMS recognition (94% had stroke screen among recognized vs. 31% among non - recognized cas es). This relationship is intuitive and been noted elsewhere as well, 85 - 88 however we also noted that compliance with OST, LKW documentation, and prenotification rates were all higher among EMS recognized stroke cases , suggest ing that recognition of stroke may be associated with superior EMS performance overall . The benefits of EMS recognition of stroke appear to extend beyond prehospital compliance. Several studies have established strong associations between EMS recogniti on of stroke and favorable evaluation and alteplase delivery times . 68, 86, 89, 90 However, EMS recognition of stroke has also been shown to be inconsistent, with one quarter to one half of EMS transported stroke patients unrecognized as such in the prehospital setting . 85, 86, 91 - 95 suggesting that monitoring EMS stroke recognition rates may be another important marker of quality prehospital care. A n assortment of educational interventions 96 - 98 and feedback programs 82, 98 targeting EMS stroke recognition and performance have demonstrated potential to improve prehospit al stroke care . These studies have generally been successful in improving EMS stroke recognition, 98 compliance with prehospital stroke quality measures , 82 and door - to - needle times. Unfortunately, the long - term impact of these interventions is unce rtain . 98, 99 Taken together, this literature suggests that ongoing monitoring and reinforcement of appropriate prehospital stroke care may improve outcomes . However , performance metrics that are easily measured, reliably associated with favorable outcomes, and modifiable through education and feedback programs are needed. Furthermore, sources of variability in EMS performance have been largely 9 unexplored. To date , no comprehensive assessment of EMS performance metrics derived from real - world, state - level EMS data has been performed. Leveraging Existing Data Systems Large scale evaluations of EMS performance require both prehospital and in - hospital data from a wide variety of geographic locations, EMS systems, and hospitals. For EMS care, the need for a systematic and comprehensive data collection system has been recognized since the 100 Initially targeted toward out - of - hospital cardiac arrest, the Utstein collaboration standardize d definitions of clinical terms to allow comparisons of resuscitation attempts across various regions and organizations. 101 This approach ultimately lead to the development of a uniform repor ting structure for data describing all prehospital care. 102 W ith funding provided by NHTSA and Centers for Disease C ont rol and Prevention , a centralized electronic data collection system was developed that ultimately became T he N ational EMS Information System (NEMSIS) . 100, 102 Over the past two decades, NEMSIS participation has expanded to all 50 states and now collects over 400 standardized data elements derived from E MS records describing patient demographics, response times, interventions delivered, and clinical impressions , 102 a subs et of which are directly relevant to stroke. 103 State level EMS data have been used in Utah and North Carolina to develop quality improvement programs directed toward prehospital stroke ca re , 104 - 106 however it is not clear how comparable the quality of data is across states . All licensed EMS agencies in Michigan are currently required to participate in data reporting to em (MI - EMSIS) , which now receives data for over 1.8 million EMS transports per year . While the volume of data collected is large, a r ecent review discovered relatively high rates of missingness across several variables . 107 This was thought to arise from a combination of insufficient data entry at the point of care as well as data mapping issues between EMS agency electronic 10 records and MI - EMSIS. 107 To date, MI - EMSIS data has not been used systematically to assess prehospital stroke ca re . The Paul Coverdell National Acute Stroke Registry (PCNASR) was developed through the Centers for Disease Control and Prevention to improve care delivered to patients hospitalized for acute strok e in the US. 16 The State of Michigan has participated in the Accelerate Improvements in Care (MOSAIC), this registry contains a wealth of information describing in - hospital stroke car e in Michigan. During the period from January 2018 to July 2019 , 38 hospitals participate d in the registry, providing records for over 25,000 stroke admissions per year (over half of all stroke admissions in Michigan). However, it contains very little da ta describing care delivered by EMS. In order to gain a better understanding of care delivered to acute stroke patients fr o m initial 9 - 1 - 1 contact to hospitalization, I recently developed a probabilistic matching process to link these two databases. 108 About two thirds of all stroke records in MOSAIC that were coded as having arrived by EMS were successfully matched to an EMS record using this process. Furthermore, these matched EMS - transported stroke cases were similar to unmatched cases across all d emographic and clinical variables, suggesting that this matched sample is representative of the underlying population of EMS - transported stroke cases in Michigan. 108 This matched dataset offers a unique opportunity to perform a comprehensive state - wide assessment of EMS stroke care performance in Michigan, as well as examine the relationship between EMS performance and downstream care. Specific Aims Building on work I have previously done, this thesis address ed th ree specific aims related to characterizing the quality and impact of prehospital stroke care in Michigan. 11 Aim One: Patient - level EMS Performance Assessment : The first aim of this analysis was to assess the quality of EMS stroke care in Michigan and ident ify sources of variation in EMS care. To assess quality of care, compliance with 6 prehospital stroke performance measures ( performance of a validated prehospital stroke screen, obtaining a point of care glucose test, recognizing stroke, documenting the L KW time, maintaining an on - scene time of 15 minutes or less, and hospital prenotification ) was quantified for all EMS - transported stroke cases in the matched MOS AI C - MI - EMSIS dataset . B ecause of marked clinical variability of stroke presentations, as well as the diverse populations, practice settings, and EMS delivery systems across Michigan, an analysis was performed to identify patient - level factors associated with EMS performance measure compliance . Aim 2: Group - Level Variation in EMS Compliance : Because of the wide variation in practice settings and EMS systems across Michigan, an analysis of agency - and destination hospital - level effects on EMS compliance was also performed . Agency - hospital pair - specific compliance rates were calcula ted for each EMS performance measure and the relative contribution of between agency and between hospital variation to overall variation in compliance was quantified . Aim 3: Associations Between EMS Performance and In - H ospital Stroke Care : The third aim o f this analysis was to determine the extent to which compliance with each of the 6 EMS performance measures predict ed favorable ED stroke care after hospital arrival . An ideal prehospital quality measures would independently and favorably impact ED stroke evaluation times and, in the case of ischemic stroke, increase the likelihood and speed of reperfusion therapy. This analysis sought to quantify the independent associat ion between each of the 6 EMS performance measures and faster evaluation as measured by DTCT times, alteplase delivery for patients with ischemic stroke, and DTN times for those receiving alteplase. 12 CHAPTER 2: METHODS Study Design This i s a retrospec tive analysis of a cohort of stroke patients who were transported by EMS to a hospital participating in MOSAIC, , between January 2018 and July 2019 . Data Sources and Analytic Population As noted above, MI - EMSIS is a registry that receives data from all licensed EMS entities in the state of Michigan. Housed within the Bureau of EMS , Trauma , and Preparedness in - EMSIS collects over 400 standardized data elements describing EMS care. These include patient demographics, nature of complaint, response times, location, destination hospital, and information describing clinical assessments and interventions undertaken by EMS providers . These data are derived from EMS patient care reports (PCR) , which are directly uploaded from EMS agency - level electronic medical records (EMR) using a web - based platform (ImageTrend®, Lakeville, MN) . This platform has developed software bridges to various preho spital EMR vendors to facilitate standardized data reporting. EMS agencies are required to upload PCR data to MI - EMSIS for all calls by the 15 th day of the following month. - level branch of the PCNASR . MOSAIC data is abstracted directly from hospital records using the GWT G - Stroke Patient Management Tool by trained staff at voluntarily participating hospitals across Michigan. A map of currently participating hospitals is provided in Figure 1. During the analytic period, 38 hospitals participated in the registry and contributed data for over 25,000 individuals, representing over half of all stroke cases that occurred in Michigan during that time . Data elements include patient demographics, arrival mode, clinical characterist ics describing the type and severity of stroke, stroke evaluation and 13 treatment data, complications, and discharge information. The only data specific to EMS performance recorded in MOSAIC during the analytic period was a field for hospital receipt of EMS prenotification. Data in MOSAIC is routinely audited and period education of abstractors is performed. Previous analyses of the GWTG - Stroke abstraction process 109 and MOSAIC have demonstrated high rates of completeness and accuracy of data. 110 F igure 1: Locations of MOSAIC - participating hospitals A s previously described, the population under analysis for this thesis was assembled by probabilistically matching MOSAIC stroke patients who arrived by EMS to records from MI - EMSIS , resulting in a cohort of EMS - transported hospital - confirmed stroke pati ents . 108 For this analysis p atients with DTCT of less than 0 minutes or greater than 360 minutes following hospital arrival were excluded. A list of variables used in this analysis and their sources is provided in the Appendix. 14 Descriptive statistics were used to characterize the demographic and clinical characteristics of study population using proportions with 95% confidence intervals (CI) for counts, means with standard deviations (SD) for normally distributed continuous varia bles, and medians with quartiles for non - normally distributed continuous variables. Statistical Methods Aim 1 : Patient - level EMS Performance Assessment This analysis focused on 6 recommended indicators of EMS performance (Table 1). 45 For the first 5 measures, compliance was defined as documentation of the task within the corresponding field in MI - EMSIS or, in the case of prenotification, MOSAIC. EMS was considered to have recognized a stroke case if the primary or secondary field impression included stroke or transient ischemic atta ck (TIA). Documentation of any validated stroke screen (FAST, CPSS, LAPSS) was sufficient for satisfying the stroke screen metric. Hospital Table 1: Prehospital stroke performance measure definitions. For each measure, the compliance rate is calculated as the number of compliant cases divided by all EMS - transported stroke cases. Measure Definition Prehospital Stroke Scale EMS documentation of a validated stroke screen Glucose Check Documented glucose level EMS Stroke Recognition EMS primary or secondary impression of stroke or TIA On - Time from EMS scene arrival to beginning transport to hospital is less than or equal to 15 minutes LKW Documentation Cases with EMS documentation of last known well date/time Hospital Prenotification Documentation of prenotification in MOSAIC database 15 prenotification does not have a dedicated field within MI - EMSIS and so compliance with prenotification was based on documentation within the MOSAIC registry. Phi correlation coefficients (or mean square contingency coefficient) were calculated to quantify the correlation in compliance between quality measures. For two binary variables, this may be calculated as follows : = (Equation 1) Where n represents the count of cases with the combination of variable values indicated by the subscripts. The statistic returns a value between - 1 and 1 with interpretation similar to The proportion of cases with documentation of compliance with each measure was calculated across all EMS - transported stroke cases . Following this, bivariate odds ratios (OR) that measure the associations between compliance with each measure and demographic and clinical characteristic s were generated using logistic regression. Demographic characteristics included age, sex, and race (white , black, or other/unknown ), and clinical characteristics included stroke subtype (ischemic /TIA , subarachnoid hemorrhage, or intracerebral hemorrhage) , stroke severity as assessed by the NIHSS, and time from LKW to hospital arrival. To determine patient - level factors independently associated with EMS performance measure compliance, multivariable logistic regression models were constructed for each qu ality measure , including the demographic and clinical covariates listed above . Because it was likely that EMS compliance with prehospital stroke quality measures would be correlated within EMS agencies due to similarities in EMR templates and organizational culture , it was necessary to account for clustering by EMS agency in these models . Furthermore, it was likely that prehospital care delivered to patients taken to the same hospital may be similar due to shared local EMS protocols and region al oversight via regional medical control authorit ies , necessitating that destination hospital - level clustering also be addressed. However, EMS 16 agencies frequently delivered patients to more than one hospital and destination hospitals all received patient s from more than one EMS agency. Therefore, multi - level logistic regression models including crossed random effects for both agency and hospital were developed as follows: 111 Logit [Pr(PM = 1 | a j , b k , ijk 0 + a j + b k + s ijk , (Equation 2 ) where PM represents a given performance measure, a j represents the agency - level random intercept, b k represents the destination hospital - level random intercept, and ijk represents the linear combination of fixed effects (demographic and clinical covariates) for individual i who was transported by agency j to destination hospital k . Inclusion of crosse d effects helps ensure appropriate standard error estimates for fixed effects while providing level - specific estimates of variance for the random effects terms . 112 The Stata (StataCorp, College Station, TX) codes for the logistic regression models is provided in the Appendix. Aim 2: Group - Level Variation in EMS Compliance To assess group - level variations in care, we used two methods for defining groups. First, we considered each of the 147 EMS agencies of transport and 38 destination hospitals as independent group - level variables. In this analysis, the agency - and destin ation hospital - level variance estimates obtained from the crossed random effects logistic regression models (Equation 2 ) were used to calculate intraclass correlation coefficients (ICC) for each group level. This statistic estimates the proportion (range 0 - 1 ) of overall variance in EMS compliance attributable to each level using the formulae : ICC hospital hospital 2 agency 2 hospital 2 + ( 2 / 3)] , (Equation 3a ) ICC agency agency 2 ) agency 2 hospital 2 + ( 2 / 3)] , (Equation 3b ) w here 2 term represents the estimated variance of the random effects term for the specified level of the model and ( 2 / 3) was used to estimate the level 1 variance. 113, 114 Additionally, t 17 random effects models for each quality measure were performed without any fixed effects (i.e., no patient le vel demographic or clinical variables included in the model) . Next , a single - level group variable was created by assigning cases to each unique agency - hospital pair (N=292) and group - level compliance rates were calculated for each performance across all such pairs . Unweighted mean (with standard deviation) agency - hospital pair - level compliance rates (percentage) with each of the 6 quality measures was calculated. To further explore the distribution of compliance across the a gency - hospital pair s, the compliance rates that represented the 25 th , 50 th , 75 th , and 90 th percentiles were calculated as well . Since many agency - hospital clusters contained fewer than 5 cases, a sensitivity analysis was conducted by repeating the above analysis among clusters wit h more than 10 cases. To quantify the magnitude of group - level variability, random effect logistic regression models were constructed including all agency - hospital pairs as a group level variable to estimate the M edian O dds R atio (MOR) , accounting for age , sex, race, stroke type, NIHSS, and onset to door. The MOR is the median value for the distribution of odds ratios comparing odds of quality measure compliance between randomly chosen pairs of individuals from different agency - hospital groups, holding fi xed effects constant. 113 The MOR is calculated by the following formula: MOR = (Equation 4) Where z 0.75 represents the 75 th 2 is the estimator of cluster variance for agency - hospital groups . Derived from the variance of the random effect intercept, this statistic offers the advantage of contextualizin g agency - hospital level variation by allowing more intuitive comparisons between the magnitude of the random and fixed effects. 115 As with overall compliance, s ensitivity analysis was again conducted by repeating the analysis among agency - hospital pa irs with greater than 10 cases . 18 Aim 3 : Associations Between Prehospital Performance and In - hospital Stroke Care , a common benchmark in monitoring acute stroke care, was chosen as the primary outcome for this analysis since it is relevant to both ischemic and hemorrhagic stroke patients . Two secondary outcomes were evaluated in subsets of the full cohort of EMS - transported strokes: (1) delivery of alteplase among all patients with ischemic stroke or TIA who presented within the 4.5 - hour thrombolysis window , and (2) delivery of thrombolytics within 45 minutes of arrival ( door - to - needle [ DTN ] ) among patients who received alteplase . Similar to the an alysis of EMS compliance, l ogistic regression models were created t o quantify bivariate associations between patient demographic and clinical factors and the primary outcome. M ultivariable logistic regression models were then constructed to examine the independent association of each factor with . As with models for EMS compliance, crossed random effects for EMS agency and destination hospital were included to account for and quantify the contribution of agenc y and destination hospital to overall variation using the following mode l : Logit [Pr(Outcome = 1 | a j , b k , ijk 0 + a j + b k + s ijk , (Equation 5 ) where a j represents the random intercept for agency, b k the random intercept for hospital, and ijk the linear combination of fixed effects for individual i transported by agency j to hospital k . Each of the secondary outcomes was then approached in similar fashion. For this portion of the analysis, crossed random effects for agency and hospital group were attempted , however g iven the smaller sample size s for the secondary outcomes, it was anticip ated that crossed random effects models may not converge for these outcomes. In this event, the agency - hospital pair single - level group variable was used as the cluster variable for standard random effects logistic regression models. Finally , t o explor e the relationship between EMS quality measure compliance and ED performance, another set of random effects logistic regression models was constructed with 19 , covariates as above, and each EMS quality measure as primary exposure using the following model: Logit [Pr(Outcome = 1 | p j , PM ij , ij 0 + p j 1 PM ij + s ij , (Equation 6 ) Where PM ij represents the binary variable for compliance with each of the 6 performance measures, p j represents the agency - hospital random intercept ij represents the linear combination of the demographic and clinical fixed effects. For this analysis 1 coefficient represents the independent association between compliance with a given performance measure, accounting for patient - level demographic and clinical fixed effects as well as crossed random effects for agency - hospital pair . Secondary outcome s of alteplase delivery and delivery of alteplase within 45 minutes the respective outcome. Alteplase delivery outcome was analyzed only among patients who were p otentially candidates for this therapy (ischemic stroke patients who arrived during the 4.5 only among patients who received alteplase. 20 CHAPTER 3: RESULTS Analytic Population Stroke registry and EMS registry records for 5,708 stroke cases transported by 147 EMS agencies to one of 38 MOSAIC hospitals were successfully matched over the 18 - month study period . D emographic s, clinical characteristics, and ED - based outcomes for the study population are sum marized in Table 2 . Table 2: Characteristics of the 5708 EMS - transported Stroke Patients Characteristic All Patients N=5708 (% ) Age Category <60 1081 (18.9) 60 - 69 1197 (21) 70 - 79 1383 (24.2) 80 - 89 1437 (25.2) >89 610 (10.7) Female 2971 (52.1) White 4148 (72.7) Year 2018 (12 months) 3684 (64.5) 2019 (6 months) 2024 (35.5) Stroke Type IS/TIA/ s troke NOS 4732 (82.9) SAH 169 (3.0) ICH 806 (14.1) NIHSS 0 - 5 2609 (45.7) 6 - 11 1093 (19.2) 12 - 20 808 (14.2) >20 532 (9.3) Missing 666 (11.7) Onset - t o - d oor 0 - 120 1894 (33.2) 121 - 360 885 (15.5) 136 - 720 645 (11.3) >720 1345 (23.6) Missing 939 (16.5) 21 Table 2 (cont ) 3214 (56.3) Median DTCT (IQR) 21 (10 - 55) Thrombolysis 959 (19.9) 469 (49.0) Median DTN (IQR) 46 (36 - 64) EMS agencies 147 Destination hospitals 38 Agency - hospital pairs 292 The 38 hospitals received stroke cases from a median of 9 distinct agencies (range 1 - 23, IQR 6 - 13) and the 147 agencies transported to a median of 3 different hospitals (range 1 - 10, IQR 2 - 6). There were 292 unique agency - hospital pairs. A summary of the distribution of stroke cases by group level is provided in Figure 2. Figure 2 : Bar graphs demonstrating the distribution of EMS - transported stroke cases across hospitals, EMS agencies, and unique EMS agency - destination hospital pairs. (a) EMS agencies (N=147) 0 100 200 300 400 500 600 700 800 1 11 21 31 41 51 61 71 81 91 101 111 121 131 141 Cases 22 Figure 2 ( ) (b) Destination hospitals (N=38) (c) Unique agency - hospital pairs (N=292) 0 100 200 300 400 500 600 1 6 11 16 21 26 31 36 Cases 0 50 100 150 200 250 300 1 51 101 151 201 251 Cases 23 Aim 1 : Patient - level EMS Performance Assessment Overall EMS compliance with each of the 6 quality is provided in Figure 3. The most consistently performed measure was documentation of a glucose check (82.5%). Hospital prenotification was recorded in MOSAIC for 60.1% of cases. EMS documentation of a prehospital stroke screen was present in 55% of cases while EMS appropriately recogniz ed 5 0.9% of stroke cases transported. A bout half (49.5%) of cases had on - scene times of 15 minutes or less. The least consistent measure was documentation of the time patients were LKW (24.1%). Phi correlation coefficients were all less than 0.5 suggest ing only modest correlation in compliance between individual metrics (Table 3). Figure 3 : Percentage of all 5708 EMS - transported stroke cases with documented prehospital stroke performance metric compliance with 95% confidence intervals 55.0% 82.5% 50.9% 49.5% 24.1% 60.1% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% PSS Document Glucose Check EMS Stroke Recognition LKW Document Prenotification 24 Table 3: Phi correlation coefficients for compliance between pairs of performance metrics across all 5708 EMS - transported stroke cases. PSS Documented Glucose Check EMS Stroke Recognition Minutes LKW Documented Prenotification PSS Documented 1 Glucose Check 0.1295 1 EMS Stroke Recognition 0.379 0.1729 1 0.0781 - 0.0458 0.1964 1 LKW Documented 0.3506 0.1204 0.3059 0.0791 1 Prenotification 0.1937 0.0799 0.2501 0.0689 0.1434 1 A ssociations between patient - level demographic and clinical characteristics and EMS compliance with each quality measure are presented in Table 4. Among demographic document ation compared to younger patients. Females had higher odds of receiving a glucose check, but lower odds of EMS stroke recognition and having scene times less than 15 minutes. Black race was not significantly associated with EMS compliance for any measur e in the adjusted models. For clinical variables , stroke severity (NIHSS) was the most consistent predictor of EMS quality measure compliance, with statistically significant associations observed for all 6 performance measures. M oderate stroke sever ity ( NIHSS 11 - 20) consistently demonstrated the strongest positive associations with compliance. Patients presenting earlier from time LKW had greater odds with EMS compliance for 5 of the 6 measures (the exception being glucose) . Factors associated with redu ced odds of EMS compliance included unknown or missing values for NIHSS (3/6 measures), time from LKW well (5/5), as well as final diagnosis of SAH (4/6) . 25 Table 4: Relationship between demographic and clinical characteristics and EMS perfo rmance measure compliance among 5708 EMS - transported stroke cases in unadjusted and adjusted ORs generated from multivariable crossed random effects logistic regression models. PSS Documentation Glucose Check EMS Stroke Recognition Covariate Unadjusted Adjusted Unadjusted Adjusted Unadjusted Adjusted Age <60 Ref Ref Ref Ref Ref Ref 60 - 69 1.37 (1.16 - 1.61) 1.4 (1.14 - 1.71) 1.1 (0.89 - 1.36) 1.19 (0.93 - 1.52) 1.26 (1.07 - 1.48) 1.3 (1.08 - 1.57) 70 - 79 1.37 (1.17 - 1.61) 1.31 (1.08 - 1.6) 1.08 (0.88 - 1.33) 1.22 (0.96 - 1.56) 1.15 (0.98 - 1.35) 1.12 (0.93 - 1.35) 80 - 89 1.34 (1.14 - 1.57) 1.24 (1.01 - 1.51) 1.14 (0.93 - 1.4) 1.28 (1 - 1.64) 1.31 (1.11 - 1.53) 1.21 (1 - 1.46) 90 1.38 (1.13 - 1.68) 1.33 (1.03 - 1.71) 1.22 (0.93 - 1.58) 1.4 (1.02 - 1.92) 1.30 (1.06 - 1.58) 1.23 (0.97 - 1.55) Female 0.94 (0.85 - 1.04) 0.93 (0.81 - 1.05) 1.15 (1 - 1.31) 1.18 (1.02 - 1.38) 0.82 (0.74 - 0.91) 0.79 (0.70 - 0.89) Race White Ref Ref Ref Ref Ref Ref Black 0.64 (0.56 - 0.72) 0.96 (0.79 - 1.17) 1.24 (1.04 - 1.47) 1.22 (0.96 - 1.55) 0.64 (0.57 - 0.73) 0.88 (0.73 - 1.06) Other/missing 1.02 (0.81 - 1.28) 1.04 (0.78 - 1.38) 1.06 (0.79 - 1.42) 1.09 (0.78 - 1.54) 0.67 (0.55 - 0.87) 0.76 (0.58 - 1) Stroke Subtype IS/TIA Ref Ref Ref Ref Ref Ref SAH 0.33 (0.24 - 0.47) 0.57 (0.38 - 0.86) 0.79 (0.54 - 1.15) 0.98 (0.64 - 1.51) 0.24 (0.16 - 0.35) 0.37 (0.24 - 0.57) ICH 0.7 (0.6 - 0.81) 1.12 (0.91 - 1.37) 1.1 (0.9 - 1.35) 1.18 (0.93 - 1.51) 0.85 (0.73 - 0.98) 1.11 (0.91 - 1.34) NIHSS 0 - 6 Ref Ref Ref Ref Ref Ref 6 - 11 1.52 (1.31 - 1.76) 1.69 (1.41 - 2.02) 1.28 (1.06 - 1.54) 1.33 (1.08 - 1.65) 2.32 (2.00 - 2.68) 2.41 (2.05 - 2.83) 12 - 20 1.47 (1.25 - 1.73) 1.67 (1.36 - 2.04) 1.75 (1.39 - 2.21) 1.82 (1.41 - 2.37) 2.72 (2.30 - 3.22) 2.9 (2.41 - 3.49) >20 1.04 (0.86 - 1.26) 1.03 (0.82 - 1.3) 1.8 (1.36 - 2.38) 1.79 (1.31 - 2.44) 2.14 (1.76 - 2.59) 2.15 (1.73 - 2.66) Missing 0.36 (0.3 - 0.44) 0.4 (0.32 - 0.51) 0.78 (0.64 - 0.96) 0.83 (0.64 - 1.08) 0.34 (0.28 - 0.41) 0.48 (0.37 - 0.61) LKW to Door 0 - 120 Ref Ref Ref Ref Ref Ref 121 - 360 0.9 (0.77 - 1.07) 0.81 (0.66 - 0.99) 1.08 (0.86 - 1.35) 1.03 (0.8 - 1.32) 0.85 (0.72 - 1.0) 0.8 (0.67 - 0.96) 361 - 720 0.74 (0.62 - 0.89) 0.69 (0.55 - 0.86) 0.84 (0.66 - 1.06) 0.9 (0.69 - 1.17) 0.66 (0.55 - 0.80) 0.65 (0.53 - 0.79) >720 0.57 (0.49 - 0.66) 0.52 (0.44 - 0.62) 0.86 (0.71 - 1.04) 0.89 (0.72 - 1.1) 0.37 (0.32 - 0.43) 0.38 (0.32 - 0.44) Missing 0.34 (0.29 - 0.4) 0.28 (0.23 - 0.34) 0.59 (0.48 - 0.71) 0.58 (0.47 - 0.73) 0.16 (0.14 - 0.19) 0.17 (0.14 - 0.21) Agency ICC 0.52 * 0.55 0.26 * 0.27 0.09 * 0.10 Hospital ICC 0.01 * 0.01 0.02 * 0.02 0.01 * 0.01 *Unadjusted ICC is derived from the unconditional means model containing no fixed effects . IS/TIA = ischemic stroke/transient ischemic attack; SAH = subarachnoid hemorrhage; ICH = intracerebral hemorrhage; NIHSS = NIH stroke score; LKW = Time last known well ; ICC = intraclass correlation coefficient 26 Table 4 ( ) *Unadjusted ICC is derived from the unconditional means model containing no fixed effects. IS/TIA = ischemic stroke/transient ischemic attack; SAH = subarachnoid hemorrhage; ICH = intracerebral hemorrhage; NIHSS = NIH stroke score; LKW = Time last known well ; ICC = intraclass correlation coefficient LKW Documentation On - Scene 15 Minutes Prenotification Covariate Unadjusted Adjusted Unadjusted Adjusted Unadjusted Adjusted Age <60 Ref Ref Ref Ref Ref Ref 60 - 69 1.11 (0.91 - 1.36) 1.2 (0.92 - 1.57) 0.80 (0.68 - 0.94) 0.8 (0.67 - 0.95) 0.99 (0.84 - 1.17) 1.01 (0.83 - 1.24) 70 - 79 1.20 (0.99 - 1.45) 1.11 (0.86 - 1.43) 0.77 (0.66 - 0.91) 0.73 (0.62 - 0.87) 1.12 (0.95 - 1.32) 1.14 (0.94 - 1.4) 80 - 89 1.29 (1.07 - 1.53) 1.12 (0.86 - 1.44) 0.71 (0.61 - 0.84) 0.65 (0.54 - 0.77) 1.07 (0.91 - 1.25) 0.98 (0.8 - 1.19) 90 1.33 (1.06 - 1.68) 1.15 (0.84 - 1.58) 0.61 (0.50 - 0.74) 0.57 (0.45 - 0.71) 0.92 (0.76 - 1.13) 0.89 (0.69 - 1.15) Female 0.91 (0.80 - 1.02) 0.91 (0.77 - 1.07) 0.83 (0.75 - 0.92) 0.87 (0.78 - 0.97) 0.87 (0.79 - 0.97) 0.89 (0.78 - 1.01) Race White Ref Ref Ref Ref Ref Ref Blac k 0.42 (0.35 - 0.50) 0.88 (0.68 - 1.15) 0.83 (0.73 - 0.94) 0.89 (0.75 - 1.05) 0.69 (0.61 - 0.79) 1.05 (0.86 - 1.27) Other/missi ng 1.08 (0.84 - 1.38) 0.96 (0.68 - 1.36) 0.79 (0.63 - 1.00) 0.85 (0.66 - 1.09) 1.36 (1.04 - 1.73) 0.97 (0.72 - 1.3) Stroke Subtype IS/TIA Ref Ref Ref Ref Ref Ref SAH 0.40 (0 .25 - 0.65) 0.4 (0.22 - 0.71) 0.77 (0.57 - 1.05) 0.77 (0.55 - 1.08) 0.56 (0.41 - 0.76) 0.63 (0.43 - 0.92) ICH 0.81 (0.68 - 0.98) 1.01 (0.78 - 1.31) 1.02 (0.88 - 1.18) 1.01 (0.85 - 1.2) 0.98 (0.84 - 1.15) 1.13 (0.93 - 1.38) NIHSS 0 - 6 Ref Ref Ref Ref Ref Ref 6 - 11 1.35 ( 1.15 - 1.58) 1.58 (1.28 - 1.96) 1.22 (1.06 - 1.41) 1.21 (1.04 - 1.4) 1.59 (1.37 - 1.85) 1.61 (1.35 - 1.91) 12 - 20 1.53 (1.28 - 1.82) 1.81 (1.43 - 2.29) 1.55 (1.32 - 1.82) 1.53 (1.29 - 1.81) 1.72 (1.45 - 2.04) 1.77 (1.45 - 2.16) >20 1.18 (0 .95 - 1.46) 1.6 (1.2 - 2.14) 1.32 (1.10 - 1.59) 1.36 (1.11 - 1.66) 1.59 (1.31 - 1.94) 1.54 (1.22 - 1.95) Missing 0.6 3 (0.50 - 0.79) 0.89 (0.63 - 1.26) 0.81 (0.68 - 0.96) 0.96 (0.78 - 1.18) 0.62 (0.52 - 0.73) 0.7 (0.55 - 0.89) LKW to Door 0 - 120 Ref Ref Ref Ref Ref Ref 121 - 360 0.8 4 (0.71 - 1.00) 0.78 (0.63 - 0.98) 0.62 (0.53 - 0.73) 0.62 (0.53 - 0.74) 0.96 (0.81 - 1.14) 0.91 (0.74 - 1.11) 361 - 720 0.81 (0.67 - 0.98) 0.81 (0.63 - 1.04) 0.52 (0.43 - 0.62) 0.52 (0.43 - 0.62) 0.84 (0.70 - 1.02) 0.78 (0.63 - 0.97) >720 0.54 ( 0.46 - 0.64) 0.52 (0.43 - 0.64) 0.50 (0.43 - 0.57) 0.5 (0.43 - 0.58) 0.51 (0.44 - 0.59) 0.48 (0.41 - 0.57) Missing -- -- 0.39 (0.33 - 0.46) 0.41 (0.34 - 0.49) 0.45 (0.39 - 0.53) 0.44 (0.37 - 0.54) Agency ICC 0.56 * 0.59 0.05 * 0.06 0.03 * 0.03 Hospital ICC 0.00 * 0.00 0.01 * 0.01 0.40 * 0.41 27 Aim 2: Group - Level Variation in EMS Compliance In the crossed random effects models above, a significant amount of overall variability in EMS compl iance was attributable to between EMS agency variation. For metrics such as PSS documentation and LKW documentation, this accounted for more than half of all variation. In e to EMS agencies with ICCs of 0.1 and 0.06, respectively. The contribution of destination hospital - level variation to overall variation was very small, except for prenotification documentation, where it accounted for over 40% of total variation. This is likely attributable to hospital - level differences in capturing prenotification events and recording them in MOSAIC. Among the 292 unique agency - hospital pairs that comprised the single - level group variable the median number of stroke cases transported o ver the full 18 months in the dataset was 5 (range 1 - 255, inter - quartile range 2 - 18). Average performance across the 292 unique EMS agency - hospital pairs as well as performance in the 25 th , 50 th (median), 75 th , and 90 th percentiles are presented in Table 5. There was a large degree of variability between agency - hospital pairs in compliance for PSS documentation, LKW documentation and hospital the very small volu mes in many agency - hospital pairs, a sensitivity analysis excluding agency - hospital pairs with 10 or fewer runs was also performed (Table 6). By excluding the small clusters, overall performance at the high and low extremes was attenuated such that 90 th percentile performance was slightly lower and 25 th percentile performance slightly higher, but median performance was only minimally affected. The variability attributable to transport by a given agency - hospital pair estimated by the MOR analysis further illustrates the wide variation in care seen across the agency - hospital pairs. MORs represent the median value among all odds ratios for compliance comparing randomly chosen individuals from different agency - hospital pairs, holding fixed effects constant. 28 Thus, the MORs interpretation is similar to other OR, where values >1 imply that the relative odds of compliance differs over across agency hospital pairs and larger values reflecting greater between cluster variability . For example, the highest MOR was for LKW documentation (6.6), which implies that if all agency - hospital groups are compared, the median odds of LKW documentation are 6.6 times higher among the higher performing compared to the lower performing group. The MORs were also high for PSS docu mentation (5.98) and hospital (1.58) MORS were relatively modest. Sensitivity analysis excluding agency - hospital pairs with fewer than 10 cases did not meaning fully change MOR estimates. Table 5: Group - level performance measure compliance across all 292 unique agency - hospital pairs. Mean Compliance (SD) 25 th Percentile 50 th Percentile 75 th Percentile 90 th Percentile MOR* (95% CI) PSS Documentation 50 (38.6) 0.0 55.0 83.8 100.0 5.98 (4.61 - 8.08) Glucose Check 76.5 (30.7) 67.0 89.0 100.0 100.0 2.83 (2.41 - 3.41) EMS Stroke Recognition 48.2 (33.8) 20.0 50.0 70.8 100.0 1.86 (1.66 - 2.13) 50 (32.6) 30.3 50.0 68.8 100.0 1.58 (1.45 - 1.77) LKW Documented 23.4 (31.3) 0.0 6.0 42.8 69.7 6.66 (5.01 - 9.29) Prenotification 54.9 (37.1) 23.3 57.0 94.8 100.0 4.26 (3.49 - 5.37) SD = standard deviation; PSS = prehospital stroke scale; OST = on - scene time; LKW = last known well * Median Odds Ration = m edian value for the distribution of odds ratios comparing odds of compliance between two randomly selected individuals from different agency - hospital pairs who have the same values for age, sex, race, stroke type, stroke severit y, and time from onset of symptoms. 29 Table 6 : Group - level performance measure compliance across 101 unique agency - hospital pairs with greater than 10 stroke transports . Mean Compliance (SD) 25 th Percentile 50 th Percentile 75 th Percentile 90 th Percentile MOR* PSS Documentation 51 (28.6) 31.0 55.0 72.5 84.0 5.97 (4.41 - 8.59) Glucose Check 80 (18.5) 75.0 87.0 93.0 96.0 2.82 (2.37 - 3.49) EMS Stroke Recognition 52.1 (17.4) 40.5 55.0 63.0 74.4 1.82 (1.63 - 2.1) 27 (24.4) 41.0 53.0 61.0 73.8 1.6 (1.45 - 1.8) LKW Documented 52.6 (15.1) 6.0 19.0 47.0 65.0 7.07 (5.04 - 10.66) Prenotification 58.7 (26.9) 37.5 59.0 82.0 93.0 4.33 (3.44 - 5.71) SD = standard deviation; PSS = prehospital stroke scale; OST = on - scene time; LKW = last known well * Median Odds Ration = m edian value for the distribution of odds ratios comparing odds of compliance between two randomly selected individuals from different agency - hospital pairs who have the same values for age, sex, race, stroke type, stroke severity, and time from onset of symptoms. Aim 3 : Associations Between Prehospital Performance and In - H ospital Stroke Care Models examining the association between demographic and clinical covariates with the primary and secondary outcomes are presented in Table 7 . For the primary outcome of demographic characteristics were generally noncontributory with odds ratios at or near to 1. T he most influential clinical characteristics were a final diagno sis of SAH and presentation delays beyond 360 minutes from LKW, which were both associated with lower odds of early CT, and moderate or greater stroke severity (NIHSS >6) , which was associated with much higher odds of early CT . Group - level variability was more modest than for EMS compliance and was attributable entirely to hospital effects (ICC for EMS agency was 0). For the secondary outcome of alteplase delivery among IS/TIA patients who presented in the thrombolytic window, the strongest predictor of treatment was again NIHSS, 30 although black race and old age particularly beyond 90 years were both associated with l ower odds of treatment. Only NIHSS was associated the outcome of achieving a DTN time within 45 minutes of arrival. crossed random effects model treating agency and destination as separate group variables did not converge, but age ncy - hospital pairs accounted for about 10% of overall variability in rapid alteplase delivery. Table 7 : Relationship between clinical and demographic variables and ED outcomes in unadjusted and adjusted (multivariable crossed random effects or single - l evel random effect) logistic regression models . DTCT 25 Minutes* Alteplase Delivery** Covariate Unadjusted Adjusted Unadjusted Adjusted Unadjusted Adjusted Age <60 Ref Ref Ref Ref Ref Ref 60 - 69 1.03 (0.87 - 1.21) 1.03 (0.85 - 1.25) 0.84 (0.65 - 1.08) 0.83 (0.63 - 1.1) 1.23 (0.84 - 1.80) 1.22 (0.80 - 1.87) 70 - 79 1.02 (03.87 - 1.21) 1.02 (0.84 - 1.23) 0.62 (0.49 - 0.81) 0.59 (0.45 - 0.78) 1.26 (0.87 - 1.84) 1.18 (0.78 - 1.79) 80 - 89 1.08 (0.92 - 1.27) 1.01 (0.83 - 1.23) 0.64 (0.50 - 0.82) 0.55 (0.42 - 0.72) 1.23 (0.85 - 1.78) 1.12 (0.74 - 1.71) 90 0.99 (0.81 - 1.20) 0.97 (0.76 - 1.24) 0.43 (0.31 - 0.60) 0.34 (0.23 - 0.49) 1.05 (0.61 - 1.81) 0.89 (0.48 - 1.64) Female 0.92 (0.83 - 1.02) 0.92 (0.81 - 1.04) 0.98 (0.84 - 1.16) 1.01 (0.85 - 1.22) 0.92 (0.81 - 1.18) 0.92 (0.70 - 1.22) Race White Ref Ref Ref Ref Ref Ref Black 0.85 (0.78 - 0.93) 0.85 (0.7 - 1.02) 1.07 (0.87 - 1.33) 0.7 (0.54 - 0.92) 0.75 (0.54 - 1.04) 0.89 (0.58 - 1.22) Other/missing 1.36 (1.28 - 1.44) 0.85 (0.65 - 1.13) 1.49 (1.05 - 2.11) 1.17 (0.79 - 1.72) 1.28 (0.77 - 2.13) 1.25 (0.72 - 2.18) Stroke Subtype IS/TIA Ref Ref N/A N/A N/A N/A SAH 0.32 (0.23 - 0.44) 0.45 (0.3 - 0.66) N/A N/A N/A N/A ICH 0.87 (0.75 - 1.01) 1.11 (0.91 - 1.35) N/A N/A N/A N/A 31 Table 7 (cont NIHSS 0 - 6 Ref Ref Ref Ref Ref Ref 6 - 11 2.65 (2.28 - 3.09) 2.6 (2.2 - 3.08) 3.64 (2.95 - 4.49) 3.8 (3.05 - 4.74) 1.35 (0.98 - 1.85) 1.53 (1.08 - 2.17) 12 - 20 3.43 (2.87 - 4.10) 3.76 (3.08 - 4.59) 4.16 (3.26 (5.32) 4.42 (3.42 - 5.72) 2.15 (1.51 - 3.08) 2.32 (1.57 - 3.41) >20 2.21 (0.81 - 2.69) 2.35 (1.88 - 2.93) 2.87 (2.15 - 3.82) 3.16 (2.34 - 4.27) 1.10 (0.71 - 1.69) 1.20 (0.74 - 1.92) Missing 0.40 (0.33 - 0.48) 0.57 (0.45 - 0.71) 0.42 (0.19 - 0.93) 0.46 (0.2 - 1.06) 0.55 (0.11 - 2.88) 0.69 (0.12 - 4.00) LKW to Door 0 - 120 Ref Ref N/A N/A N/A N/A 121 - 360 0.74 (0.62 - 0.88) 0.72 (0.6 - 0.87) N/A N/A N/A N/A 361 - 720 0.51 (0.43 - 0.62) 0.49 (0.4 - 0.6) N/A N/A N/A N/A >720 0.21 (0.18 - 0.24) 0.2 (0.17 - 0.24) N/A N/A N/A N/A Missing 0.13 (0.11 - 0.16) 0.14 (0.11 - 0.17) N/A N/A N/A N/A Agency ICC 0.00 0.00 -- Hospital ICC 0.07 0.05 -- Agency - Hospital ICC -- -- 0.1 IS/TIA = ischemic stroke/transient ischemic attack; SAH = subarachnoid hemorrhage; ICH = intracerebral hemorrhage; NIHSS = NIH stroke score; LKW = Time last known well ; ICC = intraclass correlation coefficient * Among 5708 EMS - transported strokes ** Among 2438 ischemic stroke or TIA patients who presented withi n 4.5 hours of LKW. *** Among 959 alteplase treated patients. A model with crossed random effects between agency and destination hospital did not converge, therefore the single - level group random effect was used . LKW was not included in the secondary outcomes as only those patients presenting early are eligible for treatment. Table 8 reports the results of unadjusted bivariate associations between EMS performance of each quality measure and the three ED perfor mance outcomes as well as the adjusted odds ratios for each measure following adjustment for demographic and clinical covariates as well as agency - hospital pair cluster effects. All 6 measures had statistically significant positive associations with early CT acquisition in the ED. The strongest association was between EMS recognition of stroke and early CT ( aOR 6.24) with glucose documentation representing the weakest ( aOR 1.77). For alteplase delivery and DTN times, glucose check 32 was not significant for either outcome , 45 minutes. The remaining measures all maintained statistically significant associations with each outcome and EMS recognition remained the strongest predictor (a OR 1.69 for alteplase delivery and 2.74 for a DTN . Analysis for the primary outcome was repeated dropping agency - hospital pairs with 10 or fewer runs from the multivariable models and the results were unchanged. Table 8 : Associations between EMS quality measure compliance and ED outcomes Alteplase Delivery** Unadjusted Adjusted Unadjusted Adjusted Unadjusted Adjusted PSS Documentation 1.29 (1.22 - 1.36) 2.64 (2.29 - 3.03) 1.30 (1.10 - 1.55) 1.28 (1.05 - 1.56) 1.57 (1.19 - 2.06) 1.49 (1.09 - 2.05) Glucose Check 1.84 (1.61 - 2.12) 1.77 (1.49 - 2.09) 1.33 (1.06 - 1.69) 1.19 (0.92 - 1.54) 1.38 (0.94 - 2.01) 1.36 (0.88 - 2.09) EMS Stroke Recognition 7.69 (6.83 - 8.66) 6.24 (5.42 - 7.20) 1.95 (1.63 - 2.32) 1.69 (1.39 - 2.06) 2.39 (1.78 - 3.23) 2.74 (1.93 - 3.88) 2.16 (1.94 - 2.40) 1.89 (1.66 - 2.14) 1.51 (1.28 - 1.78) 1.40 (1.16 - 1.68) 1.16 (0.89 - 1.51) 1.09 (0.82 - 1.46) LKW Documented 2.91 (2.54 - 3.33) 2.30 (1.92 - 2.74) 1.09 (0.92 - 1.30) 1.04 (0.85 - 1.27) 1.40 (1.07 - 1.83) 1.44 (1.03 - 1.99) Prenotification 2.84 (2.54 - 3.16) 2.48 (2.15 - 2.85) 1.62 (1.35 - 1.94) 1.51 (1.23 - 1.85) 1.86 (1.38 - 2.50) 2.02 (1.43 - 2.84) DTCT = door - to - CT; DTN = door - to - needle; PSS = prehospital stroke scale; OST = on - scene time; LKW = last known well * Among 5708 EMS - transported strokes. ** Among 2438 ischemic stroke or TIA patients who presented within 4.5 hours of LKW. *** Among 959 alteplase treated patient s. Adjusted for age, sex, race, stroke type, stroke severity, time from onset to arrival, and clustering by age n cy - hospital pairs. Adjusted for age, sex, race, stroke severity, and clustering by ag e ncy - hospital pairs. 33 CHAPTER 4: DISCUSSION As the first point of contact for most patients with stroke symptoms have with the healthcare system, EMS has the earliest opportunity to recognize stroke and coordinate a rapid response. EMS - transported stroke cases, when compared to patients who arrive at the hospital by other means such as private transportation, receive fas ter ED stroke evaluations and treatment. 49 However, previous work has suggested that these benefits are not universally expe rienced but are tied to the quality of EMS care, including recognition of stroke and compliance with performance measures such as prehospital notification . 67, 84, 89, 90, 116 Despite longstanding recommendations for optimal prehospital stroke care being included in stroke clinical guidelines , 64, 65, 117 studies auditing EMS compliance with recommendations in the real world are lacking. This thesis provides a u nique comprehensive evaluation of EMS stroke care delivered across diverse regions of Michigan , leveraging two existing state - wide registries to both quantify EMS compliance with recommended stroke care practices and to estimate the impact of the quality o f EMS care on downstream stroke treatment . Consistent with the limited existing data on the topic, 82, 84, 86, 88, 89, 95, 118 we identified significant variability in EMS stroke care . This analysis provides new insights into both patient - level and healthcare system - level factors that contribute to this variability and provides additional evidence that compliance with recommended practices for prehospital stroke care is associated with superior early stroke care following hosp ital arrival. With respect to our first aim to examine patient - level variability in EMS performance, we documented compliance with 6 markers of high - quality EMS stroke care: performance of a validated PSS, obtaining a point - of - care glucose level , accurat e ly identifying stroke patients , determin ing during transport . Each of these measures is supported by expert consensus and clinical guideline s 64, 65 34 79 and are commonly used by EMS agencies for monitoring the quality of prehospital stroke care. 119 Among EMS - transported stroke cases , glucose documentation was the performance measure most consistently documented (82.5%). This degree of compliance is identical to that observed in several smaller cohorts of EMS - transported stroke cases . 82, 118 Given that checking glucose is consistently required by stroke protocols, 78 is also important for other commonly encountered complaints such as altered mental status, 120 and the fact that hypoglycemia is correctable in the prehospital setting, it would seem reasonable to seek compliance approaching 100% for this metric . P rehospital stroke scale documen tation was reported in about 55% of EMS - transported strokes . This level of compliance is again in keeping with prior population - based assessments of EMS stroke care , 82, 84, 118 although one study reported rates near 100% . 88 Perhaps related to this , overall EMS stroke recognition was also very modest ( 5 1 % ) compared to other published reports, which suggest EMS recognition ranges from 50 - 75% 86, 88, 89, 92, 94, 95 Given the well - described positive association between PSS performance and accurate EMS recognition, 75, 86, 87, 92 as well as evidence that EMS education can improve recognition, 98 the low rate of stroke recogni tion by EMS would be a logical target for additional quality improvement efforts . The measure with the lowest rate of compliance was documentation of LKW time . This is again comparable to another report of EMS performance 82 and may arise at least in part from idiosyncrasies regarding how this data element is collected. Within Mi - EMS IS , nearly all cases EMS transports . The LKW data element comes from a separate, dedicated field specific to documentation of LKW in the context of stroke. We suspect that there is variability in EMS documentation such that the LKW field is not consistently used as intended. The large value for the agency ICC supports this theory and implies that agency - level EMR structure or documentation practice s may be a key driver of the lo w observed rates of compliance . 35 A similar phenomenon may be at work for the hospital prenotification me tric . Our 60% compliance rate was determined based on documentation in the MOSAIC registry . While this rate is similar to that observed across the GWTG - Stroke registry, 80 there was again substantial variability by MOSAIC hospital (ICC=0.4). This might arise from hospital - level variation in the availability or quality of hospital documentation regarding prenotification. Since the abstraction standard demands explicit docum entation of both receipt of EMS communication prior to arrival , 80 and pr enotification occurs before the patient is actually present or registered in the ED, it seems likely that this documentation may be lacking in hospital electronic records even when prenotification was provided by EMS crews . Given that this prehospital str oke performance metric has the largest body of evidence suggesting that compliance positively impact s downstream care and outcomes , 68, 69, 72, 84, 89 there is a significant opportunity to target improvement both the rate of and the quality of hospital prenotification by ensuring a standardized prenotification communication and documentation process between EMS agencies and recipient hospitals . This information might also be targeted for improvement within t he MI - EMSIS data, where it is inconsistently recorded currently. On - scene times across Michigan demonstrated relatively little variability and met the goal of 15 minute s in about half of cases. This is very similar to reported rates in other areas. 81, 1 18 One recent study from Florida suggested more than 60% compliance with this goal, however differences in the definition of on - scene time may partially explain the lower rate in Michigan since OST was subdivided into time from arrival to patient and time from patient contact to departur e in that study . 121 This metric demonstrated the lowest agency - level variability, which we suspect demonstrates a higher degree of completeness and accuracy of this documentation as these time stamps are carefully recorded for all EMS transports. In benchmarking this metric, it will be important to balance the need for expediency against potentially valuable but time - consuming aspec ts such as determining an accurate last known well. 36 Interestingly, it did not appear that compliance with any one quality measure was necessarily predictive of compliance with other quality measures. This was unexpected as it seemed intuitively likely th at performing recommended practices outlined in stroke transport protocols would tend to occur together as a bundle. However, t he phi correlation coefficients , which were all less than 0 .4, suggest ed relatively little correlation between the individual it ems varied little regardless of which pair of measures was examined. The low degree of correlation between measures suggests that, at least for assessment of overall performance and their association with outcomes , these measures may be treated independe ntly. Demographic characteristics demonstrated only modest associations with EMS performance measure compliance. In multivariable models, o lder patients had greater odds of PSS documentation and glucose check, which might indicate EMS providers maintaini ng a higher index of suspicion for stroke in this population. However, age was not associated with EMS recognition of stroke or prenotification. This latter finding may suggest that the driver of EMS impression and prenotification practices may rely on h istory and exam findings suggestive of stroke rather than risk factors such as age. There also appeared to be a steady decrease in the likelihood of leaving the scene within 15 minutes for each decade of life beyond 60, which may be driven by mobility dif ficulties , nursing home residence , or possibly even a perception of lower likelihood of intervention among older stroke patients . Female sex was associated with slightly higher odds of receiving a glucose check, but slightly lower odds of EMS stroke EMS stroke recognition among females has been described in at least one other previo us study . 122 One possible explanation for this might be higher rates of atypical symptoms among women as has been reported in hospital - based studies , 122 however f urther research is needed to confirm and investigate potential causes for this finding. Fin ally, although black race (compared to white or other/missing race) was associated with lower odds of EMS compliance for several measures in unadjusted analysis , theses associations became non - significant following adjustment in multivariable 37 models , sugge sting that other factors such as clinical presentation account for much of the crude differences observed by race. Given the known racial disparities in stroke treatment and outcomes generally, 2 these findings are somewhat reassuring with respect to equity in prehospital stroke care . Several patient - level clinical characteristics were predictive of compliance with prehospital stroke performance measures. Most notably, moderately severe strokes, ischemic stroke s , and patients who presented sooner after symptom onset tended to receive higher quality prehospital care . Conversely, SAH patients, m ilder strokes , and those with unknown duration of symptoms tended t o have lower odds of compliant EMS care. It is perhaps not surprising that strokes that are more obvious or early in their course might prompt a more aggressi ve response from EMS, knowing that stroke treatment is more likely in these populations and is time dependent. Furthermore, lower EMS performance among SAH patients likely arises from the markedly different clinical presentation of that condition, which i s characterized more by severe headache than focal deficits. As a result, these patients would be less likely to screen positive on PSS tools and more likely to be transported and pre - alerted as a benign headache. Taken together, t hese results encouragin g in that patients most likely to be candidates for intervention (early presenting, moderately severe ischemic strokes) were more likely to receive optimal prehospital care. Nevertheless, further education may assist EMS providers in recognizing more subt le or atypical presentations that could still benefit from a rapid response . One example of this is posterior stroke patients, who are known to experience delays in diagnosis in the ED and often experience delays in care. 123 - 126 We previously demonstrated that a brief educational intervention improved posterior stroke recognition rates by EMS. 127 The primary goal of our second aim was to quantify the degree of variability that exists between agencies in this very diverse sample of EMS transported strokes. Variation in EMS care has been previously documented for out - of - hospital cardiac arrest 128 but has not been examined for stroke. G iven that both the EMS agency of transport and the destination hospital 38 (as a marker for regional practices) had plausible relationships to EMS practice, we used crossed random effects models to quantify the contribution of each of those factors to overall variation in care. Although techniques such as this have been used els ewhere to assess the validity of quality profiling of surgeons, for example, 129 this i s the first analysis of its kind that we are aware of that ad dresses prehospital stroke care . This analysis is important because if all variability in care is attributable to patient - level factors, profiling individual EMS agency performance may not be a useful exercise. This analysis resulted in three interesting findings. First, group - level variation in performance measures was attributable almost entirely to the source from which performance data were derived . For EMS metrics derived from EMS documentation, group - leve l variability was overwhelmingly related to differences between EMS agencies ; whereas for prenotification, which was derived from hospital data, the group level variability arose almost entirely from hospital - level variation. There wa s no variable for whi ch both agency of transport and destination hospital groups contributed meaningful ly to total variance. Second, the magnitude of association s between patient - level fixed effect associations with performance measure compliance were not altered significantl y by inclusion of group level effects in the models , n or was group - level variation significantly different in models with and without patient - level covariates. These findings imply that the two levels of variability operate mostly independent of one anoth er. Third, the proportion of overall variation in EMS care accounted for by group - level effects was substantial for several metrics central to optimal prehospital stroke care. More than ha lf of variation in documentation of PSS (ICC=0.5 5 ) and LKW (ICC= 0. 59) was attributable to agency - level variability. Since variables such as EMS field impression and OST are infrequently missing from MI - EMSIS, 107 and performance measures derived from these fields demonstrated much lower agency - level va riability (ICC 0.1 for EMS impression of stroke and ICC of 0.06 or OST), it seems likely that much of the agency - level variation in PSS and LKW documentation arise from differences in data entry and upload processes between agencies rather than true 39 differ ences in care . Similarly, group - level variation in prenotification documentation was attributed entirely to destination hospital (the source of this data , ICC=0.41) even though all other metrics varied little by destination hospital. This underscores the need to standardize and optimize methods for collecting prehospital data in both MI - EMSIS and MOSAIC . When EMS - transported strokes cases were grouped into unique agency - hospital pairs, the first important observation we made wa s that many of these group s contained very few cases. In fact, over 18 months, only 101/292 (35%) of such pairs transported more than 10 cases. While there did not appear to be much difference between the pairs with smaller caseloads compared to larger ones in terms of performanc e, this underscores a key challenge in prehospital stroke care quality improvement in that many transporting agencies do not see a large volume of stroke cases. S tratum - specific performance estimates for each unique agency destination hospital pair in the sample also demonstrated highly variable performance. As estimated by the MOR for the random effects of agency hospital pair, t he magnitude of difference in the relative odds of compliance between the pairs was quite high . In the case of PSS documentation (MOR=5.98) , LKW documentation (MOR=6.66) , and hospital prenotification (MOR 4.26) the agency - hospital odds ratio was much larger than any other demographic or clinical factor in the model . These results are unlikely to reflect real differenc es in care but rather highlight the opportunity to look more closely at documentation and data upload practices at lower - performing agencies to ensure comparable reporting methods. To be of value for patients, prehospital compliance with performance measu res should positively influence patient outcomes. EMS transport offers two opportunities to do this : identify stroke early and expedite evaluation of stroke patients to reduce time to treatment through reducing prehospital delay and facilitating a faster ED response following arrival . Each prehospital metric is directed toward one or the other of these goals. In the literature to date, the only metrics that have been positively associated with hospital - based outcomes have been EMS recognition 86, 89 and prenotification. 67, 84, 116, 130, 131 We found that EMS compliance with 40 each of the 6 performance measures was associated with early CT acquisition independent of patient demographics, stroke type, stroke severity, or timing of presentation and accounting for clustering by EMS agency and hospital. For some quality measures, this relationship was somewhat surpr ising. Obtaining a glucose check, for example, is intended to help EMS providers exclude hypoglycemia. Thus, in this cohort of confirmed stroke cases, there is no logical reason why such documentation would directly impact hospital care. Similarly, main impact on the process of stroke evaluation following hospital arrival. Instead of operating within the causal chain of faster ED care, we suspect these measures serve more as a marker of adherence to the stroke protocol, which may be influenced by EMS confidence in their diagnostic impression of stroke in more obvious stroke cases . On the other hand, hospital prenotification has an obvious influence over ED care , theref ore w e expected to see an association between these variables . Indeed, the observed adjusted odds ratio s for prenotification on early CT (2.48), alteplase delivery (1.51), and rapid alteplase delivery (2.02) were all higher than previously reported for prenotification in a large analysis from GWTG - Stroke. 67 Given that the GWTG - Stroke analysis was conducted using data prior to 2012, some of the increased effect may be the result of improvements in stroke alert processes over time. While these findings are encouraging, they do not imply that prenotification should be performed or would be beneficia l for every EMS - transported stroke case. Prenotification in the setting of stroke patients who are not candidates for intervention could result in unnecessary mobilization of ED resources and potentially undermine the value of the process. Previous work has documented frustration among prehospital providers regarding conflicting expectations regarding the appropriate clinical context for activating a stroke alert prior to arrival in the ED . 132 Adding to this confusion is the ever - changing treatment landscape for acute stroke. The decline in prenotification rates among later - presenting strokes likely reflects EMS provider knowledge regarding alteplase treatment windows but may be a problem 41 for patients who could still be candidates for EVT for LVO stroke despite delays in presentation. Therefore, maximiz ing the utility of this metric and setting appropriate benchmarks will require focus not only on increasing the frequency of prenotificatio n s and consistency of documentation , but standardization of the appropriate clinical context for a nd content to be communicated during prenotification . This need is likely to become even more acute as stroke systems of care increasingly rely on EMS to mak e disposition determinations based on stroke severity to triage potential LVO strokes . 133 The Role of EMS Stroke Recognition The EMS stroke recognition metric is qualitatively different from the other 5 performance measures in that it does not measure a specific action or process. Rather, it serves an indicator that EMS correctly identified the medical emergency they were trans porting. As such it may be the most useful overall marker of optimal performance as appropriate EMS recognition of stroke likely influences both prehospital care and hospital response . Previous studies documented clinical characteristics that predict accurate stroke recognition, including higher stroke severity, having commonly recognized presenting symptoms such as unilateral weakness, and early presentation following symptom onset. 85, 92, 94, 95 G iven that stroke screening tools were developed specifically to facilitate appropriate stroke recognition by EMS, we had expected to find a high degree of association between PSS documentat ion and EMS stroke recognition , as has been demonstrated in previous studies . 86, 88, 92 Although there was a modestly higher correlation between these two metric s than the others, the degree of correlation was still fairly modest (phi correlation coefficient = 0.379) . Since PSS documentation is not a goal unto itself, but is meant to facilitate appropriate recognition, f urther study is needed to parse out the use and impact of PSS in the real world . Given the very high contribution of agency - level variation to compliance variation overall, it is likely that differences in EMS medical record software or data upload processes are obscuring the relationship 42 between PSS and recognition. Performance may b e improved through a combination of e ducating providers regarding content and use of PSS , ensuring screening is performed in ambiguous presentations , and improving data entry and process for uploading of data into MI - EMSIS. The value of EMS recognition as a marker of optimal EMS stroke care is demonstrated by the fact that this measure had the strongest association with all three hospital - based outcomes. Furthermore, our previous work demonstrated a close relationship between EMS stroke recognition and co mpliance with nearly all prehospital quality measures. 84 Since PSS documentation, glucose check, minimizing OST, documentation of LKW, and hospital prenotification are all contained in suspected stroke transport EMS protocols, 79 it stands to reason that recognition of stroke by EMS would naturally result in greater prehospital quality meas ure compliance as well as superior care following hospital arrival . Implications for EMS in Michigan Taken together, th is analysis suggests that there are several reasonable targets for prehospital quality improvement in stroke care. Existing practices encouraged by clinical guidelines 45, 64, 65 S suspected stroke transfer protocol 79 are followed to varying degrees among EMS - transported strokes. Investigating data entry and upload practices at EMS agencies and hospitals with lower performance on metrics such LKW documentation, PSS d ocumentation, and prenotification documentation may identify technical or clerical issues that may be corrected to ensure accurate, reliable measurements of these performance measures. Such a technical fix could have a dramatic effect on observed complian ce levels. Improving care delivered by EMS in the field will likely require a multi - faceted approach. Previous work has identified gaps in EMS provider stroke knowledge, suggesting a role for EMS education. 59, 134 Studies of educational initiatives have generally demonstrated favorable results 43 in terms of post - training knowledge 60, 135 and, in a small number of studies, improved EMS recognition in the field 98 or prenotification rates. 98, 136 Based on the findings of predictors of EMS performance, education around atypical stroke presentations and SAH may be of particular benefit . An important barrier to ideal EMS stroke care is the fact that EMS providers are typically unable to continue to follow a patient after hospital arrival. As such, there is little opportunity for providers to develop their clinical acumen by learning diagnosis , clinical course, or outcome . Lack of or suboptimal feedback regarding care has in fact been identified by EMS providers as a barrier to improving their perofrmance . 59, 137, 138 There is also a small body of evidence suggesting that feedback may be useful in improving prehospital care, 82, 98 although the durability of its impact is not clear. 98 In light of the success achieved by registry - based feedback, benchmarking, and continuous quality improvement for ED stroke care, 20 it seems likely that similar methods could help realize the benefits of optimal prehospital stroke care. Limitations The first major limitation of this an alysis is that, due to the observational nature of these data, the relationships identified by this analysis cannot be considered causal. It is not known to what degree EMS performance may be improved or if improvements would necessarily impact hospital o utcomes from this analysis . We have previously undertaken a small - scale EMS quality improvement in Kent County that successfully, but transiently, improved EMS recognition of stroke, prenotification rates, and speed of alteplase delivery 98 offering some reason to believe that such intervention at the state level may garner similar results. Quality improvement in the context of diverse practice patterns and regional variation will undoubtedly be more complex. Another important limitation t o this analysis is a possible lack of g eneralizability . Although state - level prehospital data collections systems are harmonized in terms of data definitions, there is undoubtedly variation in how data is collected and uploaded. No studies 44 exist to direc tly compare stroke - related data from different state - level prehospital registries . Our findings would require replication in other states prior to drawing firm conclusions. Another threat to generalizability of our findings is our use of a cohort of EMS - transported, confirmed stroke cases. Assessing stroke care using prehospital records alone (for example, a population of EMS suspected stroke cases) would be very different and could yield different estimates of compliance compared to this cohort of confi rmed stroke cases . 84 As one example, compliance rates for EMS prehospital stroke scale documentation among E MS 88 As our dataset was assembled using data from MOSAIC - participating hospitals in Michigan, another threat to generalizability is introduced by the fact that hospitals that participate in registries may perform better than or respond differently to EMS than n on - participating hospitals. This issue is common to all large - scale registry - based assessments of stroke care. In the end, this analysis is likely most reliable for informing EMS quality improvement within the population in which it was conducted. Anoth er limitation is missing data. Although MOSAIC data is periodically audited, is collected by trained abstractors, and has been demonstrated to be highly complete and accurate, 110, 139 MI - EMSIS data has none of those characteristics. In fact, a recent review of MI - EMSIS data suggested m oderately high rates of missing data for many different variables. 107 This was apparent during the matching process, when approximately 1/3 of cases that were coded by MOSAIC as having arrived by EMS were not successfully matched to an EMS record, primarily due to missing or inaccurate matching variables. 108 Despite this limitation, it appears that the matched population remained representative of the underlying population of EMS - transported strokes across a wide variety of demographic and clinical characteristics. However, it remains possible that unmeasured differences limit inferences drawn from this analysis to the target population. 45 The missingness issue leads to another limitation of this analysis. Since compliance with performance measures was determined based on documentation in MI - EMSIS, there are two potential errors : (1) EMS may have performed an action, but failed to document it , and (2) EMS documentation may have occurred, but failed to map appropriately to MI - EMSIS. In both cases, this would result in an under - estimate of actual EMS performance. Given the m oderately high rates of missing data across a variety of data elements in MI - EMSIS , 107 we expect the absolute estimates of EMS compliance are almost certainly underestimated. N evertheless, the fact that documented compliance with performance measures was consistently associated with favorable hospital care implies that despite these limitations, existing documentation retains value as an indicator of prehospital quality of care even if the absolute compliance rates are likely skewed downward. Conclusions In this database of EMS - transported strokes in Michigan, compliance with prehospital stroke performance measures is moderate, but inconsistent. Sources of variability includ e patient - level and group - level factors that may help target interventions. Nevertheless, higher compliance with prehospital performance metrics was associated with better in - hospital care. These data suggest several potential targets for quality improve ment efforts and provide evidence that compliance is associated with favorable downstream care. Quality improvement interventions should be performed in the context of s ystematic (and ideally controlled) studies to quantify the magnitude of their effect on EMS stroke care performance as well as measure t he impact of changes in EMS performance on downstream care and patient outcomes . 46 APPEDICES 47 APPENDIX A : DATA ELEMENTS AND SOURCES Data Elements Summary MI - EMSIS Data Elements EMS agency code (generic code assigned to individual EMS agencies) Dispatch Complaint = stroke/TIA (1 or 0) EMS Provider Primary or Secondary impression of stroke/TIA (1 or 0) EMS response time ( difference between EMS notifie d time and scene arrival time in minutes) EMS on - scene time ( difference between EMS scene arrival time and left scene time in minutes) EMS transport to hospital time ( difference between EMS left scene time and hospital arrival time in minutes) EMS document ation of glucose (1 or 0) EMS documentation of prehospital stroke scale (1 or 0) EMS documentation of last known well time (1 or 0) MOSAIC Data elements Destination hospital code Sex (male, female, missing) Race (white, black, other/not recorded) Age category (<60, 60 - 69, 70 - 79, 80 - 89, 90 or greater) Final d ischarge Diagnosis (ischemic stroke or TIA , intracerebral hemorrhage, subarachnoid hemorrhage) First documented NIHSS following arrival (0 - 5, 6 - 11, 12 - 20, 21 or greater, missing) Time from LKW to ho spital arrival in minutes (0 - 120, 121 - 360, 631 - 720, >720, missing) Door - to - CT time (time from arrival at hospital to CT performance in minutes) Delivery of IV alteplase (1 or 0) Door - to - needle (time from hospital arrival to delivery of alteplase bolus) Yea r in which event occurred (2018 or 2019) Documentation of EMS hospital prenotification (1 or 0) 48 APPENDIX B : SAMPLE STATA PROGRAMS Stata programs for selected mixed effects logistic regression models Multivariable crossed random effects models for clin ical/demographic predictors of compliance melogit pssdoc i.agecat female i.nonwhite i.stroketype2 i.nihsscat i.otdcat || _all: R.agency || _all: R.dest, or melogit gluc i.agecat female i.nonwhite i.stroketype2 i.nihsscat i.otdcat || _all: R.agency || _all: R.dest, or melogit emsstroke i.agecat female i.nonwhite i.stroketype2 i.nihsscat i.otdcat || _all: R.agency || _all: R.dest, or melogit lkwdoc i.agecat female i.nonwhite i.stroketype2 i.nihsscat i.otdcat || _all: R.agency || _all: R.dest, or melogit ost15 i.agecat female i.nonwhite i.stroketype2 i.nihsscat i.otdcat || _all: R.agency || _all: R.dest, or melogit prenot_hosp i.agecat female i.nonwhite i.stroketype2 i.nihsscat i.otdcat || _all: R.agency || _all: R.dest, or Multivariable random intercept models for association between EMS compliance and early CT among all strokes xtset ahpair xtlogit dtct25 pssdoc i.agecat female i.nonwhite i.stroketype2 i.nihsscat i.otdcat , or nolog xtlogit dtct25 gluc i.agecat female i.nonwhite i.stroketype2 i.nihsscat i.otdcat , or nolog xtlogit dtct25 emsstroke i.agecat female i.nonwhite i.stroketype2 i.nihsscat i.otdcat , or nolog xtlogit dtct25 ost15 i.agecat female i.nonwhite i.stroketype2 i.nihsscat i.otdcat , or nolog xtlogit dtct25 lkwdoc i.agecat female i.nonwhite i.stroketype2 i.nihsscat i.otdcat , or nolog xtlogit dtct25 prenot_hosp i.agecat female i.nonwhite i.stroketype2 i.nihsscat i.otdcat , or nolog 49 APPENDIX C : COMPARISON OF DIFFERENT RANDOM EFFECTS Table 9: Comparison of logistic regression models for PSS compliance utilizing different random effects . Covariate Bivariate (no random effect) Crossed Random Effects Agency - Hospital Pair Agency alone Hospital alone Multivariable Model with No Random Effec ts Age <60 Ref Ref Ref Ref Ref Ref 60 - 69 1.37 (1.16 - 1.61) 1.4 (1.14 - 1.71) 1.37 (1.12 - 1.68) 1.39 (1.14 - 1.7) 1.37 (1.14 - 1.64) 1.34 (1.12 - 1.59) 70 - 79 1.37 (1.17 - 1.61) 1.31 (1.08 - 1.6) 1.32 (1.08 - 1.61) 1.3 (1.07 - 1.58) 1.32 (1.1 - 1.57) 1.27 (1.07 - 1.5) 80 - 89 1.34 (1.14 - 1.57) 1.24 (1.01 - 1.51) 1.21 (0.99 - 1.49) 1.22 (1 - 1.49) 1.25 (1.05 - 1.5) 1.18 (0.99 - 1.4) 1.38 (1.13 - 1.68) 1.33 (1.03 - 1.71) 1.29 (1 - 1.68) 1.3 (1.01 - 1.68) 1.35 (1.07 - 1.69) 1.23 (0.99 - 1.53) Female 0.94 (0.85 - 1.04) 0.93 (0.81 - 1.05) 0.92 (0.81 - 1.05) 0.93 (0.81 - 1.05) 0.96 (0.85 - 1.07) 0.97 (0.87 - 1.08) Race White Ref Ref Ref Ref Ref Ref Black 0.64 (0.56 - 0.72) 0.96 (0.79 - 1.17) 0.95 (0.77 - 1.16) 0.96 (0.79 - 1.17) 0.92 (0.77 - 1.09) 0.67 (0.59 - 0.77) Other/missing 1.02 (0.81 - 1.28) 1.04 (0.78 - 1.38) 1.06 (0.79 - 1.42) 1.06 (0.8 - 1.41) 1.01 (0.78 - 1.31) 1.18 (0.93 - 1.5) Stroke Subtype IS/TIA Ref Ref Ref Ref Ref Ref SAH 0.33 (0.24 - 0.47) 0.57 (0.38 - 0.86) 0.58 (0.39 - 0.87) 0.57 (0.38 - 0.84) 0.59 (0.4 - 0.85) 0.55 (0.39 - 0.79) ICH 0.7 (0.6 - 0.81) 1.12 (0.91 - 1.37) 1.12 (0.91 - 1.38) 1.1 (0.9 - 1.35) 1.05 (0.88 - 1.26) 0.94 (0.79 - 1.11) NIHSS 0 - 6 Ref Ref Ref Ref Ref Ref 6 - 11 1.52 (1.31 - 1.76) 1.69 (1.41 - 2.02) 1.62 (1.35 - 1.94) 1.72 (1.44 - 2.05) 1.48 (1.27 - 1.74) 1.49 (1.28 - 1.73) 12 - 20 1.47 (1.25 - 1.73) 1.67 (1.36 - 2.04) 1.64 (1.34 - 2) 1.68 (1.38 - 2.06) 1.48 (1.24 - 1.77) 1.44 (1.22 - 1.7) >20 1.04 (0.86 - 1.26) 1.03 (0.82 - 1.3) 1.02 (0.81 - 1.29) 1.04 (0.82 - 1.31) 1 (0.81 - 1.23) 1.03 (0.85 - 1.25) Missing 0.36 (0.3 - 0.44) 0.4 (0.32 - 0.51) 0.38 (0.3 - 0.49) 0.42 (0.33 - 0.53) 0.43 (0.35 - 0.54) 0.48 (0.39 - 0.59) LKW to Door 0 - 120 Ref Ref Ref Ref Ref Ref 121 - 360 0.9 (0.77 - 1.07) 0.81 (0.66 - 0.99) 0.82 (0.67 - 1) 0.81 (0.67 - 0.99) 0.86 (0.72 - 1.03) 0.89 (0.75 - 1.05) 361 - 720 0.74 (0.62 - 0.89) 0.69 (0.55 - 0.86) 0.7 (0.56 - 0.88) 0.69 (0.55 - 0.86) 0.71 (0.59 - 0.87) 0.74 (0.62 - 0.9) >720 0.57 (0.49 - 0.66) 0.52 (0.44 - 0.62) 0.53 (0.45 - 0.63) 0.52 (0.43 - 0.61) 0.6 (0.52 - 0.7) 0.61 (0.53 - 0.71) Missing 0.34 (0.29 - 0.4) 0.28 (0.23 - 0.34) 0.27 (0.22 - 0.34) 0.28 (0.23 - 0.34) 0.34 (0.29 - 0.41) 0.4 (0.34 - 0.48) Agency ICC 0.52 0.55 n/a n/a n/a n/a Destination ICC 0.01 0.01 n/a n/a n/a n/a ICC n/a n/a 0.52 (0.44 - 0.59) 0.58 (0.48 - 0.67) 0.14 (0.08 - 0.21) n/a 50 REFERENCES 51 REFERENCES 1. Krishnamurthi RV, Ikeda T, Feigin VL. Global, Regional and Country - Specific Burden of Ischaemic Stroke, Intracerebral Haemorrhage and Subarachnoid Haemorrhage: A Systematic Analysis of the Global Burden of Disease Study 2017. Neuroepidemiology . 2020;54(2): 171 - 179. 2. Virani SS, Alonso A, Benjamin EJ, et al. Heart Disease and Stroke Statistics: 2020 Update: A Report From the American Heart Association. Circulation . 2020;141(9):e139 - e596. 3. Skolarus LE, Meurer WJ, Shanmugasundaram K, Adelman EE, Scott PA , Burke JF. Marked Regional Variation in Acute Stroke Treatment Among Medicare Beneficiaries. Stroke . Jul 2015;46(7):1890 - 6. 4. Seabury S, Bognar K, Xu Y, Huber C, Commerford SR, Tayama D. Regional disparities in the quality of stroke care. Am J Emerg Me d . Sep 2017;35(9):1234 - 1239. 5. Runchey S, McGee S. Does This Patient Have a Hemorrhagic Stroke? Clinical Findings Distinguishing Hemorrhagic Stroke From Ischemic Stroke. JAMA: The Journal of the American Medical Association . 2010;303(22):2280 - 2286. 6. Hemphill JC, 3rd, Greenberg SM, Anderson CS, et al. Guidelines for the Management of Spontaneous Intracerebral Hemorrhage: A Guideline for Healthcare Professionals From the American Heart Association/American Stroke Association. Stroke . Jul 2015;46(7):2 032 - 60. 7. The National Institute of Neurological Disorders and Stroke rt - PA Stroke Study Group. Tissue Plasminogen Activator for Acute Ischemic Stroke. N Engl J Med . December 14, 1995 1995;333(24):1581 - 1588. 8. Engelstein E, Margulies J, Jeret JS. Lac k of t - PA use for acute ischemic stroke in a community hospital: High incidence of exclusion criteria. The Am J Emerg Med . 2000;18(3):257 - 260. 9. Barber PA, Zhang J, Demchuk AM, Hill MD, Buchan AM. Why are stroke patients excluded from TPA therapy? An an alysis of patient eligibility. Neurology . Apr 24 2001;56(8):1015 - 20. 10. Katzan IL, Hammer MD, Hixson ED, Furlan AJ, Abou - Chebl A, Nadzam DM. Utilization of intravenous tissue plasminogen activator for acute ischemic stroke. Arch Neurol . Mar 2004;61(3):3 46 - 11. California Acute Stroke Pilot Registry (CASPR) Investigators. Prioritizing interventions to improve rates of thrombolysis for ischemic stroke. Neurology . Feb 22 2005;64(4):654 - 9. 12. Brown DL, Barsan WG, Lisabeth LD, Gallery ME, Morgenstern LB. Survey of emergency physicians about recombinant tissue plasminogen activator for acute ischemic stroke. Ann Emerg Med . Jul 2005;46(1):56 - 60. 52 13. Katzan IL, Furlan AJ, Lloyd LE, et al. Use of Tissue - Type Plasminogen Activator for Acute Ischemic Stroke: Th e Cleveland Area Experience. JAMA . March 1, 2000 2000;283(9):1151 - 1158. 14. Olson DM, Constable M, Britz GW, et al. A qualitative assessment of practices associated with shorter door - to - needle time for thrombolytic therapy in acute ischemic stroke. J Neu rosci Nurs . Dec 2011;43(6):329 - 36. 15. Institute of Medicine Committee on Quality of Health Care in America. To Err is Human: Building a Safer Health System. In: Kohn LT, Corrigan JM, Donaldson MS, eds. National Academies Press (US) 2000. 16. The Paul Co verdell Prototype Registries Writing Group*. Acute Stroke Care in the US: Results from 4 Pilot Prototypes of the Paul Coverdell National Acute Stroke Registry. Stroke . June 1, 2005 2005;36(6):1232 - 1240. 17. Fonarow GC, Reeves MJ, Smith EE, et al. Charact eristics, Performance Measures, and In - Hospital Outcomes of the First One Million Stroke and Transient Ischemic Attack Admissions in Get With The Guidelines - Stroke. Circulation: Cardiovascular Quality and Outcomes . 2010;3(3):291 - 302. 18. Reeves MJ, Parker C, Fonarow GC, Smith EE, Schwamm LH. Development of Stroke Performance Measures: Definitions, Methods, and Current Measures. Stroke . July 1, 2010 2010;41(7):1573 - 1578. 19. Task Force Members, Schwamm LH, Pancioli A, et al. Recommendatio ns for the Establishment of Stroke Systems of Care: Recommendations From the American Stroke Association's Task Force on the Development of Stroke Systems. Stroke . March 1, 2005 2005;36(3):690 - 703. 20. Fonarow GC, Zhao X, Smith EE, et al. Door - to - needle times for tissue plasminogen activator administration and clinical outcomes in acute ischemic stroke before and after a quality improvement initiative. JAMA . 2014;311(16):1632 - 1640. 21. Sauser K, Levine DA, Nickles AV, Reeves MJ. Hospital variation in thrombolysis times among patients with acute ischemic stroke: the contributions of door - to - imaging time and imaging - to - needle time. JAMA Neurol . Sep 2014;71(9):1155 - 61. 22. Messe SR, Khatri P, Reeves MJ, et al. Why are acute ischemic stroke patients not receiving IV tPA? Results from a national registry. Neurology . Sep 14 2016;87(15):1565 - 74. 23. Domino JS, Baek J, Meurer WJ, et al. Emerging temporal trends in tissue plasminogen activator use: Results from the BASIC project. Neurology . Nov 22 2016;87(21):2184 - 2191. 24. Lansberg MG, Schrooten M, Bluhmki E, Thijs VN, Saver JL. Treatment Time - Specific Number Needed to Treat Estimates for Tissue Plasminogen Activator Therapy in Acute Stroke B ased on Shifts Over the Entire Range of the Modified Rankin Scale. Stroke . June 1, 2009 2009;40(6):2079 - 2084. 53 25. Saver JL, Smith EE, Fonarow GC, et al. The "Golden Hour" and Acute Brain Ischemia: Presenting Features and Lytic Therapy in >30 000 Patients Arriving Within 60 Minutes of Stroke Onset. Stroke . July 1, 2010 2010;41(7):1431 - 1439. 26. Saqqur M, Tsivgoulis G, Molina CA, et al. Symptomatic intracerebral hemorrhage and recanalization after IV rt - PA: a multicenter study. Neurology . Oct 21 2008;71(1 7):1304 - 12. 27. Hui W, Wu C, Zhao W, et al. Efficacy and Safety of Recanalization Therapy for Acute Ischemic Stroke With Large Vessel Occlusion: A Systematic Review. Stroke . Jul 2020;51(7):2026 - 2035. 28. Ospel JM, Menon BK, Demchuk AM, et al. Clinical C ourse of Acute Ischemic Stroke Due to Medium Vessel Occlusion With and Without Intravenous Alteplase Treatment. Stroke . Nov 2020;51(11):3232 - 3240. 29. Levine GN, Bates ER, Blankenship JC, et al. 2015 ACC/AHA/SCAI Focused Update on Primary Percutaneous Cor onary Intervention for Patients With ST - Elevation Myocardial Infarction: An Update of the 2011 ACCF/AHA/SCAI Guideline for Percutaneous Coronary Intervention and the 2013 ACCF/AHA Guideline for the Management of ST - Elevation Myocardial Infarction. Circulat ion . 2016;133(11):1135 - 1147. 30. Broderick JP, Palesch YY, Demchuk AM, et al. Endovascular Therapy after Intravenous t - PA versus t - PA Alone for Stroke. New En gl J Med . 2013;368(10):893 - 903. 31. Ciccone A, Valvassori L, Nichelatti M, et al. Endovascular Treatment for Acute Ischemic Stroke. New England Journal of Medicine . 2013;368(10):904 - 913. 32. Kidwell CS, Jahan R, Gornbein J, et al. A Trial of Imaging Selection and Endovascular Treatment for Ischemic Stroke. New Engl J Med . 2013;368(10):914 - 923. 3 3. Berkhemer OA, Fransen PSS, Beumer D, et al. A Randomized Trial of Intraarterial Treatment for Acute Ischemic Stroke. New Engl J Med . 2015;372(1):11 - 20. 34. Goyal M, Menon BK, van Zwam WH, et al. Endovascular thrombectomy after large - vessel ischaemic st roke: a meta - analysis of individual patient data from five randomised trials. The Lancet . 2016;387(10026):1723 - 31. 35. Snelling B, McCarthy DJ, Chen S, et al. Extended Window for Stroke Thrombectomy. J Neurosci Rural Pract . Apr - Jun 2019;10(2):294 - 300. 36. Sarraj A, Savitz S, Pujara D, et al. Endovascular Thrombectomy for Acute Ischemic Strokes: Current US Access Paradigm s and Optimization Methodology. Stroke . Apr 2020;51(4):1207 - 1217. 37. Saver JL, Goyal M, van der Lugt A, et al. Time to Treatment With Endovascular Thrombectomy and Outcomes From Ischemic Stroke: A Meta - analysis. JAMA . Sep 27 2016;316(12):1279 - 88. 38. Jahan R, Saver JL, Schwamm LH, et al. Association Between Time to Treatm ent With Endovascular Reperfusion Therapy and Outcomes in Patients With Acute Ischemic Stroke Treated in Clinical Practice. JAMA . Jul 16 2019;322(3):252 - 263. 54 39. Dastmalchi YS, Oostema JA. The Yield of Multimodal Computed Tomography among Emergency Depar tment Patients with Suspected Large Vessel Occlusion Stroke. J Stroke Cerebrovasc Dis . Sep 4 2019:104353. 40. Shah S, Xian Y, Sheng S, et al. Use, Temporal Trends, and Outcomes of Endovascular Therapy After Interhospital Transfer in the United States. Cir culation . Mar 26 2019;139(13):1568 - 1577. 41. Nikoubashman O, Pauli F, Schurmann K, et al. Transfer of stroke patients impairs eligibility for endovascular stroke treatment. Journal of neuroradiology Journal de neuroradiologie . Feb 2018;45(1):49 - 53. 42. Schwamm LH, Audebert HJ, Amarenco P, et al. Recommendations for the Implementation of Telemedicine Within Stroke Systems of Care: A Policy Statement From the American Heart Association. Stroke . July 1, 2009 2009;40(7):2635 - 2660. 43. Goyal M, Ospel JM. S troke Systems of Care: Current State of Affairs and Future Directions. Stroke . Jul 2020;51(7):1928 - 1931. 44. McTaggart RA, Holodinsky JK, Ospel JM, et al. Leaving No Large Vessel Occlusion Stroke Behind: Reorganizing Stroke Systems of Care to Improve Time ly Access to Endovascular Therapy. Stroke . Jul 2020;51(7):1951 - 1960. 45. Powers WJ, Rabinstein AA, Ackerson T, et al. Guidelines for the Early Management of Patients With Acute Ischemic Stroke: 2019 Update to the 2018 Guidelines for the Early Management of Acute Ischemic Stroke: A Guideline for Healthcare Professionals From the American Heart Association/American Stroke Association. Stroke . 2019;50(12):e344 - e418. 46. Leifer D, Bravata DM, Connors JJ, III, et al. Metrics for Measuring Quality of Care in Comprehensive Stroke Centers: Detailed Follow - Up to Brain Attack Coalition Comprehensive Stroke Center Recommendations: A Statement for Healthcare Professionals From the American Heart Association/American Stroke Association. Stroke . January 13, 2011 2011: STR.0b013e318208eb99. 47. Asaithambi G, Tong X, Lakshminarayan K, Coleman King SM, George MG, Odom EC. Emergency Medical Services Utilization for Acute Stroke Care: Analysis of the Paul Coverdell National Acute Stroke Program, 2014 - 2019. Prehosp Emerg Ca re . Jan 19 2021:1 - 9. doi:10.1080/10903127.2021.1877856 48. Gache K, Couralet M, Nitenberg G, Leleu H, Minvielle E. The Role of Calling EMS Versus Using Private Transportation in Improving the Management of Stroke in France. Prehospital Emergency Care . 201 3;17(2):217 - 222. 49. Ekundayo OJ, Saver JL, Fonarow GC, et al. Patterns of Emergency Medical Services Use and Its Association With Timely Stroke Treatment: Findings From Get With the Guidelines - Stroke. Circulation: Cardiovascular Quality and Outcomes . May 1, 2013 2013;6(3):262 - 269. 50. Kim DG, Kim YJ, Shin SD, et al. Effect of emergency medical service use on time interval from symptom onset to hospital admission for definitive care among patients with 55 intracerebral hemorrhage: a multicenter observational study. Clinical and E xperimental E mergency M edicine . Sep 2017;4(3):168 - 177. 51. Kim S, Shin SD, Ro YS, et al. Effect of Emergency Medical Services Use on Hospital Outcomes of Acute Hemorrhagic Stroke. Prehosp Emerg Care . May - Jun 2016;20(3):324 - 32. 52. Sayre MR, White LJ, Brown LH, McHenry SD. The National EMS Research Strategic Plan. Prehospital Emergency Care . 2005;9(3):255 - 266. 53. Pepe PE, Zachariah BS, Sayre MR, Floccare D. Ensuring the chain of recovery for stroke in your community. Acad Emerg Med . Apr 1998;5(4):352 - 8. 54. Glickman SW, Kit Delgado M, Hirshon JM, et al. Defining and Measuring Successful Emergency Care Networks: A Research Agenda. Acad Emerg Me d . 2010;17(12):1297 - 1305. 55. Kaji AH, Lewis RJ, Beavers - May T, et al. Summary of NIH Medical - Surgical Emergency Research Roundtable Held on April 30 to May 1, 2009. doi: DOI: 10.1016/j.annemergmed.2010.03.014. Ann Emerg Me d . 2010;56(5):522 - 537. 56. Committee on the Future of Emergency Care in the Uniited States Health System, Institute of Medicine. Emergency Medical Services: at the Crossroads . Washington, DC, USA: National Academy Press ; 200 6 . 57. Delbridge TR, Bailey B, Chew Jr JL, et al. EMS Agenda for the Future: Where We Are ... Where We Want to Be. doi: 10.1016/S0196 - 0644(98)70316 - 6. Annals of emergency medicine . 1998;31(2):251 - 263. 58. Crocco TJ, Kothari RU, Sayre MR, Liu T. A nationwide prehospital stroke surv ey. Prehosp Emerg Care . Jul - Sep 1999;3(3):201 - 6. 59. Li T, Munder SP, Chaudhry A, Madan R, Gribko M, Arora R. Emergency Medical Services Providers' Knowledge, Practices, And Barriers To Stroke Management. Open A ccess E mergency M edicine : OAEM . 2019;11:29 7 - 303. 60. DiBiasio EL, Jayaraman MV, Oliver L, et al. Emergency medical systems education may improve knowledge of pre - hospital stroke triage protocols. J Neurointerv Surg . Apr 2020;12(4):370 - 373. 61. Pap R, Lockwood C, Stephenson M, Simpson P. Indica tors to measure prehospital care quality: a scoping review. JBI database of systematic reviews and implementation reports . Nov 2018;16(11):2192 - 2223. 62. Howard I, Cameron P, Wallis L, Castren M, Lindstrom V. Quality Indicators for Evaluating Prehospital Emergency Care: A Scoping Review. Prehospital and disaster medicine . Feb 2018;33(1):43 - 52. 63. Taymour RK, Abir M, Chamberlin M, et al. Policy, Practice, and Research Agenda for Emergency Medical Services Oversight: A Systematic Review and Environmental Scan. Prehospital and disaster medicine . Feb 2018;33(1):89 - 97. 56 64. Jauch EC, Saver JL, Adams HP , et al. Guidelines for the Early Management of Patients With Acute Ischemic Stroke: A Guideline for Healthcare Professionals From the American Heart Association/American Stroke Association. Stroke . March 1, 2013 2013;44(3):870 - 947. 65. Acker JE, III, Pa ncioli AM, Crocco TJ, et al. Implementation Strategies for Emergency Medical Services Within Stroke Systems of Care: A Policy Statement From the American Heart Association/ American Stroke Association Expert Panel on Emergency Medical Services Systems and the Stroke Council. Stroke . November 1 2007;38(11):3097 - 3115. 66. Patel MD, Brice JH, Moss C, et al. An Evaluation of Emergency Medical Services Stroke Protocols and Scene Times. Prehospital Emergency Care . 2014 ;18(1):15 - 21 67. Lin CB, Peterson ED, Smith EE, et al. Emergency Medical Service Hospital Prenotification Is Associated With Improved Evaluation and Treatment of Acute Ischemic Stroke. Circulation: Cardiovascular Quality and Outcomes . 2012;5(4):514 - 522. 68. Mosley I, Nic ol M, Donnan G, Patrick I, Kerr F, Dewey H. The Impact of Ambulance Practice on Acute Stroke Care. Stroke . October 1, 2007 2007;38(10):2765 - 2770. doi:10.1161/strokeaha.107.483446 69. Kim DH, Nah HW, Park HS, et al. Impact of Prehospital Intervention on De lay Time to Thrombolytic Therapy in a Stroke Center with a Systemized Stroke Code Program. J Stroke Cerebrovasc Dis . Jul 2016;25(7):1665 - 70. 70. Abdullah AR, Smith EE, Biddinger PD, Kalenderian D, Schwamm LH. Advance hospital notification by EMS in acute stroke is associated with shorter door - to - computed tomography time and increased likelihood of administration of tissue - plasminogen activator. Prehosp Emerg Care . Oct - Dec 2008;12(4):426 - 31. 71. Kim S, Lee S, Bae H, et al. Pre - hospital notification reduc ed the door - to - needle time for iv t - PA in acute ischaemic stroke. European Journal of Neurology . 2009;16(12):1331 - 1335. 72. Patel MD, Rose KM, O'Brien EC, Rosamond WD. Prehospital Notification by Emergency Medical Services Reduces Delays in Stroke Evalua tion. Stroke . 2011;42(8):2263 - 2268. 73. Kothari R, Pancioli A, Liu T, Brott T, Broderick J. Cincinnati Prehospital Stroke Scale: reproducibility and validity. Ann Emerg Med . 1999;33(4):373 - 8. 74. Kidwell CS, Starkman S, Eckstein M, Weems K, Saver JL. Identifying Stroke in the Field : Prospective Validation of the Los Angeles Prehospital Stroke Screen (LAPSS). Stroke . January 1, 2000 2000;31(1):71 - 76. 75. Brandler ES, Sharma M, Sinert RH, Levine SR. Prehospital stroke scales in urban environments: A systematic review. Neurology . May 21 2014;82(24):2241 - 9. 76. National Model EMS Guidelines. National Association of EMS Officials. Updated January 2019. https://nasemso.org/proje cts/model - ems - clinical - guidelines/ . Accessed January 22, 2021. 57 77. Brice JH, Evenson KR, Lellis JC, et al. Emergency Medical Services Education, Community Outreach, andProtocols for Stroke andChest Pain in North Carolina. Prehospital Emergency Care . 2008;12(3):366 - 371. 78. Glober NK, Sporer KA, Guluma KZ, et al. Acute St roke: Current Evidence - based Recommendations for Prehospital Care. The W estern J ournal of E mergency M edicine . Mar 2016;17(2):104 - 28. 79. Michigan State Protocol 3.2: Stroke/Suspected Stroke . Michigan Department of Health and Human Service ; 2018. h ttps:// www.michigan.gov/documents/mdhhs/Section_3_Adult_ Specific_ t reatment_613177_7.pdf. Accessed April 20, 2021. 80. Lin CB, Peterson ED, Smith EE, et al. Patterns, Predictors, Variations, and Temporal Trends in Emergency Medical Service Hospital Prenotificat ion for Acute Ischemic Stroke. Journal of the American Heart Association . August 24, 2012 2012;1(4):e002345. 81. Schwartz J, Dreyer RP, Murugiah K, Ranasinghe I. Contemporary Prehospital Emergency Medical Services Response Times for Suspected Stroke in t he United States. Prehosp Emerg Care . Mar 8 2016;20(5):560 - 565. 82. Choi B, Tsai D, McGillivray CG, Amedee C, Sarafin J - A, Silver B. Hospital - Directed Feedback to Emergency Medical Services Improves Prehospital Performance. Stroke . July 1, 2014 2014;45(7 ):2137 - 2140. 83. Stroke - 01 Measure Package. National EMS Qaultiy Alliance. https://www.nemsqa.org/wp - content/uploads/2020/11/J. - NEMSQA - Stroke - 01.pdf . Accessed April 20, 2021. 84. Oostema JA, Nasiri M, Chassee T, Reeves MJ. The Quality of Prehospital Ischemic Stroke Care: Compliance with Guidelines and Impact on In - hospital Stroke Response. J Stroke Cerebrovasc Dis . Nov - Dec 2014;23(10):2773 - 9. 85. Bra ndler ES, Sharma M, McCullough F, et al. Prehospital Stroke Identification: Factors Associated with Diagnostic Accuracy. Journal of Stroke and Cerebrovascular Diseases . 2015;24(9):2161 - 2166. 86. Oostema JA, Konen J, Chassee T, Nasiri M, Reeves MJ. Clinic al Predictors of Accurate Prehospital Stroke Recognition. Stroke . Apr 28 2015;46(6):1513 - 7. 87. Hansson PO, Andersson Hagiwara M, Herlitz J, Brink P, Wireklint Sundstrom B. Prehospital assessment of suspected stroke and TIA: An observational study. Acta neurologica Scandinavica . Aug 2019;140(2):93 - 99. 88. Mould - Millman NK, Meese H, Alattas I, et al. Accuracy of Prehospital Identification of Stroke in a Large Stroke Belt Municipality. Prehosp Emerg Care . Nov - Dec 2018;22(6):734 - 742. 89. Abboud ME, Band R, Jia J, et al. Recognition of Stroke by EMS is Associated with Improvement in Emergency Department Quality Measures. Prehosp Emerg Care . May 31 2016;20(6):729 - 736. 58 90. Tennyson JC, Michael SS, Youngren MN, Reznek MA. Delayed Recognition of Acute Stroke by Emergency Department Staff Following Failure to Activate Stroke by Emergency Medical Services. The W estern J ournal of E mergency M edicine . Mar 2019;20(2):342 - 350. 91. Ramanujam P, Guluma KZ, Castillo EM, et al. Accuracy of Stroke Recognition by Emergency Medical Dispatchers andParamedics San Diego Experience. Prehospital Emergency Care . 2008;12(3):307 - 313 92. Gropen TI, Gokaldas R, Poleshuck R, et al. Factors related t o the sensitivity of emergency medical service impression of stroke. Prehosp Emerg Care . Jul - Sep 2014;18(3):387 - 92. 93. Sharma M, Helzner E, Sinert R, Levine SR, Brandler ES. Patient characteristics affecting stroke identification by emergency medical se rvice providers in Brooklyn, New York. Internal and E mergency M edicine . Mar 2016;11(2):229 - 36. 94. Jia J, Band R, Abboud ME, et al. Accuracy of Emergency Medical Services Dispatcher and Crew Diagnosis of Stroke in Clinical Practice. Front Neurol . 2017;8:466. 95. Andersson E, Bohlin L, Herlitz J, Sundler AJ, Fekete Z, Andersson Hagiwara M. Prehospital Identification of Patients with a Final Hospital Diagnosis of Stroke. Prehospital and D isaster M edicine . Feb 2018;33(1):63 - 70. 96. Madhok DY, Keenan KJ, Cole SB, Martin C, Hemphill JC, 3rd. Prehospital and Emergency Department - Focused Mission Protocol Im proves Thrombolysis Metrics for Suspected Acute Stroke Patients. J Stroke Cerebrovasc Dis . Dec 2019;28(12):104423. 97. Dickson R, Nedelcut A, Nedelcut MM. Stop Stroke: A Brief Report on Door - to - Needle Times and Performance After Implementing an Acute Car e Coordination Medical Application and Implications to Emergency Medical Services. Prehospital and disaster medicine . Jun 2017;32(3):343 - 347. 98. Oostema JA, Chassee T, Baer W, Edberg A, Reeves MJ. Brief Educational Intervention Improves Emergency Medica l Services Stroke Recognition. Stroke . May 2019;50(5):1193 - 1200. 99. Brown A, Onteddu S, Sharma R, et al. A Pilot Study Validating Video - Based Training on Pre - Hospital Stroke Recognition. Journal of N eurology, N eurosurgery & P sychiatry R esearch . Jan - Jun 2019;1(1) : Epub 2019 Jan 17 100. The History of NEMSIS. NHTSA's Office of EMS. https://nemsis.org/what - is - nemsis/history - of - nemsis/ . Accessed April 21, 2021. 101. Recommended guidelines for uniform reporting of data from out - of - hospital cardiac arrest: the 'Utstein style'. Prepared by a Task Force of Representatives from the European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, Australian Resu scitation Council. Resuscitation . Aug 1991;22(1):1 - 26. 102. Dawson DE. National Emergency Medical Services Information System (NEMSIS). Prehosp Emerg Care . Jul - Sep 2006;10(3):314 - 6 . 59 103. NEMSIS . Acute Stroke Translation Table. Intermountain Injury Control Research Center. https://nemsis.org/using - ems - data/performance - measures/. Accessed May 5, 2021 104. Shaeffer Z, Gohdes D, Legler J, Taillac P, Larsen B. Monitoring prehospital stroke care in Utah to assess the feasibility of using EMS data for surveillance. Prev Chronic Dis . Oct 2009;6(4):A137. 105. Williams I, Mears G, Raisor C, Wilson J. An emergency medical services toolkit for improving systems of care for stroke in North Carolina. Prev Chronic Dis . Apr 2009;6(2):A67. 106. Mears GD, Pratt D, Glickman SW, et al. The North Carolina EMS Data System : A Comprehensive Integrated Emergency Medical Services Quality Improvement Program. Prehospital Emergency Care . 2010;14(1):85 - 94. 107. Abir M, Taymour RK, Goldstick JE, et al. Data missingness in the Michigan NEMSIS (MI - EMSIS) dataset: a mixed - methods s tudy. Int J Emerg Med . Apr 14 2021;14(1):22. 108. Oostema JA, Nickles A, Reeves MJ. A Comparison of Probabilistic and Deterministic Match Strategies for Linking Prehospital and in - Hospital Stroke Registry Data. J Stroke Cerebrovasc Dis . Oct 2020;29(10):10 5151. 109. Xian Y, Fonarow GC, Reeves MJ, et al. Data quality in the American Heart Association Get With The Guidelines - Stroke (GWTG - Stroke): results from a national data validation audit. Am Heart J . Mar 2012;163(3):392 - 8, 398.e1. 110. Reeves MJ, Nickl es AV, Roberts S, Hurst R, Lyon - Callo S. Assessment of the completeness and accuracy of case ascertainment in the Michigan Stroke Registry. Circulation Cardiovascular quality and outcomes . Sep 2014;7(5):757 - 63. 111. Tibaldi FS, Verbeke G, Molenberghs G, Renard D, Van den Noortgate W, de Boeck P. Conditional mixed models with crossed random effects. Br J Math Stat Psychol . Nov 2007;60(Pt 2):351 - 65. 112. Meyers JL, Beretvas SN. The Impact of Inappropriate Modeling of Cross - Classified Data Structures. Mult ivariate Behav Res . Dec 1 2006;41(4):473 - 97. 113. Austin PC, Merlo J. Intermediate and advanced topics in multilevel logistic regression analysis. Statistics in medicine . 2017;36(20):3257 - 3277. 114. David W Hosmer SL, And Rodney X. Sturdivant. Applied L ogistic Regression . Third ed. John Wiley & Sons, Inc.; 2013. 115. Merlo J, Chaix B, Ohlsson H, et al. A brief conceptual tutorial of multilevel analysis in social epidemiology: using measures of clustering in multilevel logistic regression to investigate contextual phenomena. J Epidemiol Community Health . Apr 2006;60(4):290 - 7. 116. Sheppard JP, Mellor RM, Greenfield S, et al. The association between prehospital care and in - hospital treatment decisions in acute stroke: a cohort study. Emerg Med J . Feb 2015 ;32(2):93 - 9. 60 117. Powers WJ, Rabinstein AA, Ackerson T, et al. 2018 Guidelines for the Early Management of Patients With Acute Ischemic Stroke: A Guideline for Healthcare Professionals From the American Heart Association/American Stroke Association. Stro ke . Mar 2018;49(3):e46 - e110. 118. Li T, Cushman JT, Shah MN, Kelly AG, Rich DQ, Jones CMC. Prehospital time intervals and management of ischemic stroke patients. Am J Emerg Med . Feb 7 2020;doi:10.1016/j.ajem.2020.02.006 119. Redlener M, Olivieri P, Loo G T, et al. National Assessment of Quality Programs in Emergency Medical Services. Prehosp Emerg Care . Jan 3 2018:1 - 9. 120. Sanello A, Gausche - Hill M, Mulkerin W, et al. Altered Mental Status: Current Evidence - based Recommendations for Prehospital Care. The W estern J ournal of E mergency M edicine . May 2018;19(3):527 - 541. 121. Heemskerk JL, Domingo RA, Tawk RG, et al. Time Is Bra in: Prehospital Emergency Medical Services Response Times for Suspected Stroke and Effects of Prehospital Interventions. Mayo Clinic proceedings Mayo Clinic . Mar 10 2021;doi:10.1016/j.mayocp.2020.08.050 122. Bushnell C, Howard VJ, Lisabeth L, et al. Sex d ifferences in the evaluation and treatment of acute ischaemic stroke. Lancet Neurol . Jul 2018;17(7):641 - 650. 123. Venkat A, Cappelen - Smith C, Askar S, et al. Factors Associated with Stroke Misdiagnosis in the Emergency Department: A Retrospective Case - Co ntrol Study. Neuroepidemiology . Aug 9 2018;51(3 - 4):123 - 127. 124. Madsen TE, Khoury J, Cadena R, et al. Potentially Missed Diagnosis of Ischemic Stroke in the Emergency Department in the Greater Cincinnati/Northern Kentucky Stroke Study. Acad Emerg Med . Oc t 2016;23(10):1128 - 1135. 125. Sarraj A, Medrek S, Albright K, et al. Posterior circulation stroke is associated with prolonged door - to - needle time. International Journal of Stroke . 2015;10(5):672 - 678. 126. Newman - Toker David E, Moy E, Valente E, Coffey R, Hines Anika L. Missed diagnosis of stroke in the emergency department: a cross - sectional analysis of a large population - based sample. Diagnosis . 2014;1(2):155 - 166. 127. Oostema JA, Chassee T, Baer W, Edberg A, Reeves MJ. Educating Paramedics on the F inger - to - Nose Test Improves Recognition of Posterior Stroke. Stroke . Oct 2019;50(10):2941 - 2943. 128. Abir M, Fouche S, Lehrich J, et al. Variation in pre - hospital outcomes after out - of - hospital cardiac arrest in Michigan. Resuscitation . Jan 2021;158:201 - 207. 129. Quinn CM, Bilimoria KY, Chung JW, Ko CY, Cohen ME, Stulberg JJ. Creating Individual Surgeon Performance Assessments in a Statewide Hospital Surgical Quality Improvement Collaborative. J Am Coll Surg . Sep 2018;227(3):303 - 312.e3. 130. Nielsen V M, DeJoie - Stanton C, Song G, Christie A, Guo J, Zachrison KS. The Association between Presentation by EMS and EMS Prenotification with Receipt of Intravenous 61 Tissue - Type Plasminogen Activator in a State Implementing Stroke Systems of Care. Prehosp Emerg Ca re . Oct 2 2019:1 - 7. 131. Rostanski SK, Shahn Z, Elkind MSV, et al. Door - to - Needle Delays in Minor Stroke: A Causal Inference Approach. Stroke . Jul 2017;48(7):1980 - 1982. 132. Sheppard JP, Lindenmeyer A, Mellor RM, et al. Prevalence and predictors of hos pital prealerting in acute stroke: a mixed methods study. Emerg Med J . Jul 2016;33(7):482 - 8. 133. Jauch EC, Schwamm LH, Panagos PD, et al. Recommendations for Regional Stroke Destination Plans in Rural, Suburban, and Urban Communities From the Prehospita l Stroke System of Care Consensus Conference: A Consensus Statement From the American Academy of Neurology, American Heart Association/American Stroke Association, American Society of Neuroradiology, National Association of EMS Physicians, National Associa tion of State EMS Officials, Society of NeuroInterventional Surgery, and Society of Vascular and Interventional Neurology: Endorsed by the Neurocritical Care Society. Stroke . Mar 11 2021: doi:10.1161/strokeaha.120.033228 134. McNamara MJ, Oser C, Gohdes D, et al. Stroke knowledge among urban and frontier first responders and emergency medical technicians in Montana. J Rural Health . Spring 2008;24(2):189 - 93. 135. Oser CS, McNamara MJ, Fogle CC, Gohdes D, Helgerson SD, Harwell TS. Educational Outreach to Improve Emergency Medical Services Systems of Care for Stroke in Montana. Prehospital Emergency Care . 2010;14(2):259 - 264. 136. Gorchs - Molist M, Solà - Muñoz S, Enjo - Perez I, et al. An Online Training Intervention on Prehospital S troke Codes in Catalonia to Improve the Knowledge, Pre - Notification Compliance and Time Performance of Emergency Medical Services Professionals. International J ournal of E nvironmental R esearch and P ublic D ealth . Aug 26 2020;17(17) 137. Morrison L, Cassidy L, Welsford M, Chan TM. Clinical Performance Feedback to Paramedics: What They Receive and What They Need. AEM Education and Training . 2017;1(2):87 - 97. 138. Hodell E, Hughes SD, Corry M, et al. Paramedic Perspectives on Barriers to Prehospital Acute Str oke Recognition. Prehosp Emerg Care . Feb 8 2016:1 - 10. 139. Reeves MJ, Mullard AJ, Wehner S. Inter - rater reliability of data elements from a prototype of the Paul Coverdell National Acute Stroke Registry. BMC Neurol . Jun 11 2008;8:19.