A SECONDARY ANALYSIS OF SBHC-REPORTED MENTAL HEALTH SERVICES, STRUCTURAL CHARACTERISTICS, AND STATE LEVEL SUPPORT FROM 2005-2014 By Tatiana Elisa Bustos A THESIS Submitted to Michigan State University in partial fulfillment of the requirements for the degree of Psychology—Master of Arts 2018 ABSTRACT A SECONDARY ANALYSIS OF SBHC-REPORTED MENTAL HEALTH SERVICES, STRUCTURAL CHARACTERISTICS, AND STATE LEVEL SUPPORT FROM 2005-2014 By Tatiana Elisa Bustos Background: More than 20% of children and youth in the U.S. experience mental health difficulties, with only about 30% receiving adequate mental health treatment services. School based health centers (SBHCs)—a comprehensive service delivery model integrating physical and mental health services within school settings—reduce barriers to health services faced by low- income families and children. Given the potential of SBHCs to improve the lives of children with mental healthcare needs, it is necessary to explore the delivery of mental health (MH) services among SBHCs longitudinally, and identify key structural characteristics, networks, and state level supports that promote delivery of MH services across U.S. states. Method: Guided by the contextualist approach, secondary analyses of two longitudinal datasets (National SBHC Census & State Policy Survey) were carried out to: (1) identify the number of MH services reported to be delivered by state over time, and (2) identify inner and outer contexts of SBHCs with a MH component related to number of services. The consolidated framework for implementation (CFIR) was used to organize variables and guide interpretation of findings related to the interplay of contexts and delivery of services. Findings: Results suggest that specific inner and outer setting variables are related to more MH services from 2005 to 2014, but the variables had differential impacts on which type of MH service was delivered. Moreover, mandatory policies for state-funded SBHCs demonstrated more MH services over time than those otherwise. Discussion: Understanding the factors facilitating delivery of MH services is necessary to better inform policy efforts that can increase service access among underserved youth. ACKNOWLEDGEMENTS I would first like to acknowledge my late sister, Thalia Karina Bustos, for being my guardian and for giving me the strength to overcome all my struggles and challenges thus far. You are the only reason I am here. I would like to thank my close friends and family in Florida for their distant, but constant, support throughout my transition to Michigan and my graduate program. Thank you, to my ECO peers and professors, who have dedicated time and resources to help me reach my accomplishments. I am forever grateful for my advisor, Dr. Amy Drahota, for her support and for believing in my abilities as a scholar. I want to thank my supportive and enthusiastic thesis committee, Drs. Kaston Anderson Carpenter and Miles McNall for their contributions to the development of this project as committee members. I want to further acknowledge Dr. Deb Kashy's support in developing the analytic plan for this project and for her patience in teaching me how to successfully tackle challenging statistical approaches. Lastly, I want to thank Kevin James Graham for balancing me with his energy, everlasting patience and ongoing encouragement. Thank you! TABLE OF CONTENTS LIST OF TABLES..........................................................................................................................vi LIST OF FIGURES......................................................................................................................viii INTRODUCTION...........................................................................................................................1 School-Based Health Center Model.........................................................................2 Variations in School-Based Health Centers.............................................................4 Limitations of Mental Health Services through School Based Health Centers.......6 Organizational Context Influences SBHC Implementation.....................................7 LITERATURE REVIEW..............................................................................................................11 SBHC Mental Health Services...............................................................................12 SBHC Effectiveness...............................................................................................13 METHODS....................................................................................................................................21 state from 2005 to 2014?............................................................................21 Reported mental health services................................................................21 Data cleaning & missing data....................................................................22 RQ1. What is the number of mental health services reported to be delivered by RQ2. Which inner setting variables (structural characteristics and networks) are related to number of mental health services reported to be delivered from 2005 to 2014?.............................................................................................22 Data cleaning & missing data....................................................................23 RQ3. Which outer setting variables (e.g., funding sources, policy and standards, oversight and support) are related to the number of mental health services reported from 2005 to 2014?......................................................................24 Data cleaning & missing data....................................................................24 Participants.............................................................................................................25 Providers....................................................................................................25 Policy makers.............................................................................................25 Measures................................................................................................................25 National Census of SBHC Survey.............................................................25 SBHC State Policy Survey........................................................................26 Procedures..............................................................................................................27 School Based Health Alliance (SBHA).....................................................27 Data collection procedures.........................................................................28 Data Analysis.........................................................................................................28 Exploratory data analysis...........................................................................28 Linear mixed model (LMM) analysis........................................................30 RESULTS......................................................................................................................................33 Descriptive analyses.............................................................................................33 iv Screening & assessment services...............................................................33 Medication management services..............................................................34 Substance use treatment services...............................................................35 Referral services.........................................................................................36 Inner Setting Variables..............................................................................38 The National Census Survey-Structural Characteristics subscale..........................................................................................38 The National Census Survey- Network Communications subscale..........................................................................................39 Outer Setting Variables..............................................................................40 The State Policy Survey- Funding Sources subscale....................40 The State Policy Survey- Oversight and Support subscale...........41 The State Policy Survey- Policy and Standards subscale.............42 LMM Analyses: Inner Setting Variables & MH Services.....................................42 Screening & assessment services...............................................................42 Medication management services..............................................................44 Substance use treatment services...............................................................47 Referral services.........................................................................................49 LMM Analyses: Outer Setting Variables & MH Services....................................50 Screening & assessment services...............................................................52 Medication management services..............................................................54 Substance use treatment services...............................................................56 Referrals services.......................................................................................59 DISCUSSION................................................................................................................................61 state from 2005 to 2014?............................................................................61 RQ1. What is the number of mental health services reported to be delivered by RQ2. Which inner setting variables (structural characteristics and networks) are related to number of mental health services reported to be delivered from 2005 to 2014?.............................................................................................62 The National Census Survey - Structural Characteristics subscale...........62 The National Census Survey - Networks Communications subscale........66 RQ3. Which outer setting variables (e.g., funding sources, policy and standards, oversight and support) are related to number of mental health services reported to be delivered from 2005 to 2014?.............................................67 The State Policy Survey - Funding Sources subscale................................67 The State Policy Survey - Oversight and Support subscale.......................68 The State Policy Survey - Policy and Standards subscale.........................69 Limitations.............................................................................................................70 Conclusion.............................................................................................................70 APPENDICES...............................................................................................................................73 APPENDIX A: National SBHC Census Survey....................................................74 APPENDIX B: State Policy Survey......................................................................94 REFERENCES............................................................................................................................. 98 v LIST OF TABLES Table 1: Consolidated Framework for Implementation Research Constructs, Definitions, Variables & Dataset.............................................................................................................9 Table 2: Sample Sizes Across Time-Points...................................................................................33 Table 3: Screening & Assessment Services...................................................................................34 Table 4: Medication Management Services...................................................................................35 Table 5: Substance Use Treatment Services..................................................................................36 Table 6: Referral Services..............................................................................................................37 Table 7: Summary of Structural Characteristics by Time Point....................................................39 Table 8: Percent of Sites’ Reported Health Center Partnerships by Time.....................................40 Table 9: Summary of External Policy & Incentives by Percentage of Sites (N) and Time...................................................................................................................................41 Screening/Assessment Services, Controlled for Time.......................................................43 Controlled for Time...........................................................................................................49 Table 17: Estimated Marginal Means of Referrals by Inner Setting Variables.............................50 vi Management Services, Controlled for Time......................................................................45 Variables............................................................................................................................44 Table 10: LMM: Type III Test of Fixed Effects for Inner Setting Variables on Table 11: Estimated Marginal Means for Screening/Assessment by Inner Setting Table 12: LMM: Type III Test of Fixed Effects for Inner Setting Variables on Medication Table 13: Estimated Marginal Means for Medication Management by Inner Setting Table 14: LMM: Type III Test of Fixed Effects for Inner Setting Variables on Substance Table 15: Estimated Marginal Means of Substance Use Treatment Services by Inner Table 16: LMM: Type III Test of Fixed Effects for Inner Setting Variables on Referrals, Variables............................................................................................................................46 Use Treatment Services, Controlled for Time...................................................................47 Setting Variables................................................................................................................48 Table 18: Type III Tests of Fixed Effects for Outer Context Variables on Screening/Assessment, Controlled for Time.....................................................................51 Table 19: Type III Tests of Fixed Effects for Outer Context Variables on Medication Management, Controlled for Time....................................................................................51 Table 20: Type III Tests of Fixed Effects for Outer Context Variables on Substance Use Treatment, Controlled for Time.........................................................................................52 Table 21: Type III Tests of Fixed Effects for Outer Context Variables on Referrals, Controlled for Time...........................................................................................................52 Table 22: Estimated Marginal Means of Screening/Assessment by Outer Setting Variables............................................................................................................................54 Table 23: Estimated Marginal Means of Medication Management by Outer Setting Variables............................................................................................................................56 Table 24: Estimated Marginal Means of Substance Use Treatment by Outer Setting Variables............................................................................................................................58 Table 25: Estimated Marginal Means of Referrals by Outer Setting Variables............................60 vii LIST OF FIGURES Figure 1: Stacked Bar Count ("Yes") of School Characterization by Time..................................72 viii INTRODUCTION It is estimated that more than 20% of children and youth in the U.S. experience mental health difficulties, with only about 30% receiving adequate mental health treatment services (Bains & Diallo, 2016; Brown & Bolen, 2003; Merikangas et al., 2011; Simon, Pastor, Reuben, Huang & Goldstrom, 2015). In fact, most children between the ages of 6 to 17 years, who are in need of mental health services, do not receive treatment (Langer et al., 2015; Jensen et al., 2011; Katoaka, Zhang, & Wells, 2002; Bains, Cusson, White-Frese, & Walsh, 2017). Moreover, children from low-income families are reported to have higher rates of mental health difficulties than children from higher income families who do not experience economic hardships (Guo, Wade, & Keller, 2008; Hill, Ohmstede, & Mims, 2012). Given these rates, there is a critical need to increase access and funding of mental health services in underserved areas. School based health centers (SBHCs) have proven successful in reducing barriers to reach children and youth with the greatest level of need (Allison et al., 2007; Armbruster & Lichtman, 1999; Larson & Chapman, 2013; Mason-Jones et al., 2012). Families in low-income areas tend to face more challenges related to limited resources (e.g., availability of health care providers or clinics) and more barriers in access to quality health services (Bains & Diallo, 2016; Baquiran, Webber, & Appel, 2002), including mental health services. These barriers include transportation, language, costs of services, work schedules, availability of appointments, culturally compatible services, and insurance status (Anderson, Howarth, Vainre, Jones, & Humphrey, 2017; Agudelo-Suarez, 2012, Brown & Bolen, 2003; Gulliver, Griffiths, & Christensen, 2010; Guo, Wade, Pan, & Keller, 2010). Further, children from low-income families are more likely to be uninsured and lack a usual source of health care, which decreases their likelihood of having health professional visits 1 (Allison et al., 2007; Bains & Diallo, 2016; Baquiran et al., 2002; Bloom, Jones & Freeman, 2013). SBHCs offer a comprehensive source of care to children without medical coverage (Baquiran et al., 2002). For example, through school-based health centers, mental health services can be made affordable to families in need, either because the services are free-of-charge, low- cost, or billable to Medicaid (Brindis, Kapphahn, McCarter, & Wolfe, 1995; McNall, Lichty & Mavis, 2010). In the U.S., 70% of SBHCs include a mental health component. However, among schools with a SBHC, 30% do not have a mental health provider on staff (School-Based Health Alliance, 2016; Larson, Spetz, Brindis, & Chapman, 2017). While 70% of SBHCs can offer a range of mental health services, there remains a need for expansion of comprehensive mental health care that can be optimized with more organizational resources, such as mental health providers on staff (Larson et al., 2017). The purpose of this project is to identify key structural, network, and state level support characteristics of SBHCs that have contributed to the delivery of mental health services over time. School-Based Health Center Model School-based health centers (SBHCs), a comprehensive service delivery model that integrates physical and mental health services, responds to children’s unmet needs by increasing access to treatment and preventive services within the school setting (Baquiran et al., 2002; Harold & Harold, 1993; Langer et al., 2015; McNall et al., 2010; Silberburg & Cantor, 2008). The SBHC model was designed to function as a medical home for children in urban, low-income areas (Dowden, Calvert, Davis, & Gullota, 1997; Gullota & Noyes, 1995). The model shares a common goal to optimize students’ potential and success by increasing access to prevention and treatment services for improved health (School Based Health Alliance, 2016). In fact, SBHCs are 2 in a unique position to reduce barriers commonly faced by low-income families (Guo et al., 2008). Schools have been identified as effective primary settings for mental health care, particularly among low-income communities (Bains & Diallo, 2016; Brindis et al., 1995; Hill et al., 2012). SBHCs have an opportunity to make a direct positive impact on children’s health and education outcomes at a location where the child is present all day. SBHCs have helped reduce barriers related to stigma, conflicting work schedules, and financial obstacles by augmenting access to services that are affordable, confidential, and convenient (Guo et al., 2008; Armbuster & Lichtman, 1999; Lai, Guo, Ijadi-Maghsoodi, Puffer, & Kataoka, 2016). Since inception, SBHCs have continued to expand throughout the United States. According to the 2013-2014 SBHC Census, there are 2,315 centers in 49 of the 50 U.S. states and District of Columbia (School-Based Health Alliance, 2016). Fifty percent of SBHCs are located in urban areas, 34.6% in rural areas, and 14.2% in suburban areas. Ninety four percent of these SBHCs are located on school property, and the majority are affiliated with traditional public schools, community schools, or magnet schools. Centers can remain open annually or open only during the school year, depending on resources. SBHCs serve children in schools including all grade levels from K-12 (27.9%); high schools (9-12; 23.4%); elementary schools (15.3%); middle schools (8.8%) and non-traditional grade levels (24.6%). According to a census survey from 2013-2014, student demographics of schools with SBHCs is reported as 31% White, 30% Hispanic/Latino, and 25% Black/African American (Larson et al., 2017). Shared characteristics across the SBHCs include: a multidisciplinary team of (1) health care providers, including registered nurse, nurse practitioner, physician assistants, social workers, physicians, counselors, and other health care professionals; (2) collaboration with the school system; (3) providing a comprehensive range of services to meet physical and behavioral health needs of 3 children and youth in the community; (4) providing clinical services through a hospital, health department or medical practice; (5) requiring parents to sign written consents in order for children to receive all services provided; and (6) having an advisory board consisting of community representatives, parents, youth, family organizations for planning and oversight (School Based Health Alliance, 2016). Variations in School-Based Health Centers The seven core competencies of SBHCs (e.g., access, student focus, school integration, accountability, school wellness, systems coordination, and sustainability) help guide the development of new SBHC models. However, standards and guidelines for a center are defined by the states in which they are located. Centers may also vary structurally in center demographics, hours of operations, staffing size, networks (e.g., partnerships), sponsorships and available resources (Bains & Diallo, 2016; Brindis et al., 2003; Dreyfoos, 1995). For example, general staffing profiles for centers can include only a primary provider or a primary provider with a behavioral health professional on site (Price, 2016). In SBHCs with a mental health component, the most common staff profile comprised of a licensed social worker, therapist, or counselor with medical assistants, registered nurses, and health educators (Larson et al., 2017). Differences in networks and partnerships can influence availability of resources and services. For example, centers that are coordinated with larger health systems, such as the local hospital and community health center can offer more facilities, appointments, referrals to off-site services, and funding opportunities (Dreyfoos, 1994). Differences in sponsorships have also been found among SBHCs with and without a mental health component. Centers with a mental health component are more likely to be sponsored by school and university departments than public health departments when compared to centers without a mental health component (Brindis et al., 4 2003). There is a critical need to identify the role of variations in structural characteristics within SBHCs with a mental health component on services. In the absence of such knowledge, further understanding of variations impacting mental health service delivery over time remains unlikely. Variations in SBHCs may be attributed to state level support. State level support refers to “the allocation of funding directly to school health centers, having state agency staff dedicated to SBHC program, promulgating and monitoring program standards, providing technical assistance for school health center operations and evaluation, convening the statewide network, collecting and reporting program data and performance measures, and establishing reimbursement policies for Medicaid and SCHIP” (Schlitt, Juszczak, & Eichner, 2008; p. 733). Sources and allocation of funding varies from state to state. To date, only 18 states direct funds to a SBHC grant program (National School-Based Health Care Census, 2014). However, some SBHCs have developed partnerships with regional grant makers directly for the evaluation of services and outcomes (Rose, Mansour, & Kohake, 2005. Other funding sources for SBHCs include patient revenue, government grants, partner contributions, and private sector funding (School-Based Health Alliance, 2016). Patient revenue sources include Medicaid, private insurance, Child Health Insurance Program (CHIP) or self-pay. Government grants include state, federal, and local level grants. Partner contributions refer to funds from community and school settings. Private sectors refer to funding from foundations, managed care, or other corporations, and are the least common funding source reported. State government and managed care organizations have been found to be the most common funding source among SBHCs with a mental health component (Larson et al., 2017). There is a critical need to identify key factors in state level support that impact development of mental health services in SBHCs over time. 5 Variations in structural characteristics and state level support can significantly influence availability of mental health services (Price, 2016). While 70% of SBHCs in the U.S. include a mental health component (Bains et al., 2017; Larson et al., 2017), specific intervention services are not consistent across sites that include a mental health component. SBHCs offer mental health services that range from risk assessments, referrals, screening, evaluation and treatment, substance abuse counseling, assessments of learning problems, to prescription management (Bains & Diallo, 2016; Larsen et al., 2017; Lofink, Kuebler, & Juszczak, 2013). Some centers may not be as well-equipped as others to provide these services. Structural characteristics, such as a larger student body, increased hours of operation, partnerships, and higher grade levels, increase the likelihood of making mental health services more available (Dreyfoos, 1998; Larson et al., 2017). Moreover, state funding and policy support can influence the sustainability of available services over time (Anyon et al., 2013; Armbruster, 2002; Hacker & Wessel, 1998). Limitations of Mental Health Services through School Based Health Centers Among SBHCs with a mental health component, sites can differ significantly. To the researcher’s knowledge, only one study has compared characteristics of SBHCs with and without a mental health component to examine differences in reported structural characteristics and funding sources (Larsen et al., 2017). SBHCs with a mental health component were reported to have more resources, more students, a longer history of establishment, and more state funding than SBHCs that do not offer mental health services at their site (Larsen et al., 2017). While this helps inform characteristics of SBHCs offering mental health services, studies have not examined differences within these characteristics (e.g., structural characteristics, state level support). 6 Most studies on SBHCs with a mental health component discuss their findings as an aggregated group (Bersamin et al., 2016; Keeton, Soleimanpour, & Brindis, 2012; Silberberg & Cantor, 2008). Exploring differences in structural characteristics and state level support within SBHCs that offer a mental health component can reveal factors that have shaped types of mental health services reported to be delivered over time. Further, most studies have paid little to no attention to the diversity of contexts (Keeton et al., 2012; Silberberg & Cantor, 2008). Limited attempts have been made to explore the influence of site components related to structural characteristics and state level support on types of mental health services reported to be delivered (Silberberg & Cantor, 2008). There is a need to compare contexts of SBHCs with a mental health component to understand what factors have facilitated or promoted types of mental health services delivered through SBHCs. Organizational Context Influences SBHC Implementation Specifically, context refers to the unique set of circumstances surrounding an implementation effort, such as the service delivery model for mental health services through SBHCs (Damschroder et al., 2009). According to a contextualist approach, research should explore contexts and their interconnections to better understand an organization’s process of change (Pettigrew, Woodman, & Cameron, 2001). Organizational contexts are complex and can generate variations in services from setting to setting (Bauer, Damschroder, Hagedorn, Smith, & Kilbourne, 2015). Inner contexts, outer contexts, and their interconnections are hypothesized to mediate an organization’s development over time (Damschroder et al., 2009; Pettigrew et al., 2001). The outer context refers to political, social, and economic context in which the organization was established (Damschroder et al., 2009; Pettigrew et al., 2001). The inner 7 context refers to the organization’s structural, cultural, and political features that direct the process in which service provision proceeds (Damschroder et al., 2009; Pettigrew et al., 2001). Guided by this approach, this thesis project is designed to identify inner and outer contexts of SBHCs with a mental health component that have influenced the number of mental health services reported to be delivered over time. Using longitudinal datasets from the School Based Health Alliance, the inner and outer contexts will be informed by two surveys, the Census Survey and State Policy Survey, administered to SBHCs and stakeholders across the U.S. from 2005 to 2014. The Census Survey will inform inner contexts, which include variations among centers’ demographics, hours of operation, staffing, networks, and student demographics. The State Policy Survey will be used to inform outer contexts of SBHCs, emphasizing the role of state level support on the number of mental health services reported to be delivered over time. Building on the contextualist approach, dissemination and implementation science theories also consider context in evaluating implementation efforts. The Consolidated Framework for Implementation Research (CFIR) provides a more comprehensive framework of constructs relevant to an implementation effort and justification for how these constructs are applied in context (Damschroeder et al., 2009). The CFIR includes five major domains: intervention characteristics, outer setting, inner setting, characteristics of individuals, and process (Damschroeder et al., 2009). Each of these domains also includes 39 constructs and sub- constructs that are theorized to impact change processes related to a specific program or practice (Damschroeder et al., 2009). For the thesis project, CFIR will be used to organize the inner and outer contexts of SBHCs with a mental health component and will guide data interpretation. Focus will be placed on two of the five domains: outer setting and inner setting (See Table 1). 8 While CFIR provides the framework for organizing contexts, it does not specify or explain how the interaction between inner and outer level settings may affect the delivery of an innovation (Damschroeder et al., 2009). However, the framework can be used to examine the process of change in organizations over time. As a variant of the contextualist approach, studies of organizational development emphasize the role of contexts in explaining changes in service delivery (Pettigrew et al., 2001). This approach in organizational development studies the Table 1: Consolidated Framework for Implementation Research Constructs, Definitions, Variables & Dataset Construct Description Variable II. Outer Setting State Level Support D. External Policy & Incentives Funding, oversight and support, policy and standards Includes external strategies to spread innovations, including policy and regulations (governmental or other central entity, external mandates, recommendations and guidelines, pay for performance, collaboratives, and public or benchmark reporting III. Inner Setting Structural Characteristics A. Structural Characteristics The social architecture, age, maturity, and size of an organization B. Networks & Communications The nature and quality of webs of social networks and the nature and quality of formal and informal communications within an organization Health center demographics, health center operations, health center care team, school characterization Health center partnerships Data Set State Policy Survey Data Set Census Survey Census Survey relationship between multiple levels of context to show how inner and outer contexts interact and shape change processes (Pettigrew et al., 2001). The thesis project aims to study change processes related to structural characteristics, networks and state level support, and explore how these contexts have shaped the number of mental health services reported to be delivered by SBHCs from 2005 to 2014. 9 Because SBHCs are targeting at-risk communities with high rates of mental health disparities, it is important to examine how inner and outer contexts have influenced the types of mental health services reported to be delivered. Changes in inner and outer contexts can influence implementation efforts (Damschroeder et al., 2009). Exploring inner and outer contexts of SBHCs with a mental health component can reveal conditions needed for the gradual development of more comprehensive services. The thesis project will address the following questions: RQ1. What is the number of mental health services reported to be delivered by state from 2005 to 2014? RQ2. Which inner setting variables (e.g., structural characteristics and networks) are related to number of mental health services reported to be delivered from 2005 to 2014? RQ3. Which outer setting variables (e.g., funding sources, policy and standards, oversight and support) are related to number of mental health services reported to be delivered from 2005 to 2014? In the following section, this paper will briefly discuss the current state of mental health disparities among U.S. children in underserved communities, provide a brief history of SBHCs, and discuss the expansion of mental health services. Then, the paper will discuss the current state of literature on SBHC effectiveness and prospects of SBHCs with mental health services, and, lastly, explain the role of policy and funding on SBHC development and sustainability. 10 LITERATURE REVIEW School-based health centers, also known as school-based health care and school-based health clinics, are one of the most effective strategies for delivering physical, mental, and preventive health services to underserved youth populations (Mason-Jones et al., 2012; Schlitt et al., 2008). School based health centers (SBHCs) represent a service delivery model capable of functioning as a medical home for children, providing primary care for both their physical and behavioral health care needs, with the capacity to promote health and prevent illness (O’Leary et al., 2014; School Based Health Center Alliance, 2016). For this thesis project, school based health centers will be defined as a comprehensive health service delivery model located at or near the school but will only include SBHCs with a mental health component. SBHCs were first established during the 1970s in Dallas, Texas, St. Paul, Minnesota, and Cambridge, Massachusetts (Brindis et al., 2003; Dryfoos, 1994; Jennings, Pearson, & Harris, 2000; Marone, Kilbreth, & Langwell, 2001). SBHCs were initially developed in response to high pregnancy rates in inner city high schools, offering only preventive and primary care services to youth in urban areas (Brindis et al., 2003; Brown & Bolen, 2003; Dryfoos, 1998, Flaherty, Weist, & Warner, 1996). Currently, more than half of SBHCs are located within urban low-income areas (Bains & Diallo, 2016). However, since the inception of the SBHC model, SBHCs have expanded to rural and suburban locations (Silberberg & Cantor, 2008; Bains & Diallo, 2016) and widened their target population to include preschool, elementary and middle school aged children (Bains & Diallo, 2016; Silberberg & Cantor, 2008). With the ongoing school reform movement to improve child and adolescent health, more comprehensive SBHCs began to integrate mental healthcare into their model (Dreyfoos, 1994; Flaherty et al., 1996). Mental health services were deemed necessary to curtail increased rates of 11 risky health behaviors, such as suicide, school drop-out and homicide, which was prevalent in targeted urban communities during the early 90s Dreyfoos, 1994; Flaherty et al., 1996). Expanding mental health services through SBHC not only provided services for treatment, but also incorporated preventive services to enhance children’s well-being (Flaherty et al., 1996). The mental health component of SBHCs is now considered a prominent model for school mental health (Armbuster, 2002; Flaherty et al., 1996). SBHC Mental Health Services Early onset of mental health difficulties during childhood is positively correlated with severity of mental health illness across the lifespan and into adulthood, if untreated (Simon et al., 2015). In school-aged children, mental health illness has also been linked to poorer academic performance, higher risk-taking behaviors, substance abuse, and developmental difficulties (Padilla-Frausto, Grant, Aydin, Anguilar-Goxiola, 2014). As a result, early identification and treatment of mental health difficulties is critical to optimize the outcomes of children experiences mental health disorder symptoms. SBHCs are designed to provide prevention of early onset mental illnesses in young children (Santor, Poulin, LeBlanc, Kusumakar, 2006), and thus are a vital service delivery mechanism to meet the need of children who are experiencing mental health symptoms but may have limited access or utilization of community-based mental health services. Over 70% of SBHCs in the U.S. offer some type of mental health assessment and treatment services (Bains & Diallo, 2016; Lofink et al., 2013; Morone et al., 2001). These services can include referrals, assessments, screenings, grief and loss therapy, counseling, crisis intervention, substance abuse counseling, family therapy, and other therapeutic interventions (Bains & Diallo, 2016; Lofink et al., 2013). Services are offered by providers from various 12 disciplines and training backgrounds, including social workers, psychologists, counselors, and nurses (Flaherty et al. 1996). However, master’s level social workers and mental health counselors have previously been reported as the most common mental health providers among all the centers (Brown & Bolen, 2003). These services are designed to target a range of disorders related to attention, depression, anxiety, eating disorders, substance use, suicide, trauma, and grief. Some centers, however, vary on capacity to offer services to treat specific behaviors. For example, one site may not have the proper training to address grief because its primary target for the given school population is treating suicide and substance use (Price, 2016). Inner context variables, such as structural characteristics of SBHCs (e.g., length of establishment and staffing), can influence the provision of services (Pettigrew et al., 2001). Similarly, outer context variables, such as funding sources, technical support, networks (e.g., partnerships), sponsorships, available resources and policies, can also influence development of such services over time (Pettigrew et al., 2001). SBHC Effectiveness SBHCs that offer mental health services have improved mental health outcomes, improved academic performance, and increased access to treatment for children from various ages and mental health illnesses (Allison et al., 2007; Armbruster & Lichtman, 1999; Larson & Chapman, 2013; Mason-Jones et al., 2012). SBHCs continue to optimize student’s health for success. Adolescent and children's mental health outcomes have been greatly improved with the integration of SBHCs in different geographical locations (Jennings et al., 2000; Wade et al., 2008). A high school SBHC located in Baltimore showed that students receiving mental health treatment reported more improvements in self-concept and reduced depression scores than those who had not received any mental health services (Weist, Paskewitz, Warner, & Flaherty, 1996). 13 Students have reported improvements in dealing with stress and anxiety with the help of their SBHC (Soleimanpour, Geierstanger, Kaller, McCarter, & Brindis, 2010). SBHCs have also promoted prosocial behaviors and facilitated the development of relationships with caring and supportive school-based staff (Stone, Whitaker, Anyon, & Shields, 2013). Mental health is strongly correlated to children’s academic performance (Hill et al., 2012). SBHCs can reduce impacts from poor health on academic related outcomes (Knopf et al., 2016; Sprigg, Wolgin, Chubinski, & Keller, 2017). A longitudinal study examined the differential impacts from medical and mental health service use in a high school-based SBHC. Mental health service use was more strongly related to increased GPAs over time than medical service use offered in SBHCs (Walker, Kerns, Lyon, Bruns, & Cosgrove, 2010). Other case studies evaluating SBHC mental health service use have demonstrated improved academic outcomes through lowered rates of absenteeism (Brown & Bolen 2003; Bains et al., 2017). Additional research has examined individual level factors facilitating mental health service offered in SBHCs. Student and parent satisfaction with services was generally reported to be high throughout several studies (Kaplan, Calonge, Guernsey, Hanrahan, 1998; Silberberg & Cantor, 2008; Soleimanpour et al., 2010). Reasons reported for SBHC satisfaction included its affordability, confidential services, convenience, and familiarity with the environment (Soleimanpour et al., 2010). This suggests that parents and students are receptive to mental health services offered through their centers because of tailored needs. Several studies support that SBHCs increase access to mental health services among children and adolescents with the highest level of needs (Armbruster & Lichtman, 1999; Bains et al., 2017; Juszczak, Melinkovich & Kaplan, 2003; Santor et al., 2006). For example, students who frequently utilize SBHC services are more likely to be children whose parents are 14 uninsured, living in low-income areas, and utilizing SBHCs as their only source of regular mental and physical care (Allison et al., 2007; Baquiran et al., 2002; O’Leary et al., 2014; Wade et al., 2008). Students who make frequent visits to their centers are also characterized as having higher risk behaviors, such as substance use or risky sexual behaviors (Wolk & Kaplan, 1993). This suggests that SBHC are successfully reaching at-risk populations and have designed culturally appropriate services to match their needs. Mental health concerns account for one of the largest proportion of reasons students visit their SBHC (Bains et al., 2017; Kaplan et al., 1998; Santor et al., 2006). In fact, students with SBHCs have been found to be ten times more likely to make mental health visits to centers when compared to students without access to SBHCs (Guo et al., 2008; Kaplan et al., 1998; Santor et al. 2006; Soleimanpour et al., 2010). Another study found that students with mood disturbance or other mental health difficulties accounted for 46% of SBHC visits (Santor et al., 2006). Students who identified a problem (e.g., mood disorder or other mental illness) were more likely to visit their SBHC than individuals who do not report any problems (Santor et al., 2006). In multiple studies with high school students, mental health visits were sought after for substance abuse treatment or other mental health related illnesses, such as depression or suicidal thoughts (Kaplan et al., 1998; Pastore, Juszczak, Fisher, & Friedman, 1998; Szumilas, Kuthcer, & LeBlanc, 2010). In sum, findings show that SBHCs have successfully increased access to physical and mental health care for students who need treatment. Additionally, mental health services provided through SBHCs have been found to be cost effective. SBHCs have decreased familial and societal costs by reducing the number of emergency room visits by children with access to SBHCs when compared to other delivery sectors, such as community health network facilities (Adam & Johnson, 2000; Brown & Bolen 15 2003, Juszczak et al., 2003; Kaplan et al., 1998; Schlitt et al., 2008; Smith, 2013; Young, D’angelo & Davis, 2001). When SBHCs are coordinated with larger health systems, such as local hospitals or community health networks, cost of mental health services can be reimbursed through managed care systems, such as Medicaid (Armbuster, Andrews, Couenhoven, & Blau, 1999; Guo et al., 2010; Ran, Chattopadhyay & Hahn, 2016). An analysis on the economic cost and benefit of SBHCs indicated that benefits of SBHCs greatly exceeded the cost to run them (Ran et al., 2016). Specifically, findings from this economic analysis demonstrated decreased costs in Medicaid, as well as positive impacts to students’ educational and health outcomes and students’ sense of responsibility toward achieving their academic goals (Ran et al., 2016). Individual-level factors, such as age, gender, and insurance status are found to moderate impacts of mental health services on students’ behavioral health outcomes and frequency of service use at their centers. Some studies found that a higher number of visits are made by older students and female students (Bains et al., 2017; Parasuraman & Shi, 2014; Wade et al., 2008; Wolk & Kaplan, 1993). Impacts from age were also documented in a SBHC in New Haven that showed 13-year-old students were 3 times more likely to visit their health center than younger children over time (Bains et al., 2017). More studies will need to account for these factors to examine maintenance of health outcomes. Organizational level factors, such as location and length of establishment, are also suggested to influence frequency of visits to centers (Johnson & Hutcherson, 2006; Wade et al., 2008). For instance, a longitudinal study found that mental health service use increased over a 3-year span among 8 SBHCs in rural and urban school districts (Wade et al., 2008). Yet, few studies have focused on the sustainability of SBHC in service delivery and in maintaining beneficial mental health outcomes over time (Brindis et al.2003; Silberburg & 16 Cantor, 2008). Limited research has been attributed to a lack of consistency across service delivery, diversity of contexts in these organizations, and limited use of rigorous designs (Dryfoos, 1994; Silberberg & Cantor, 2008). Research has also suggested for more comparisons between community-based and school-based settings of service delivery to better understand quality of care (Langer et al., 2015). Yet, another issue relates to the fact that most evaluations of SBHCs are comprised of small sample sizes, case studies, and attrition in student population (Keeton et al., 2012; Weist et al., 1996). Cases studies do not allow for comparative analyses due to measures that are not comparable to one another. More studies focusing on these dimensions can be made difficult when individual schools function within unique inner and outer contexts. Specifically, inter-organizational structures are formal or informal networks that can strongly influence adoption of innovations (Greenhalgh, Robert, Macfarlane, Bate, & Kyriakidou, 2004). Networks include partnerships, collaboration, and sponsorships. As previously discussed, SBHCs vary in these structures, which can potentially hinder full delivery of their health care model (Kaplan et al., 1998; Brindis et al., 2003). Community partnerships have been found to facilitate the process of integrating SBHCs into existing schools as well as community-based health care systems and, thus, support the sustainability of SBHCs (Armbruster, 2002; Dreyfoos, 1994; Swider & Valukas, 2004). Collaboration in systems of coordinated care can help bring in more resources, funding sources, and stronger community support (Liu, Ramowski, & Nystrom, 2010; Swider & Valukas, 2004). For example, a SBHC in Chicago developed a partnership with the school, local health department and local hospital, which led to more funding opportunities, services, and increased sustainability (Swider & Valukas, 2004). SBHCs that are not part of an integrated system of care may not have the resources needed to successfully meet standards for quality care measures for Medicaid and 17 Child Health Insurance Program (CHIP), facing more obstacles related to third party billing (Allison et al., 2007). However, inconsistent systems of coordination between schools and SBHCs have been documented for some SBHCs (Richardson, 2007). The sustainability of SBHCs is contingent upon funding from state, federal and local level grants, financial support from partner contributions and patient revenue, and community support and collaboration (Anyon et al., 2013; Hacker & Wessel, 1998; Armbruster, 2002; Rones & Hoagwood, 2000). State government entities, such as the Department of Education, carry out the greatest oversight of schools (Anyon et al., 2013). The most common funding sources have been previously reported from direct state funding, state general revenue and Title V of Social Security Act (Schlitt et al., 2008; Swider et al., 2004). Third party billing (e.g., to Medicaid or other private insurance) poses another obstacle for the implementation and sustained use of SBHCs. These billing codes are frequently denied by managed care networks (Silberberg & Cantor, 2008). Managed care organizations are reluctant to authorize mental health services provided through SBHCs because of perceived lack of quality of the mental health services (Armbruster, 2002). Furthermore, state level and national level policies directly impact the growth of SBHCs (Sprigg et al., 2017). For example, some states receive funding from the Patient Protection and Affordable Care Act (ACA), which then encourages the development of SBHCs by providing support to create and expand services through mandates or collaborations (Doll, Nastasi, Cornell, Song, 2017). State funding and resources enhanced through partnerships can increase the likelihood of a SBHC including mental health services (Larson et al., 2017). State policies can help SBHCs develop strategies to design frameworks that meet educational statues, which increases resources accessible to the center (Anyon et al., 2013). Policies with state level funding 18 for SBHCs may also reduce funding insecurity related to billing for the physical and mental healthcare services (Sprigg et al., 2017). Yet, comparisons between SBHCs with different outer contextual factors, particularly, funding and state policy support, are not well examined. School based health centers are complex systems that interact with surrounding variables. Because SBHCs function within distinct contexts, such as inner and outer settings, it is important to understand how these factors have influenced types of mental health services reported to be delivered over time. Research on SBHCs with a mental health component has not been examined through the lens of context. A contextualist approach offers an alternate perspective on previously noted study limitations. This approach examines variables (e.g., contexts) unique to each SBHC and views them as essential to the process of mental health service delivery over time. Given that structural characteristics, staffing, funding sources, policy, funding, partnerships, and provider availability can vary greatly among SBHCs with a mental health component, a contextualist approach can more closely examine how these contexts have shaped service delivery over time. Previous studies have not used CFIR to organize unique contextual factors relevant to SBHCs. Using CFIR as a framework for examining the inner and outer contexts of SBHCs can help identify barriers or facilitators to the developmental process of service provision (Damschroeder et al., 2009). Understanding contextual impacts can inform strategies for more effective SBHC designs and service delivery. As suggested by Silberberg and colleagues (2008), comparing SBHCs with a mental health component with one another can also help identify any likelihood of support (funding or policy) for a site with specific characteristics. In response to these suggestions, the current study aims to describe the number of mental health services reported to be delivered by SBHCs from 2005 to 2014. Moreover, this study aims to identify the 19 inner and outer setting variables of SBHCs from 2005 to 2014. Findings will emphasize the inner and outer contexts of SBHCs with a mental health component to focus more on differential impacts on mental health services reported to be delivered. Findings will also promote discussion on the role of state level support in mental health services delivered through SBHCs. 20 METHODS This thesis project explored the role of inner and outer contexts on the number of mental health services reported to be delivered within SBHCs across the U.S. The project involved secondary analyses of two longitudinal quantitative datasets, and utilized descriptive statistics and generalized mixed model approaches. The datasets used for the project were restricted to data that had already been collected through surveys administered from a third party, the School Based Health Alliance. Participating SBHCs for the current project needed to meet the following inclusion criteria: (1) located at a school or on school property; (2) include a mental health component in their model; and (3) provide enough data for analyses. All data analyses were conducted with SPSS 25. The study addressed the following research questions: RQ1. What is the number of mental health services reported to be delivered by state from 2005 to 2014? For the purposes of the project, mental health was defined as “successful performance of mental functioning, resulting in productive activities, fulfilling relationships with other people, and the ability to adapt to change and cope with adversity” (Hill et al., 2012, p. 120). This complements the School-Based Health Alliance’s shared vision of promoting health for student success. “Mental health” incorporates treatment and activities designed to promote youth’s mental well-being. Mental health services will refer to screening and assessments, medication management, substance use treatment, and referrals. Descriptive analyses, specifically frequencies, were used to describe and summarize mental health services reported by state on the National Census of SBHC Survey from 2005 to 2014. Reported mental health services. Variables selected for mental health (MH) services reported by state included: (a) screening/assessment; (b) medication management; (c) substance 21 use treatment; and (d) referrals. These variables were selected because of their consistency throughout the four time points. Some adjustments were made to minimize the number of outcome variables and to adhere to consistency. For example, screening and assessment were originally two independent items reported in earlier assessments, but were combined in later assessments. To maximize consistency, these variables were combined to create “screening/assessment” across all time points. All other items that were not consistent across the time points were excluded from analyses. Examples of these include grief/loss therapy and classroom behavior management. Data cleaning & missing data. Missing values were handled as random missing cases, using substitutions to recode. Substitution was considered the best approach to have two clean categories for the discrete outcomes. Moreover, outcome variables were transformed to count variables to facilitate further analyses. Count variables accounted for the number of “yes” occurrences among centers within a state by each time point. For example, if three centers in GA were assessed in 2010 and only two reported “Yes” to referrals; then the count variable for referrals would be equal to 2. This was done for all other outcome variables by state and time point. This allowed the opportunity to treat outcome variables as continuous variables, facilitating the procedure for main analysis. RQ2. Which inner setting variables (structural characteristics and networks) are related to number of mental health services reported to be delivered from 2005 to 2014? The National Census of SBHC Survey was first explored using descriptive statistics. Results provided an overview of health center demographics, health center operations, health center care teams, school characterizations and health center partnerships from 2005 to 2014. Guided by the CFIR, constructs were then organized into inner setting variables (structural 22 characteristics; networks and communications) by state. Variables for the inner setting were selected because of consistency and sufficient data reported throughout the four administrations of the Census survey. The specific inner setting variables used to describe SBHC structural characteristics included: (1) health center demographics (geographic location, year of establishment, school enrollment number, grades served: elementary, middle, high school), (2) health center operations (number of days open weekly, hours open weekly), (3) health center care team (total hours worked for mental health service providers, location of behavioral health provider), and (4) school categorization (Title I, charter, alternative, vocational, magnet, and /or public school). The inner setting variable used to describe networks and communications was agency sponsor. Frequencies were run by time points to further explore the inner setting variables within the data set. Further, a linear mixed model analysis (LMM) was conducted to identify key variables within the inner settings of SBHCs that related to the number of mental health services reported over time. Data cleaning & missing data. Some surveys had inconsistencies in response choices for items. For example, geographic location coded response items differently in 2008 when compared to all the other assessments. To promote consistency, these variables were recoded, when necessary. Missing-data imputations (average of available values) was used to handle missing responses in the following variables: days open weekly, hours open weekly, and total mental health staff hours worked. Other missing values in agency sponsor and geographic location were changed to 999 for “IDK” to promote consistency and avoid deletion of a large number of cases. Corrections were also made to have appropriate values for a given response. For example, responses reported for the number of days/week should have a maximum value of seven. However, some sites reported numbers exceeding that. Therefore, for values greater than 23 seven, it was assumed that sites referred to days open in a year. These were then converted to days/a week based on 180 days within a school year. RQ3. Which outer setting variables (e.g., funding sources, policy and standards, oversight and support) are related to the number of mental health services reported from 2005 to 2014? The SBHC Policy Survey was also first explored using descriptive statistics. Results provided an overview of funding sources, policy and standards, and oversight and support from 2005 to 2014 by state. Guided by the CFIR, constructs were organized into multiple outer setting variables to inform the domain of external policy & incentives. Outer setting variables selected for the current study explored funding (i.e., presence/absence of a SBHC grant program, funding source, and state agencies reported as funding sources); oversight and support (i.e., technical assistance and presence/absence of SBHC data collection); and policy and standards (i.e., operating standards for SBHCs, and state requirements for data collection). LMM was also used to identify key variables within the outer settings of SBHCs that related to the number of mental health services reported over time. Data cleaning & missing data. Data cleaning was done to enhance accuracy of the dataset for further analyses and to ensure consistency across survey administration time points. When appropriate, missing data was replaced with a “0” to indicate absence of a variable. Other responses that could not justifiably be replaced with absence (“0”) were recoded as a missing value (“999”) for “no response” (NR). Cases that were missing more than 3 time points or a considerable amount of data were dropped from analyses. Variables that indicated multiple responses (“select all that apply” items), were recoded to binary data (e.g., “1= yes” or 2= no”). 24 Participants Providers. The National Census of SBHC Survey targeted populations aimed at site levels. Inclusion criteria for SBHC sites that participated in the Census Survey are reported as: partnerships between schools and community health organizations that deliver health care to students within a SBHC; health care programs that are linked with SBHCs (e.g., school-linked centers); programs delivering services without a fixed site (e.g., mobile); and programs delivering services through telehealth (tele-health only sites). At each time point, the survey requests that the person most knowledgeable about the services provided in the SBHC respond to all questions. Examples of these include nurse practitioner or clinical director. Morever, Policy makers. The SBHC Policy Survey’s inclusion criteria included: persons most knowledgeable about state level policies, funding, and programming related to SBHCs. Specifically, targeted policy maker participants were State agency staff in maternal, child, adolescent, and school health divisions and Medicaid agency members who were most knowledgeable of reimbursement policies (Schlitt et al., 2008). All surveys were mailed to an identified individual within the various departments. Measures National Census of SBHC Survey. The first National Census of SBHCs was collected in 1998. The survey is administered every 3 years. The Census provides up to date information on centers’ structural demographics, prevention activities, financing strategies, staffing, services, student demographics, clinical policies, and mechanisms for quality improvement. For National Census of SBHC Survey items, see Appendix A. Data from this survey was used to identify inner contexts of SBHCs, to the extent possible. The longitudinal dataset selected for this study examined responses collected from 2005 to 2014, which equals four survey administration time 25 points. Most survey items have remained the same since 2005, with little changes (Larson et al., 2017). However, changes that have occurred include removal of several survey items, addition of new choice responses to survey questions, or rewording of items. An example of added items can refer to responses such as tele-health or mobile clinics that were not provided as response options in earlier administrations (e.g., 2005 or 2008) of the survey. Moreover, reworded items were compared to earlier administrations to determine whether they collected similar responses to previous years. Questions that were considerably missing or inconsistently reported throughout the four time points were not included in the analyses. That is, if an item was only asked in 2005, but not in the any other time-point, the researcher dropped the item for further analysis. SBHC State Policy Survey. In collaboration with the Robert Woodcock Johnson Foundation, the School Based Health Alliance created a State Policy Survey. The State Policy Survey was designed to collect information from State public health and Medicaid offices. The survey is administered every four years, with its first assessment completed in May 2005 (Schlitt et al., 2008). The survey explored the role of policy and state-level support on the development and sustainment of SBHCs. State-level support was operationalized as “allocation of funding directly to school health centers, having state agency staff dedicated to SBHC program, promulgating and monitoring program standards, providing technical assistance for school health center operations and evaluation, convening the statewide network, collecting and reporting program data and performance measures, and establishing reimbursement policies for Medicaid and SCHIP” (Schlitt et al., 2008; p. 733). The State Policy Survey was adapted from a survey conducted by the Center for Health and Healthcare in Schools, excluding collection of information regarding structural characteristics (e.g., staffing, school type). The first survey was mailed to all State public health departments and one Governor’s Office for Children, Youth, and 26 Families. To optimize the rigor of the survey, the following activities were carried out in subsequent survey administrations: identifying appropriate individuals to complete the survey (e.g., knowledgeable of SBHCs and reimbursement policies); implementing efforts for best response rates (e.g., mailing surveys to state health departments); inspecting survey content, data cleaning, and data recording once surveys were received back (Schlitt et al., 2008). Subsequent survey administrations continued to target participants who were most knowledgeable of state- level policies, funding, and program support related to SBHCs. Survey items were consistent throughout the time points collecting information on: (1) number of SBHCs for the current administration’s school year, (2) source of state funding directed to SBHCs, (3) state criteria for funding distribution, (4) technical assistance, (5) performance data collected, (6) state perspectives on future outlook, and (7) Medicaid/SCHIP policies for reimbursement (Schlitt et al., 2008). For State Policy Survey items, refer to Appendix B. Procedures School Based Health Alliance (SBHA). The School Based Health Alliance is a non- profit, multidisciplinary inter-organizational network that was founded in 1995. The Alliance, formerly known as the National Assembly on School-Based Health Care (Larson et al., 2017), serves to advocate SBHCs across the nation and increase access to children and adolescents in need of physical and mental health care. Twenty-one states have an affiliated status with the School Based Health Alliance. These partnerships share in the mission, vision and core values of the Alliance. The Alliance membership is diverse, comprised of school staff, health practitioners, researchers, physicians, professors and non-profit organization directors. In collaboration with a technical advisory committee comprised of researchers, the Alliance developed the National School Based Census Survey in 1998 to document, track, and disseminate information about 27 centers across the U.S. The survey collected a variety of information on structural demographics, services offered, clinical policies, staffing, student demographics, and strategies used to evaluate program quality assurance (Brindis et al., 2003). The School Based Health Alliance has surveyed centers throughout the U.S. from 1998 to 2017, every three years. The School Based Health Alliance has also surveyed stakeholders and policymakers on State level support for SBHCs with the State Policy Survey. Data collection procedures. To gain access to the datasets, the researcher contacted an Alliance member, as instructed on the School Health Alliance website. The Alliance member informed the researcher about the process of requesting and accessing the datasets, which included a brief telephone meeting and submission of a data request form. The telephone meeting occurred on May 31, 2017. The researcher provided the Alliance member with information regarding the thesis project’s goals and aims. After the meeting, the Alliance member emailed a data request form, which was prepared in collaboration with the researcher’s thesis advisor. The data request form was submitted on June 30, 2017. A research committee member from the SBHC Alliance contacted the researcher on August 8, 2017 with clarifying questions regarding the project design and purpose of using multiple survey time points within the data. The researcher addressed clarifying questions and was asked to revise and resubmit the data request form with the updated details. Once the revised data request form was submitted, the committee member confirmed receipt and prepared the datasets to send through email. Complete datasets were received on September 18, 2017. Data Analysis Exploratory data analysis. The National Census of SBHCs Survey and State Policy Survey are two independent longitudinal datasets. The National Census Survey dataset has 28 repeated measures that have been collected at 4 time points, each representing an academic year: 2004-2005, 2007-2008, 2010-2011 and 2013-2014. Data was collected on reported mental health services delivered, structural characteristics, and networks. Additionally, the State Policy Survey dataset has repeated measures that have been collected at 4 time points, 2004-2005, 2008-2009, 2010-2011 and 2013-2014, on outcomes related to state level support (e.g., funding sources, policy and standards, oversight and support). To better understand these datasets, descriptive analyses were conducted to explore mental health services, structural characteristics, networks and partnerships, and state level support within each SBHC at the state level. Multiple descriptive analyses were conducted for the Survey Subscales. Specifically, the National Census Survey - Services subscale was analyzed to describe the number of behavioral services reported to be delivered within each state over the four time points. The National Census Survey - Structural characteristics subscale was analyzed to describe health center demographics, health center operations, health center care team, and school characterization over the four time points. The National Census Survey- Network communications subscale was analyzed to describe health center partnerships over the four time points. The State Policy Survey - Funding sources subscale was explored to describe the source of state funding directly dedicated to SBHCs over the four time points. The State Policy Survey - Policy and standards subscale was analyzed to describe states’ criteria for operating standards and requirements for data collection defined by the states over the four time points. The State Policy Survey - Oversight and support subscale was analyzed to describe technical assistance provided and designation of data collection units over the four time points. Results from both the National Census Survey and State Policy Survey were aggregated at the state level. 29 Linear mixed model (LMM) analysis. A linear mixed model (LMM) approach fit by restricted maximum likelihood estimation (REML) was used to assess the significance of inner setting and outer setting variables in explaining variations of mental health services reported to be delivered over time. LMM is an extension of linear models that can add random effects to fixed effects models and account for variance within repeated measures that are grouped or collected from the same subject (IDRE, 2017; Mcculloch, & Neuhaus, 2013; Zhang et al., 2011; Zuur, Ieno, Walker, Saveliev, & Smith, 2009). The longitudinal data included in the current study had inter-dependent observations that did not allow for regression techniques. Therefore, the adjusted relationship between inner setting variables and mental health services were analyzed using linear mixed effects model in SPSS 25. For the current study, the dependent variables for mental health services included: screening and assessment, medication management, substance use treatment, and referrals. To facilitate LMM analysis, these binary variables (yes/no responses) were transformed to count variables, quantifying the number of “yes” occurrences within each state. The predictor variables included components of the survey that were theorized as inner and outer setting factors based on the CFIR, which were mostly categorical variables. All inner setting variables and outer setting variables selected for the current study were fixed factors. “Time” was evaluated as a random effect and then as a fixed effect to assess variance. Independent LMM analyses were carried out for each of the dependent variables (DVs). Further, inner setting and outer setting variables were assessed independently within each DV. Four different models were created and independently run for each DV in the following order: (1) intercept-only model; (2) random intercept and slope for time; (3) unadjusted fixed effects model; and (4) adjusted model. First, the researcher fit the model to each dependent variable 30 without allowing any predictor variables. This intercept-only model demonstrated the variation of each mental health service between states (Winter, 2013). Second, the researcher tested whether each mental health service significantly varied with time (e.g., random intercept model). The random intercept model assessed the total variation of services accounted for by time (Winter, 2013). Third, the researcher assessed the relationship between each independent variable and dependent variable without adjusting for any other variable in the model. The unadjusted fixed effects model, which is similar to bivariate analysis, assessed the relationship between each independent variable and dependent variable, without any interactions of other variables (Winter, 2013). Last, the researcher added all independent variables to the model to assess the relationship between each variable after adjusting for other variables (e.g., interaction term). This adjusted model helps us understand the relationship between each inner setting or outer setting variables on mental health services, given the interactions of other predictor variables within the same model (Winter, 2013). Time was run in the LMM model as random effects, but was also run as fixed effects to assess variance. All other models controlled for time as a covariate. Overall, all models assessed mental health services, inner setting, outer setting, and time effects. Results from the adjusted fixed effects model were expected to provide the most robust results and were used to report findings (Winter, 2013). The REML method was preferred over maximum likelihood (ML) method because REML helps correct for degrees of freedom resulting from estimating fixed effects (Zhang, 2011). While both models produce the same estimates for fixed effects, the models differ for random effects (Kenward & Roger, 1997). Given that the conditions of the datasets are disproportionately clustered and missing repeated measures at each time point, REML was considered the best approach to compare random effects from the variable “Time.” Fixed effects 31 model was selected because it is the most commonly used strategy for LMM, unless a theory driving the study explicitly specifies for a random effects model instead (Mcculloch, & Neuhaus, 2013; Zhang et al., 2011). The variables used for analyses were mostly categorical from multiple time points and from several sites within 41 states in the U.S. Moreover, the data had clustered observations (e.g., multiple observations) from the same SBHCs over time. The nature of the datasets violates assumptions of normality needed for a linear model approach. However, LMM allows for assessment of non-independent data with fixed and random effects (Zhang et al., 2011; Zurr et al., 2009). Given the longitudinal context of the data, LMM was needed to account for the random effects from the four time points administered with each survey (Zurr et al., 2009). LMM is considered one of the best approaches for assessing inter-dependent sources (e.g., repeated measures within subjects; Mcculloch, & Neuhaus, 2013; PennState, 2017; Winter, 2013; Zhang et al., 2011; Zurr et al., 2009). 32 RESULTS The final sample size used for the current study was 4,232 and represented 41 of the 50 U.S. states. Seven states were excluded because of a significant amount of missing responses or inconsistencies in responses for the four time points: Idaho, Kansas, Montana, Utah, Virginia, Wisconsin, and Wyoming. Two states were excluded because they did not report any SBHCs in operation from 2005-2015: Hawaii and North Dakota. The total number of eligible sites for each time point ranged from 913 to 1,244. Table 2 presents the final sample size collected from the SBHC Census Survey and the SBHC Policy Survey. Table 2: Sample Sizes Across Time-Points SBHC Census Survey SBHC Policy Survey Time Point Total SBHCs Assessed Total SBHCs Eligible Total States Assessed Total States Eligible States States Excluded Excluded (No (No SBHCs) consistency) 2004- 2005 2007- 2008 2010- 2011 2013- 2014 1,227 1,156 1,225 919 1,381 913 1,627 1,244 49 49 52 18 37 36 41 17 7 11 7 - 5 2 4 1 Descriptive analyses Screening & assessment services. Overall, the percentage of sites that reported delivering screening and assessment services varied by state and time point. More than half of participating and eligible states had 55% or greater of the sites reporting availability of screening and assessment services throughout the four time points. Moreover, there was a significant increase in the percentage of sites delivering this service over time (T2 and T4). Table 3 presents the percentage of sites that reported delivery of screening and assessment services within each state at T1, T2, T3, and T4. 33 Table 3: Screening & Assessment Services Time Point T1 T2 T3 T4 Percentage of States Sites 100% 90-99% 80-89% 70-79% 60-69% 50-59% Less than 50% None reported 100% 90-99% 80-89% 70-79% 60-69% None reported 100% 90-99% 80-89% 70-79% 60-69% 50-59% Less than 50% None reported 100% 90-99% 80-89% 70-79% 6-69% 50-59% Less than 50% None reported (Number) 9 7 7 6 1 1 6 4 30 5 1 0 1 3 10 11 2 2 4 1 3 7 11 10 10 3 1 2 3 1 States (%) 22% 17% 17% 15% 2% 2% 15% 10% States DC, DE, GA, IN, MO, PR, TN, VT, WA CO, LA, MA, ME, NM, NY, OR CT, CA, FL, IA, MN, OH, TX KY, IL, MD, MI, NC, WV NJ SC AL, MS, SD, PA, RI, OK AK, NE, NH, NV AK, AL, CO, DC, DE, GA, IA, KY, LA, MA, MD, ME, 73% MN, MO, MS, NC, NE, NH, NJ, NM, NY, OH, PA, PR, SC, 12% 2% 0% 2% 7% 24% 27% 5% 5% 10% 2% 7% 17% 27% 24% 24% 7% 2% 5% 7% 2% TN, TX, VT, WA, WV FL, IL, CA, OR, CT MI - IN OK, NV, RI AL, CO, GA, IA, MN, PA, PR, RI, SC, WA FL, CA, CT, IL, LA, ME, MI, NC, NM, NY, OR DE, MD OK, WV KY, MA, TN, SD OH DC, NJ, TX AK, MO, MS, NE, NH, NV, VT AK, AL, DC, DE, IN, ME, MN, OK, PA, RI, SD CT, IL, LA, MA, MD, NC, NM, OR, TN, TX CA, CO, GA, IA, KY, MI, NY, NE, SC, WA NV, OH, WV FL MO, VT MS, NH, NJ PR Medication management services. Overall, there was an equal distribution between the percentage of sites that reported delivery of medication management services and those that did not, with the exception of sites responding at T3. Moreover, there was a significant increase in the number of participating states that reported all sites delivering services over time. Interestingly, all sites in New Hampshire (NH) did not report the availability of medication management services at any time (see Table 4). 34 Table 4: Medication Management Services Time Point T1 T2 T3 T4 Percentage of States Sites 100% 90-99% 80-89% 70-79% 60-69% 50-59% (Number) 4 0 0 5 4 5 Less than 50% 17 None reported 100% 90-99% 80-89% 70-79% 60-69% 50-59% Less than 50% None reported 100% 90-99% 80-89% 70-79% 60-69% 50-59% Less than 50% None reported 100% 90-99% 80-89% 70-79% 60-69% 50-59% Less than 50% None reported 5 8 1 6 6 5 7 2 6 23 6 3 2 1 1 2 3 5 1 6 7 2 7 8 5 States (%) 10% 0% 0% 12% 10% 12% 41% 12% 20% 2% 15% 15% 12% 17% 5% 15% 56% 15% 7% 5% 2% 2% 5% 7% 12% 2% 15% 17% 5% 17% 20% 12% States GA, PR, VT, WA - - CO, ME, NM, NY, TN CT, IA, KY, OR DC, MN, MO, OH, TX AL, CA, DE, FL, IL, IN, LA, MA, MD, MI, MS, NC, OK, PA, RI, SD, WV AK, NE, NH, NV, SC GA, MO, MS, NJ, PA, SC, SD, VT WA CO, KY, NM, TN, TX, WV CT, DE, FL, IA, MA, OR IL, LA, MD, MN, NC AK, AL, CA, DC, ME, MI, NY IN, OH NE, NH, NV, OK, PR, RI AK, AL, CO, DE, GA, IA, KY, MA, MD, MI, MN, MS, NC, NE, NY, OR, PA, PR, RI, SC, SD, VT, WA NM, ME, LA, CT, IL, CA FL, IN, WV OH, OK TN TX DC, NJ MO, NH, NV AL, MN, NE, OK, SD OR CO, IL, MA, SC, TN, TX LA, ME, NC, NY, PA, WA, WV KY, MI CT, GA, IA, MO, NV, RI, VT CA, FL, IN, MD, MS, NJ, NM, OH AK, DC, DE, NH, PR Substance use treatment services. Over half of the eligible states had more than 50% of sites reporting delivery of substance use treatment services, with the exception of sites responding at T4. Moreover, there was a significant decrease in the number of participating states that reported 100% of sites delivering services over time. Also, sites in New Hampshire (NH) did not report delivery of substance use treatment services at any time (see Table 5). 35 Table 5: Substance Use Treatment Services Time Point T1 T2 T3 T4 Percentage of Number of Sites 100% 90-99% 80-89% 70-79% 60-69% 50-59% Less than 50% None reported 100% 90-99% 80-89% 70-79% 60-69% 50-59% Less than 50% None reported 100% 90-99% 80-89% 70-79% 60-69% 50-59% Less than 50% None reported 100% 90-99% 80-89% 70-79% 60-69% 50-59% States 5 3 7 7 4 5 5 5 19 9 4 1 0 2 1 4 6 1 6 1 7 4 8 8 1 0 0 2 4 2 Less than 50% None reported 17 15 States (%) 12% 7% 17% 17% 10% 12% 12% 12% 46% 22% 10% 2% 0% 5% 2% 10% 15% 2% 15% 2% 17% 10% 20% 20% 2% 0% 0% 5% 10% 5% 41% 37% States GA, MO, PR, VT, WA DE, IN, NM CO, LA, MA, ME, MN, NY, OR CT, DC, FL, IL MI, TX, WV CA, IA, NJ, OH KY, MD, NC, SC, TN AL, MS, OK, PA, RI, AK, NE, NH, NV, SD AL, AK, CO, DE, GA, KY, ME, MN, MO, MS, NE, NJ, OR, PA, PR, SD, VT, WA, WV FL, CT, IL, LA, MA, MD, MI, NM, TX CA, IA, NC, NY, OH IN - DC, SC TN NH, NV, OK, RI AK, AL, KY, MS, PR, SC NM DE, IA, IN, ME, MD, NC NY CA, FL, GA, IL, MA, PA, SD MI, MN, WA, WV CO, CT, LA, NJ, OH, OR, TN, TX DC, MO, NE, NH, NV, OK, RI, VT DC - - MN, TX ME, MI, NM, SC CO, MO CA, CT, DE, FL, IA, IL, IN, LA, MA, MD, NC, NY, OH, OR, PA, WA, WV AK, AL, GA, KY, MS, NE, NH, NJ, NV, OK, PR, RI, SD, TN, VT Referral services. There was significant variation among sites reporting delivery of referral services by state and over time. For example, 29 states reported that all of their sites delivered referral services at T2; however, no states reported that all sites delivered referral services at the next assessment time point (see Table 6). 36 T1 T2 T3 T4 of Sites 100% 90-99% 80-89% 70-79% 60-69% 50-59% Less than 50% None reported 7 6 7 9 1 1 6 4 100% 29 93-99% 67% None reported 100% 90-99% 80-89% 70-79% 60-69% 50-59% Less than 50% None reported 100% 90-99% 80-89% 70-79% 60-69% 50-59% Less than 50% None reported 8 1 3 0 6 3 2 1 1 12 30 7 1 3 11 4 3 8 3 Table 6: Referral Services Time Point Percentage (Number) States States (%) States DE, GA, MO, PR, TN, VT, WA LA, MA, ME, NM, NY, TX CA, CO, FL, MI, MN, OH, OR AL, CT, DC, IL, IA, KY, MD, NC, WV NJ SC IN, MS, OK, PA, RI, SD AK, NE, NH, NV AK, AL, CO, DC, DE, GA, IA, KY, LA, MA, ME, MD, MN, MO, MS, NE, NH, NJ, NM, NY, OH, PA, PR, SC, SD, TN, VT, WA, WV CA, CT, FL, IL, MI, NC, OR, TX IN NV, OK, RI CA, CT, IL, LA, ME, NM FL, IN, WV OH, OK TN TX CA, CO, DC, FL, IL, LA, MI, NM, NY, TX, WA, WV AK, AL, CO, CT, DE, GA, IA, IN, KY, MA, MD, ME, MN, MO, MS, NC, NE, NH, NJ, NV, OH, OK, OR, PA, PR, RI, SC, SD, TN, VT AK, AL, DC, MN, OK, RI, SD OR LA, WV, MD, CT, DE, FL, KY, IL, ME, MI, NV, NY, PA, TX CA, MA, NC, NM GA, MO, OH IA, IN, MS, NH, NJ, SC, TN, WA NE, PR, VT 17% 15% 17% 22% 2% 2% 15% 10% 71% 20% 2% 7% 0% 15% 7% 5% 2% 2% 29% 73% 17% 2% 7% 27% 10% 7% 20% 7% 37 Inner Setting Variables The National Census Survey – Structural Characteristics subscale. The Structural Characteristics subscale was explored to describe the maturity, size and social architecture of health centers. Maturity of health centers is represented by the site’s length of establishment. The size of health centers is represented by school enrollment and health center care team. Moreover, “health center care team” provided the location of a behavioral health provider at each site and a summary of total hours reported for all mental health staff affiliated with the SBHC. The social architecture of health centers is informed by each site’s geographic location, grade levels served, health center operations, and school characterization. School characterization determined the type of schools working with the SBHCs. These included the following categories: (1) Title I; (2) Charter school; (3) Alternative school; (4) Vocational school; (5) Magnet school; and/or (6) Public school. It is important to note that responses collected for school characterization were not mutually exclusive and often overlapped with one another across all four time points. That is, a site was able to report multiple responses for school characterization, such as "Title I" and "Public School" or "Magnet school" and "Vocational school." However, at T4, 0% of sites reported multiple responses for school characterization, with the exception of public school and Title I (5%). Figure 1 depicts the number of "yes" occurrences for school characterization by each time point. Table 7 presents a summary of frequencies for structural characteristics by time point. 38 Table 7: Summary of Structural Characteristics by Time Point Health Center Demographics Geographic Location T3 T1 T2 Rural Suburban Urban Year of Establishment Prior to 1989 1990 – 1994 1995 – 1999 2000 – 2004 2005 – 2009 After 2010 Grade Levels Serviced Pre-K to 5th grade 6th to 8th grade 9th to 12th grade School Enrollment 1 – 699 700 – 1999 2000+ 27% 15% 58% 11% 19% 31% 28% 4% - 49% 54% 48% 34% 50% 8% 15% 60% 25% 10% 14% 23% 20% 28% - 39% 46% 60% 45% 46% 9% 27% 19% 54% 9% 13% 25% 18% 25% 10% 41% 51% 61% 41% 51% 8% Health Center Care Team Behavioral Health Providers Located off school site In school but separate from SBHC In school and co-located in SBHC Total MH Staff Hours Less than 40 Greater than 40 SBHC Operations Days Open, Weekly Less than 5 Greater than 5 Hours Open, Weekly Less than 9 9 to 30 Greater than 30 School Characterization Title I Charter Alternative Vocational Magnet Public 1% 68% 14% 88% 12% 16% 84% 16% 20% 64% 40% 2% 6% 3% 5% 60% 18% 58% 17% 85% 15% 15% 85% 10% 15% 75% 27% 2% 6% 3% 6% 67% 32% 51% 15% 78% 22% 17% 83% 11% 18% 71% 68% 5% 7% 6% 9% 84% T4 19% 49% 32% 12% 10% 18% 15% 24% 21% 49% 53% 59% 44% 45% 11% 10% 49% 18% 85% 15% 20% 80% 5% 18% 76% 78% 2% 6% 6% 9% 68% The National Census Survey – Network Communications subscale. The Network Communications subscale was explored in order to describe the nature of formal and informal communications within SBHCs. The subscale included health center partnerships reported as 39 primary sponsors over time. Types of agencies included local departments of health, community health centers, school systems, hospital/medical centers, mental health agencies, universities, private/non-profit organizations, tribal government systems, and federally qualified health centers. Community health centers and hospital/medical centers were most frequently reported as primary SBHC sponsors from T1 to T3. At T4, hospital/medical centers and school systems were most frequently reported as primary sponsors. Table 8 provides a summary of the most frequently reported health center partnerships by the percentage of sites at each time-point. Table 8: Percent of Sites’ Reported Health Center Partnerships by Time Agency Sponsor Local Department of Health Community Health System School System Hospital/Medical Center Mental Health Agency University T1 18% 22% 15% 28% 1% 4% T2 17% 28% 13% 24% 1% 3% Private, non-profit Organization 12% 11% T3 13% 30% 13% T4 8% 0% 15% 29% 18% 2% 4% 7% 5% 3% 8% Outer Setting Variables The State Policy Survey - Funding Sources subscale. The Funding Sources subscale explored state funding directly dedicated to SBHCs over the four time points. The subscale included the following variables: (1) presence/absence of a state government funding or sponsoring a grant program dedicated to SBHCs; (2) grant program funding sources (e.g., Title V MCH, State general fund; Tobacco Settlement); and (3) state agency responsible for allocation of grants or funds to state’s SBHC program. Frequencies showed that more than 70% of states had a grant program dedicated specifically to SBHCs from 2005-2014. Funding sources were primarily from state general funds. Moreover, public health agencies were the most frequently reported agencies responsible for allocation of funds, see Table 9. 40 Table 9: Summary of External Policy & Incentives by Percentage of Sites (N) and Time Funding Sources Subscale SBHC Grant Program T1 T2 T3 T4 Grant Program dedicated to SBHCs No program available Grant Program Funding Source Title V MCH Block Tobacco Settlement State General Fund State Agency Public Health Only Human Services Only Public Health & Education Public Health & Human Services None Oversight & Support Subscale Technical Assistance Unit provides TA to SBHCs Does not provide TA SBHC Data State collects data State does not collect data Policy & Standards Subscale Operating Standards Yes, for state funded SBHCs only Yes, for all SBHCs regardless of funding source No operating standards State Requirements for Data Collection Mandatory for state funded SBHCs Mandatory for all SBHCs regardless of funding source Mandatory for state funded SBHCs and Voluntary for SBHCs not funded by state 81% 17% 44% 13% 66% 54% 5% 5% 2% 28% 62% 39% 27% 4% 59% 53% 0% 7% 0% 41% 72% 26% 38% 23% 65% 61% 10% 4% 3% 22% 84% 15% 31% 11% 61% 74% 4% 6% 0% 16% 69% 29% 80% 4% 48% 23% 2% 60% 17% 0% 80% 19% 71% 9% 49% 31% 18% 34% 12% 25% 63% 26% 72% 13% 46% 7% 29% 49% 13% 12% 60% 2% 62% 39% 40% 19% 1% 40% 3% 15% The State Policy Survey - Oversight and Support subscale. The Oversight and Support subscale explored external strategies in recommendations, guidelines, and public reporting provided to program office and staff to administer SBHCs. The subscale included the following variables: (1) absence/presence of a state program responsible for providing technical assistance to communities interested in developing and/or operating existing SBHCs; and (2) presence/absence of state government collecting data from SBHCs. Overall, more than 60% of 41 states reported a program responsible for providing technical assistance consistently from 2005- 2014. Further, more than half of states reported that their government collected data from SBHCs for performance measures throughout the time points (Table 9). The State Policy Survey - Policy and Standards subscale. The Policy and Standards subscale explored external mandates and governmental regulations that facilitate standards needed for SBHC programs. The subscale included the following variables: (1) state requirements for SBHC operating standards and (2) state requirements for data collection across the four time points. State requirements for SBHC operating standards included, “Yes, for state funded SBHCs only” and “Yes, for all SBHCs regardless of funding source.” Operating standards for SBHCs funded by the state was the most frequently reported from 2005-2014. State requirements for data collection were, “Mandatory for SBHCs funded by state,” “Mandatory for SBHCs funded and not funded by state,” and “Mandatory for SBHCs funded by state and voluntary for SBHCs not funded by state.” State requirements that were mandatory only for SBHCs funded by state were most frequently reported from 2005-2014 (Table 9). LMM Analyses: Inner Setting Variables & MH Services Screening & assessment services. Results from the intercept-only model showed that there was significant variation of services accounted for by state (F = 27.740, df = 40.302, p < .000). Results from the intercept-slope model showed that time also significantly accounted for variation of services (F = 71.896, df = 3, p < .000), with screening/assessment services significantly increasing from 2005 to 2014. To minimize confounding variables, time was controlled in the adjusted fixed effects model. Results from the adjusted model showed that geographic location, year of establishment, school enrollment, location of behavioral health 42 provider, hours open weekly, and agency sponsor were significantly associated with more screening/assessment services, when controlled for time (Table 10). Table 10: LMM: Type III Test of Fixed Effects for Inner Setting Variables on Screening/Assessment Services, Controlled for Time Source Intercept Geographic Location Year Established School Enrollment Elementary School Middle School High School Days Open Weekly Hours Open Weekly Total MH Hours Behavioral Health Provider Title I Charter School Alternative School Category Vocational Category Magnet Public School Agency Sponsor Time Type III Tests of Fixed Effects Numerator df Denominator df 1 2 5 2 1 1 1 1 2 1 3 1 1 1 1 1 1 9 3 43.720 3422.163 3422.116 3422.272 3422.683 3422.324 3422.549 3423.384 3422.388 3421.446 3422.346 3421.599 3421.249 3421.718 3423.814 3422.567 3421.525 3423.186 3421.279 F 26.862 11.590 5.556 9.682 2.369 1.241 1.150 1.607 9.457 .843 4.110 .013 .186 1.625 .793 6.670 .241 2.908 202.103 Sig. .000 .000 .000 .000 .124 .265 .284 .205 .000 .359 .006 .911 .666 .202 .373 .010 .624 .002 .000 More specifically, sites in rural locations, with more recent year of establishment (i.e., 2010+), school enrollment of 0-699 students, 31 or more hours open weekly, a behavioral health provider co-located within the center, school characterization reported as magnet school, and sponsorship with a mental health agency reported a significantly higher number of screening/assessment services than sites that did not report these factors from 2005 to 2014 (Table 11). 43 Table 11: Estimated Marginal Means for Screening/Assessment by Inner Setting Variables Estimated Marginal Means Mean SE df 95% Confidence Interval Lower Bound Upper Bound Geographic Location Rural Suburban Urban Year of Establishment 1989-below 1990-1994 1995-1999 2000-2004 2005-2009 2010+ School Enrollment 0-699 700-1999 2000+ Behavioral Health Provider Not in school In school, but separate from center 25.032 4.576 44.241 15.810 34.253 22.812 4.569 43.959 13.604 32.020 23.099 4.566 43.855 13.896 32.303 23.069 4.595 44.990 13.813 32.324 22.237 4.582 44.472 13.005 31.469 23.142 4.570 44.012 13.931 32.353 22.918 4.570 44.007 13.708 32.129 24.357 4.576 44.233 15.136 33.578 26.164 4.609 45.541 16.883 35.445 24.586 4.561 43.658 15.391 33.780 24.492 4.565 43.818 15.290 33.693 21.866 4.597 45.048 12.607 31.125 24.073 4.583 44.489 14.840 33.305 23.718 4.563 43.721 14.521 32.915 In school and co-locate within center 24.625 4.576 44.221 15.405 33.846 Hours Open Weekly 8 or less hours 9 to 30 hours 31 or more hours Magnet School No 22.306 4.585 44.586 13.069 31.543 23.820 4.574 44.164 14.602 33.038 24.817 4.568 43.920 15.610 34.023 22.746 4.551 43.258 13.571 31.922 Yes Agency Sponsor Local Department of Health Community Health System School System Hospital/Medical Center Mental Health Agency University Private, non-profit Organization a. Dependent Variable: ScreeningAssessment_cgt. 24.549 4.601 45.219 15.283 33.816 24.175 4.571 44.030 14.963 33.386 23.676 4.559 43.595 14.485 32.867 24.264 4.576 44.241 15.042 33.485 24.215 4.557 43.514 15.027 33.402 27.639 4.714 49.827 18.169 37.109 21.914 4.640 46.739 12.579 31.249 22.641 4.577 44.277 13.418 31.864 Medication management services. Results from the first model showed that there was significant variation of services accounted for by state (F = 26.231, df = 40.227, p < .000). Results from the second model showed that time also significantly accounted for variation of services (F = 107.275, df = 3, p < .000), with medication management services increasing from 44 2005 to 2014. Similar to the previous outcome variable, time was controlled in the adjusted fixed effects model. Results from the adjusted model demonstrated that geographic location, year of establishment, school enrollment, grade level, location of behavioral health provider, hours open weekly, school characterization, and agency sponsor was significantly associated with more medication management services over time (Table 12). Table 12: LMM: Type III Test of Fixed Effects for Inner Setting Variables on Medication Management Services, Controlled for Time Type III Tests of Fixed Effects Source Numerator df Denominator df F 51.138 3425.851 3425.746 3426.093 3427.139 3426.221 3426.854 3428.905 26.005 6.689 14.246 3.629 15.000 .000 1.009 .267 Sig. .000 .001 .000 .027 .000 .986 .315 .606 3426.170 13.080 .000 3424.029 3426.350 1.099 13.317 3424.455 3423.393 3424.640 3429.892 3426.764 3424.169 3428.379 3423.466 .000 2.845 .047 1.734 .177 4.784 2.378 285.980 .294 .000 .989 .092 .829 .188 .674 .029 .011 .000 45 Intercept Geographic Location Year Established School Enrollment Elementary School Middle School High School Days Open Weekly Hours Open Weekly Total MH Hours Behavioral Health Provider Title I Charter School Alternative School Category Vocational Category Magnet Public School Agency Sponsor Time 1 2 5 2 1 1 1 1 2 1 3 1 1 1 1 1 1 9 3 More specifically, sites reported in urban locations, with year of establishment from 2005-2009, school enrollment of 700-1999 students, serving elementary schools, 31 or more hours open weekly, no behavioral health provider in school, school characterization not reported as public school, and agency sponsorship with hospital/medical center reported a significantly higher number of medication management services than sites with other contexts from 2005 to 2014 (Table 13). Table 13: Estimated Marginal Means for Medication Management by Inner Setting Variables Marginal Means Estimates Geographic Location Rural Suburban Urban Year of Establishment 1989-below 1990-1994 1995-1999 2000-2004 2005-2009 2010+ School Enrollment 0-699 700-1999 2000+ Elementary School No Yes Behavioral Health Provider Not in school In school, but separate from center In school and co-locate within center Hours Open Weekly 8 or less hours 9 to 30 hours 31 or more hours Public School No Yes Agency Sponsor Local Department of Health Community Health System School System Hospital/Medical Center Mental Health Agency University Private, non-profit Organization Mean SE 16.266 3.366 16.882 18.104 3.362 3.357 17.250 3.414 14.318 15.636 16.956 19.974 18.370 3.388 3.365 3.365 3.376 3.441 16.874 17.908 16.470 16.174 17.994 19.379 17.340 17.725 15.017 17.635 18.599 17.609 16.559 17.442 18.132 16.352 19.190 15.218 19.002 18.194 3.347 3.355 3.417 3.358 3.358 3.389 3.350 3.376 3.394 3.373 3.360 3.340 3.377 3.366 3.344 3.377 3.339 3.640 3.499 3.378 df 52.768 51.886 51.565 55.139 53.503 52.047 52.042 52.740 56.921 50.949 51.440 55.332 51.642 51.640 53.552 51.138 52.705 53.879 52.519 51.755 50.534 52.804 52.084 50.753 52.745 50.478 71.204 60.766 52.875 95% Confidence Interval Lower Bound Upper Bound 9.493 10.135 11.366 10.408 7.524 8.884 10.204 13.202 11.479 10.154 11.173 9.624 9.434 11.254 12.583 10.614 10.953 8.212 10.869 11.856 10.902 9.785 10.688 11.419 9.579 12.484 7.960 12.005 11.417 23.039 23.629 24.843 24.091 21.113 22.388 23.708 26.747 25.261 23.593 24.642 23.317 22.914 24.734 26.175 24.065 24.496 21.822 24.402 25.343 24.316 23.333 24.196 24.846 23.125 25.896 22.477 25.999 24.970 46 Table 13 (cont’d) a. Dependent Variable: MedManagement_cgt Substance use treatment services. Results from the first model showed that there was significant variation of services accounted for by state (F = 31.927, df = 41.451, p<.000). Results from the second model showed that time also significantly accounted for variation of services (F = 306.812, df = 3, p < .000), with substance use treatment services decreasing from 2005-2014. Results from the adjusted model demonstrated that geographic location, year of establishment, grade level, hours open weekly, school characterization, and agency sponsor was significantly associated with more services over time (Table 14). Table 14: LMM: Type III Test of Fixed Effects for Inner Setting Variables on Substance Use Treatment Services, Controlled for Time Source Intercept Geographic Location Year Established School Enrollment Elementary School Middle School High School Days Open Weekly Hours Open Weekly Total MH Hours Behavioral Health Provider Title I Charter School Alternative School Category Vocational Category Magnet Public School Agency Sponsor Time Type III Tests of Fixed Effects Numerator df Denominator df 1 2 5 2 1 1 1 1 2 1 3 1 1 1 1 1 1 9 3 57.155 3428.425 3428.294 3428.713 3430.130 3428.885 3429.796 3432.509 3428.638 3425.975 3429.124 3426.584 3424.983 3426.714 3433.693 3429.531 3426.100 3431.730 3425.086 F 17.241 14.118 5.366 2.556 8.899 8.954 12.207 2.463 7.903 .480 2.073 2.626 6.258 .069 4.125 2.899 14.847 5.682 611.703 Sig. .000 .000 .000 .078 .003 .003 .000 .117 .000 .489 .102 .105 .012 .793 .042 .089 .000 .000 .000 a. Dependent Variable: SubUseTreatment_cgt. 47 More specifically, sites reported in urban locations, with a more recent year of establishment (i.e., 2010+), serving all 3 grade levels (pre-k-5, elementary, and high school), school characterization reported as vocational, but not public or charter school, and agency sponsorship with private/nonprofit organizations reported a significantly higher number of services than sites with other contexts from 2005 to 2014 (Table 15). Table 15: Estimated Marginal Means of Substance Use Treatment Services by Inner Setting Variables Mean Std. Error Marginal Means Estimates Geographic Location Rural Suburban Urban Year of Establishment 1989-below 1990-1994 1995-1999 2000-2004 2005-2009 2010+ Elementary School No Yes Middle School No Yes High School No Yes Hours Open Weekly 8 or less hours 9 to 30 hours 31 or more hours Charter School No Yes Vocational School No Yes Public School No Yes Agency Sponsor Local Department of Health Community Health System School System Hospital/Medical Center Mental Health Agency 13.729 14.066 16.769 12.974 14.872 12.797 15.412 15.583 17.488 13.969 15.740 14.030 15.679 13.771 15.938 16.009 12.759 15.796 16.775 12.934 13.481 16.228 16.023 13.686 15.564 15.217 15.636 14.271 5.096 3.617 3.596 3.588 3.673 3.634 3.600 3.599 3.616 3.713 3.590 3.590 3.591 3.585 3.615 3.567 3.643 3.611 3.593 3.495 3.815 3.522 3.756 3.563 3.618 3.601 3.568 3.617 3.561 4.005 48 df 59.704 58.328 57.829 63.445 60.869 58.574 58.579 59.660 66.300 57.947 57.936 58.016 57.649 59.558 56.488 61.479 59.309 58.115 52.094 73.860 53.702 69.363 56.211 59.769 58.611 56.559 59.646 56.106 89.506 95% Confidence Interval Lower Bound Upper Bound 6.493 6.869 9.586 5.636 7.604 5.593 8.208 8.348 10.075 6.783 8.554 6.842 8.502 6.539 8.794 8.725 5.533 8.605 9.761 5.331 6.418 8.736 8.887 6.448 8.358 8.071 8.400 7.136 -2.861 20.965 21.263 23.951 20.313 22.140 20.002 22.616 22.818 24.901 21.155 22.926 21.218 22.856 21.003 23.082 23.292 19.984 22.987 23.788 20.537 20.544 23.719 23.159 20.923 22.770 22.363 22.871 21.405 13.053 University Private, non-profit Organization a. Dependent Variable: SubUseTreatment_cgt. 16.994 17.698 3.798 3.620 72.396 59.868 9.424 10.457 24.564 24.939 Referral services. Results from the first model showed that there was significant variation of services accounted for by state (F = 24.558, df = 40.510, p < .000). Results from the second model showed that time also significantly accounted for variation of services (F = 567.985, df = 3, p < .000), with referrals services increasing from 2005-2014. Results from the adjusted model demonstrated that geographic location, school enrollment, hours open weekly, school characterization, and agency sponsor was significantly associated with more services over time, even after controlling for time (Table 16). Table 16: LMM: Type III Test of Fixed Effects for Inner Setting Variables on Referrals, Controlled for Time Source Intercept Geographic Location Year Established School Enrollment Elementary School Middle School High School Days Open Weekly Hours Open Weekly Total MH Hours Behavioral Health Provider Title I Charter School Alternative School Category Vocational Category Magnet Public School Agency Sponsor Time a. Dependent Variable: Referrals_cgt Type III Tests of Fixed Effects Numerator df Denominator df F 1 2 5 2 1 1 1 1 2 1 3 1 1 1 1 1 1 9 3 60.433 3429.649 3429.507 3429.949 3431.555 3430.144 3431.208 3434.244 3429.763 3426.889 3430.454 3427.598 3425.682 3427.668 3435.476 3430.816 3426.990 3433.302 3425.805 20.348 16.672 2.141 5.049 .167 2.638 .959 .232 13.787 .142 .909 7.780 .010 .781 1.311 6.337 1.823 2.789 1356.301 Sig. .000 .000 .058 .006 .683 .104 .328 .630 .000 .707 .436 .005 .921 .377 .252 .012 .177 .003 .000 49 More specifically, Table 17 reports that sites in rural locations, with school enrollment of more than 2010, 31 or more hours open weekly, school characterization reported as Title I or magnet school, and agency sponsorship with mental health agencies reported a significantly higher number of referral services than sites with other contexts from 2005 to 2014. Table 17: Estimated Marginal Means of Referrals by Inner Setting Variables Marginal Means Estimates Mean Std. Error Geographic Location Rural Suburban Urban Year of Establishment 1989-below 1990-1994 1995-1999 2000-2004 2005-2009 2010+ School Enrollment 0-699 700-1999 2000+ Hours Open Weekly 8 or less hours 9 to 30 hours 31 or more hours Title I No Yes Magnet School No Yes Agency Sponsor Local Department of Health Community Health System School System Hospital/Medical Center Mental Health Agency University Private, non-profit Organization a. Dependent Variable: Referrals_cgt 22.063 17.858 17.147 19.911 16.952 19.086 17.973 18.937 21.279 20.444 20.071 16.554 15.413 20.439 21.216 18.005 20.040 17.339 20.707 18.512 18.625 18.981 18.580 27.412 14.408 16.433 4.271 4.242 4.231 4.346 4.294 4.247 4.247 4.270 4.400 4.211 4.227 4.352 4.306 4.263 4.238 4.232 4.234 4.168 4.369 4.248 4.204 4.270 4.195 4.791 4.514 4.274 df 63.524 61.857 61.253 68.081 64.947 62.151 62.166 63.470 71.588 60.085 60.997 68.470 65.701 63.042 61.593 61.293 61.380 57.689 69.570 62.182 59.713 63.438 59.148 100.269 79.027 63.720 95% Confidence Interval Lower Bound Upper Bound 13.531 9.379 8.687 11.239 8.376 10.596 9.484 10.406 12.506 12.021 11.618 7.871 6.816 11.921 12.744 9.543 11.576 8.994 11.993 10.020 10.214 10.449 10.186 17.907 5.423 7.894 30.596 26.338 25.607 28.582 25.528 27.575 26.461 27.468 30.051 28.866 28.523 25.236 24.011 28.957 29.688 26.467 28.505 25.684 29.421 27.004 27.035 27.512 26.974 36.917 23.392 24.972 LMM Analyses: Outer Setting Variables & MH Services 50 Similarly, the same process used for inner setting variables and mental health services was carried out with the outer setting variables. Results showed that most outer context variables—with the exception of state general funds on substance use treatment and referrals— were significantly related to the number of screening/assessment services, medication management services, substance use treatment services, and referral services reported by states, even after controlling for time (Table 18, 19, 20, 21). More specific results for the main effects of outer setting variables on each dependent variable (controlled for time) is reported in the following sections. Table 18: Type III Tests of Fixed Effects for Outer Context Variables on Screening/Assessment, Controlled for Time Type III Tests of Fixed Effects Numerator df Denominator df F State Requirements for Data Collection a. Dependent Variable: ScreeningAssessment_cgt Table 19: Type III Tests of Fixed Effects for Outer Context Variables on Medication Management, Controlled for Time. Source Intercept Time TechnicalAssistance State Agency SBHC Data SBHC Grant Program Title V MCH Type III Tests of Fixed Effects Numerator df Denominator df F 21.062 2641.836 2649.135 2644.021 3231.054 2638.676 2648.588 22.836 419.327 30.413 91.816 153.622 17.818 224.561 1 3 1 4 1 1 1 51 Sig. .000 .000 .000 .000 .000 .000 .000 Source Intercept Time TechnicalAssistance State Agency SBHC Data SBHC Grant Program Title V MCH State General Fund Tobacco Settlement Operating Standards 1 3 1 4 1 1 1 1 1 2 2 20.119 2638.738 2643.838 2640.177 3220.288 2636.673 2643.394 2643.310 2639.881 2641.622 2639.116 61.275 274.532 58.682 6.785 44.533 341.751 51.708 282.107 937.849 13.632 215.651 Sig. .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 State General Fund Tobacco Settlement Operating Standards State Requirements for Data Collection a. Dependent Variable: Medmanagement_cgt 1 1 2 2 2648.476 2643.782 2646.082 2642.577 59.613 1360.565 225.891 562.686 .000 .000 .000 .000 Table 20: Type III Tests of Fixed Effects for Outer Context Variables on Substance Use Treatment, Controlled for Time Type III Tests of Fixed Effects Numerator df Denominator df F State Requirements for Data Collection a. Dependent Variable: SubUseTreatment_cgt Table 21: Type III Tests of Fixed Effects for Outer Context Variables on Referrals, Controlled for Time Type III Tests of Fixed Effects Numerator df Denominator df F Sig. State Requirements for Data Collection a. Dependent Variable: Referrals_cgt. Screening & assessment services. For the funding sources subscale, sources from Tobacco Settlement demonstrated more services than state general fund or Title V MCH. 52 Source Intercept Time TechnicalAssistance State Agency SBHC Data SBHC Grant Program Title V MCH State General Fund Tobacco Settlement Operating Standards Source Intercept Time TechnicalAssistance State Agency SBHC Data SBHC Grant Program Title V MCH State General Fund Tobacco Settlement Operating Standards 1 3 1 4 1 1 1 1 1 2 2 20.659 2640.641 2647.251 2642.567 3224.664 2637.865 2646.718 2646.612 2642.274 2644.436 2641.229 63.678 735.734 87.186 55.277 65.929 4.631 123.603 .017 2992.999 908.646 1443.066 Sig. .000 .000 .000 .000 .000 .031 .000 .896 .000 .000 .000 1 3 1 4 1 1 1 1 1 2 2 25.652 105.821 .000 2648.301 1048.798 .000 2640.123 2648.808 3190.017 41.820 30.347 14.650 2646.836 107.074 2643.576 2644.340 2653.000 2644.436 7.718 .014 201.205 908.646 .000 .000 .000 .000 .006 .905 .000 .000 2641.229 1443.066 .000 Notably, states with a grant program dedicated to SBHCs that did not report any agency as a funding source demonstrated lower services than those that did not. For the oversight and support subscale, states with a program unit to provide technical assistance and data collection reported more screening and assessment services than sites that did not have technical assistance over time. For the policy and standards subscale, states with requirements for SBHC data collection reported as mandatory for SBHCs funded by state and voluntary for SBHCs not funded by state demonstrated more services than any other criteria. States with any operating standard demonstrated less services than those without operating standards. See Table 22 for more results. 53 Table 22: Estimated Marginal Means of Screening/Assessment by Outer Setting Variables Estimated Marginal Means 95% Confidence Funding Sources Subscale Mean SBHC Grant Program Interval Std. Error df Lower Bound Upper Bound Grant Program Dedicated to 43.608 7.099 19.739 28.787 58.429 SBHCs No program available 68.065 7.228 21.206 53.043 83.086 Grant Program Funding Source Title V MCH Block Grant Tobacco Settlement State General Fund State Agency Public Health only Human Services only Public Health & Education Public Health & Human Services None 53.593 7.139 20.190 66.508 7.154 20.361 61.099 7.147 20.282 54.689 7.078 19.504 57.581 7.190 20.766 53.424 7.122 19.996 49.457 7.210 20.995 38.710 51.601 46.203 39.901 42.618 38.568 34.464 68.476 81.415 75.995 69.477 72.543 68.281 64.451 64.030 8.230 35.594 47.332 80.728 Oversight & Support Subscale Technical Assistance Unit provides TA to SBHCs Does not provide TA SBHC Data State collects data State does not collect data 60.755 7.127 20.055 50.917 7.196 20.839 45.890 35.945 75.620 65.890 28.610 3.872 42.611 21.185 6.308 20.874 20.800 8.061 36.420 34.310 Policy & Standards Subscale Operating Standards Yes, for state funded SBHCs 53.894 7.159 20.412 38.981 68.808 only Yes, for all SBHCs regardless 54.265 7.159 20.418 39.350 69.180 of funding source No operating standards 59.349 7.143 20.238 44.459 74.239 State requirements for data collection Mandatory for SBHCs funded 50.936 7.137 20.165 36.056 65.816 by state Mandatory for all SBHCs 51.368 7.165 20.477 36.445 66.291 regardless of funding source Mandatory for SBHCs funded by state & Voluntary for SBHCs not funded by state 65.205 7.136 20.156 50.326 80.083 a. Dependent Variable: ScreeningAssessment_cgt Medication management services. For the funding sources subscale, states with funding sources from public health and human agencies and Tobacco Settlement demonstrated more services than any other funding source. However, states with a grant program dedicated to 54 SBHCs demonstrated lower services than those that did not. For the oversight and support subscale, states with a program unit to provide technical assistance and collect data reported more medication management services than those that did not have technical assistance. For the policy and standards subscale, states with requirements for SBHC data collection reported as mandatory for SBHCs funded by state and voluntary for SBHCs not funded by state demonstrated higher number of services than any other criteria. However, states that reported presence of operating standards demonstrated lower services than those without operating standards. See Table 23 for more results. 55 Table 23: Estimated Marginal Means of Medication Management by Outer Setting Variables Estimated Marginal Means 95% Confidence Interval Funding Sources Subscale Mean SBHC Grant Program Std. Error df Lower Bound Grant Program Dedicated to SBHCs No program available 25.455 5.924 20.369 31.644 6.112 23.073 13.111 19.002 Grant Program Funding Source Title V MCH Block Grant Tobacco Settlement State General Fund State Agency Public Health only Human Services only Public Health & Education Public Health & Human Services None 23.373 5.984 21.193 42.790 6.006 21.506 31.229 5.995 21.360 37.955 5.893 19.944 9.644 6.057 22.247 36.533 5.958 20.840 41.944 6.086 22.677 16.671 7.509 52.270 Oversight & Support Subscale Technical Assistance 10.936 30.318 18.773 25.660 -2.910 24.136 29.344 1.605 Unit provides TA to SBHCs Does not provide TA SBHC Data State collects data State does not collect data 32.471 5.966 20.945 24.628 6.067 22.388 20.062 12.059 28.610 3.872 42.611 21.185 6.308 20.874 20.800 8.061 Policy & Standards Subscale Operating Standards Upper Bound 37.799 44.286 35.809 55.261 43.684 50.250 22.198 48.930 54.544 31.737 44.880 37.196 36.420 34.310 Yes, for state funded SBHCs only Yes, for all SBHCs regardless of 25.906 6.012 21.598 17.161 6.013 21.609 13.424 4.678 38.388 29.644 funding source No operating standards 42.581 5.990 21.282 30.135 55.027 State requirements for data collection Mandatory for SBHCs funded by state 29.737 5.980 21.147 11.738 6.021 21.718 Mandatory for all SBHCs regardless 17.305 -.757 42.168 24.233 of funding source Mandatory for SBHCs funded by state 44.173 5.979 21.131 31.744 56.603 & Voluntary for SBHCs not funded by state a. Dependent Variable: Medmanagement_cgt Substance use treatment services. For the funding sources subscale, states with funding sources from public health and human agencies and Tobacco Settlement demonstrated more services than any other funding source. However, having a state grant program dedicated to 56 SBHCs demonstrated lower services than those that did not. For the oversight and support subscale, states with a program unit that provided technical assistance demonstrated more services than those that did not have technical assistance over time. However, states with a government unit to collect SBHC data reported less services than sites that did not. For the policy and standards subscale, states with requirements for SBHC data collection reported as mandatory for SBHCs regardless of funding source demonstrated higher number of services than any other criteria. States with operating standards only for state funded SBHCs demonstrated more services than those that did not have any operating standards. See Table 24 for more results. 57 Table 24: Estimated Marginal Means of Substance Use Treatment by Outer Setting Variables Estimated Marginal Means Funding Sources Subscale Mean SBHC Grant Program Std. Error df 95% Confidence Interval Lower Bound Upper Bound Grant Program Dedicated to 48.960 6.282 20.095 35.859 62.060 SBHCs No program available 52.000 6.447 22.287 38.640 65.361 Grant Program Funding Source Title V MCH Block Grant Tobacco Settlement State General Fund State Agency Public Health only Human Services only Public Health & Education Public Health & Human Services None 46.778 6.334 20.765 70.838 6.353 21.019 50.524 6.345 20.901 42.703 6.255 19.747 52.283 6.399 21.622 48.286 6.312 20.478 60.656 6.424 21.968 33.596 57.626 37.326 29.645 38.999 35.139 47.332 59.959 84.049 63.722 55.761 65.567 61.432 73.980 45.504 6.871 30.679 31.485 59.522 Oversight & Support Subscale Technical Assistance Unit provides TA to SBHCs Does not provide TA SBHC Data State collects data State does not collect data 56.881 6.319 20.564 44.079 6.407 21.734 43.724 30.782 70.038 57.376 12.137 4.074 39.076 25.432 4.260 46.699 3.898 16.861 20.376 34.004 Policy & Standards Subscale Operating Standards Yes, for state funded SBHCs 66.696 6.359 21.094 53.475 79.917 only Yes, for all SBHCs regardless of 43.622 6.360 21.103 30.400 56.844 funding source No operating standards 41.122 6.339 20.837 27.932 54.312 State requirements for data collection Mandatory for SBHCs funded 31.733 6.331 20.728 18.556 44.910 by state Mandatory for all SBHCs 61.963 6.367 21.192 48.730 75.195 regardless of funding source Mandatory for SBHCs funded by state & Voluntary for SBHCs not funded by state 57.744 6.330 20.714 44.569 70.919 a. Dependent Variable: SubUseTreatment_cgt 58 Referral services. For the funding sources subscale, states with funding sources from human agencies only and Tobacco Settlement demonstrated more services than any other funding source. However, having a state grant program dedicated to SBHCs demonstrated lower services than those that did not. For the oversight and support subscale, states with a program unit to provide technical assistance and data collection demonstrated less services than those without a program unit. For the policy and standards subscale, states with requirements for SBHC data collection reported as mandatory for SBHCs regardless of funding source demonstrated higher number of services than any other criteria. States with operating standards for all SBHCs regardless of funding source demonstrated more services than those that did not have any operating standards. See Table 25 for more results. 59 Table 25: Estimated Marginal Means of Referrals by Outer Setting Variables Estimated Marginal Means Funding Sources Subscale Mean SE SBHC Grant Program df Grant Program Dedicated to SBHCs No program available Grant Program Funding Source Title V MCH Block Grant Tobacco Settlement State General Fund State Agency Public Health only Human Services only Public Health & Education Public Health & Human Services None 51.609 6.210 79.405 6.793 23.208 33.169 63.754 6.397 75.526 6.466 65.431 6.433 26.132 27.241 26.687 21.774 59.423 29.850 83.623 24.893 66.843 38.371 31.587 79.275 10.481 177.997 6.110 6.622 6.318 6.712 Technical Assistance Oversight & Support Subscale Unit provides TA to SBHCs Does not provide TA SBHC Data State collects data State does not collect data 57.110 73.904 6.342 6.653 25.224 30.476 17.555 24.910 3.985 4.246 39.206 50.492 Operating Standards Policy & Standards Subscale 95% Confidence Interval Lower Bound 38.770 65.586 50.608 62.265 52.224 46.744 70.096 53.827 24.691 58.593 44.055 60.326 9.497 16.384 Upper Bound 64.449 93.223 76.901 88.787 78.639 72.101 97.150 79.859 52.051 99.958 70.165 87.481 25.613 33.436 Yes, for state funded SBHCs only Yes, for all SBHCs regardless of funding 61.124 76.550 6.485 6.487 27.556 27.569 47.830 63.252 74.419 89.848 source No operating standards 58.846 6.417 26.481 45.667 72.026 State requirements for data collection Mandatory for SBHCs funded by state Mandatory for all SBHCs regardless of 57.655 74.929 6.387 6.511 25.951 27.998 44.526 61.591 70.784 88.267 funding source Mandatory for SBHCs funded by state & Voluntary for SBHCs not funded by state 63.937 6.384 25.923 50.813 77.061 a. Dependent Variable: Referrals_cgt. 60 DISCUSSION The current study used descriptive analyses to explore the reported number of mental health (MH) services offered by SBHCs across the U.S. by conducting secondary analyses on the National Census SBHC Survey and the State Policy Survey. Further, the study used linear mixed modelling to explore the association between inner and outer setting variables (theorized within the CFIR framework) with SBHC mental health service delivery. Quantitative results from the current project may help promote understanding the role of structural characteristics and state level support in shaping MH service delivery over time. Interpretation of the data is presented by each research question in the following sections. RQ1. What is the number of mental health services reported to be delivered by state from 2005 to 2014? The first research question aimed to assess the number of MH services reported to be delivered by state from 2005 to 2014. Descriptive statistics demonstrated that there was variation in MH services reported by each state. Further analyses demonstrated that variation in the number of MH services reported to be delivered was significantly accounted for by state and time. Screening/assessment, medication management, and referral services significantly increased from 2005 to 2014, whereas, substance use treatment services significantly decreased over time. Given the significant variation of services, evaluating the inner and outer contextual variables that may support the capacity of some SBHCs to provide a greater number of MH services than other SBHCs is critically important. 61 RQ2. Which inner setting variables (structural characteristics and networks) are related to number of mental health services reported to be delivered from 2005 to 2014? The second research question aimed to explore the relationship between inner setting variables and MH services reported to be delivered from 2005 to 2014. Linear mixed modelling demonstrated an estimation, or estimated model, of the significance in the relationship between inner setting variables and delivery of MH services over time. Mean estimates allowed for comparison of reported number of services provided by sites at the state level. Overall, inner setting variables that were significantly associated with the number of MH services reported to be delivered over time included geographic location, year of establishment, school enrollment, hours open weekly, grade levels, school characterization, behavioral health provider, and agency sponsor. Total hours worked for MH staff and days open weekly were not significant for any MH service. The National Census Survey - Structural Characteristics subscale. Organizational level factors, such as size, location, and length of establishment are suggested to shape availability of resources and influence utilization of services (Greenhalgh et al., 2004; Wade et al., 2008). Results showed that school enrollment below 2000 was significantly associated with more screening/assessment, medication management, and referral services over time, but not substance use treatment. Findings support that the size of a student body can help shape delivery of services over time by directing the types of services necessary to meet student needs. This finding should be interpreted with caution because school enrollment of more than 2000 was the least reported characteristic over the four time points, which may have skewed mean estimates. Maturity was informed by the site’s reported year of establishment. Results showed that sites with a more recent year of establishment reported more screening/assessment, medication 62 management, and substance use treatment services over time, but not referral services. This finding could reflect trajectories SBHCs have undergone, with more recent establishments having more resources due to increased community support or recognition of impacts (Dreyfoos, 1994; Flaherty et al., 1996; Swider & Valukas, 2004). Geographic location was significantly related to all four MH services over time. Sites in rural locations reported more screening/assessment and referral services than sites in urban or suburban locations. Sites in urban locations reported more medication management and substance use treatment services than sites in rural or suburban locations. These findings are inconsistent with previous studies asserting that geographic location was not found to have any effect on availability of MH services in SBHCs (Larson et al., 2017). Current findings suggest that there is potentially some interaction from geographic location in shaping delivery of MH services over time. For example, it is possible that sites in rural locations are more likely to refer students out to other sources because rural locations typically have less access to diverse services. Grade level was significantly related to medication management and substance use treatment services only. Sites that offered services to elementary schools reported more medication management services than those that did not. Sites that offered services to all grade levels reported more substance use treatment services than those that did not. This relationship was not observed for screening/assessment or referral services. These findings suggest that there are specific types of services more often to be provided for a given grade level. For example, it is possible that there is a higher need for medication management than any other service in elementary schools due to higher prevalence rates of mental health diagnoses, such as ADHD, at this age group that are often treated with medication (Baquiran, Webber, & Appel, 2002). 63 Findings are also inconsistent with previous research suggesting that higher grade levels predicted more services (Larson et al., 2017). Future research should assess the pattern of utilization among younger grade levels (elementary grade) that are suggested to direct an increase in medication management. Prior studies have found that SBHCs with a mental health component tended to have more “organizational resources,” such as more hours and days open weekly (Larson et al., 2017). The current study further assessed the direction of these relationships within sites with a MH component. Hours open weekly was significantly associated with the number of services reported to be delivered over time, but this relationship was not found significant with days open weekly. Sites with 31 or more hours open weekly reported more screening/assessment, medication management, and referral services over data collection time points than sites with less hours. These findings suggest that sites may need to accommodate for a higher demand of services with more hours open. Such information may support advocating for more SBHC organizational resources to operate sufficiently. Interestingly, sites that were open eight hours or less per week reported more substance use treatment services than sites open for more than eight hours per week. This finding might be influenced by prevention activities (which were not analyzed) targeting substance use behaviors that are taking place outside of SBHC MH services. Nonetheless, current findings offer more information on the direction of the relationship between health center operations and number of MH services reported to be delivered over time. As previously discussed, among schools with a SBHC, 30% still do not have a mental health provider on staff (School-Based Health Alliance, 2016; Larson et al., 2017). Moreover, mental or behavioral health providers on staff within SBHCs can be essential in expanding to more comprehensive MH care. In the current study, majority of sites reported having a 64 behavioral health (BH) provider either on staff (i.e., co-located within the center) or affiliated with the school (i.e., located in school, but separate from center). However, results found that location of a BH provider was only significantly associated with screening/assessment and medication management services, and not with referral and substance use services. Sites with a BH provider co-located within the center reported more screening/assessment services than any other provider location reported over time. Sites that did not have a BH provider on site reported more medication management services over time than sites with a BH provider. There is a discussion about how SBHCs are often concerned with issues related to duplicated services and coordinated care with school-employed health practitioners (Larson et al., 2017; Richardson, 2007). Perhaps, certain services are coordinated with school-employed health practitioners outside of SBHCs and behavioral health providers, which may help to explain these results. There was limited variation of responses for school characterization across the four time- points. Majority of schools were characterized as traditional public schools and/or title I. There was a significant association between school characterization and type of MH services. Magnet schools reported more screening/assessment services than other school characterizations. Public schools were significantly less likely to offer medication management and substance use treatment services than other school characterizations. Charter schools were significantly less likely to offer substance use treatment services than other school characterizations. Vocational schools were significantly more likely to offer substance use treatment services than other school characterizations. Title I and magnet schools were significantly more likely to offer referral services than other school characterizations. It is important to note that interpreting these findings is difficult because schools could identify themselves with more than one school characterization, and school characterizations varied across the four time points for some schools 65 (i.e., a school may identify itself as a Title I school at time point 1 but identify itself as a public and Title I school at later time points). It remains unclear how school characterization may impact delivery and provision of services. It was beyond the scope of the current thesis project to explore clusters of school characterization. Further research would likely be needed to better understand the specifics of school type on provision of MH services offered through SBHCs over time. The National Census Survey - Networks Communications subscale. Networks and partnerships facilitate integration of services with increased sources for support, resources and funding (Armbruster, 2002; Dreyfoos, 1994; Swider & Valukas, 2004). Prior studies have observed that SBHCs with a mental health component are more likely to be sponsored by school and university departments than public health departments (Larson et al., 2017). In the current study, community health centers and hospital/medical centers were most frequently reported sponsors from 2005 to 2010. Hospital/medical centers and school systems were most frequently reported sponsors in 2014. LMM results expanded on variations within these partnerships and demonstrated that sites with mental health agency sponsorships reported more screening/assessment and referral services than any other agency. Sites with hospital/medical center sponsorships reported more medication management services, following with university sponsorships. Sites with private/non-profit agencies reported more substance use treatment services than any other type of agency, following with university sponsorships. These findings suggest that collaboration with other service systems plays an important role in promoting delivery of mental health services within SBHCs over time. Given that these agencies are the most common sponsors reported for SBHCs with a mental health component, further research is 66 needed to understand the underlying differences within these partnerships that produce different outcomes in types of services offered. RQ3. Which outer setting variables (e.g., funding sources, policy and standards, oversight and support) are related to number of mental health services reported to be delivered from 2005 to 2014? The third research question aimed to explore the relationship between outer setting variables and mental health services reported to be delivered from 2005 to 2014. To our knowledge, prior studies have not assessed the relationship between outer context variables as collected within the Policy survey and delivery of MH services over time. Comparisons between SBHCs with different outer contextual factors – particularly, funding and state policy support – are not well examined. Overall, most outer context variables, with the exception of state general funds on substance use treatment and referral services, were significantly associated with the number of reported MH services over time. The State Policy Survey - Funding Sources subscale. Implementation studies assert that the presence of dedicated and ongoing funding increases the likelihood of adopting, implementing and sustaining an innovation (Greenhalgh et al., 2008). Building on this concept, the current study assessed the relationship between state agencies reported as funding sources, having a SBHC grant program, SBHC funding sources and the number of MH services over time. Findings showed that states with a grant program dedicated specifically to SBHCs had fewer MH services being delivered than states that did not report a program unit. This could relate to the fact that many SBHCs have diverse sources for funding that may not be directly attributed to a state program unit. Moreover, it might be the case that states not reporting a program may not actually have access to a designated state program unit, making access to other 67 sources a priority for resources. Funding sources reported from tobacco settlement were associated with more screening/assessment, medication management, substance use treatment, and referral services than sources from Title V MCH or state general funds. Interestingly, state general funds were not significantly associated with the number of referral or substance use treatment services at any time point. Moreover, states that reported both public health agencies and human agencies as funding sources demonstrated more medication management and substance use treatment than states that reported other funding agencies. However, more referral services were found only among states that reported sources from human agencies alone. These findings help explain the variation of services accounted for by funding sources reported by SBHC policy-makers and may be useful to target for more funding opportunities. The State Policy Survey - Oversight and Support subscale. “Technical capacity” refers to an organization’s technical resources and technical potential (Greenhalgh et al., 2008). Prior studies have found technical capacity positively and significantly correlates to an organization’s willingness to utilize an innovation (Greenhalgh et al., 2008). Furthering this line of research, the current study assessed whether having a state program unit providing technical assistance for developing SBHCs or for data collection purposes was significantly related to the number of MH services reported over time. Results showed that states with a program unit to provide technical assistance reported more screening/assessment, medication management and substance use treatment services than those that did not have technical assistance over time. However, this relationship was not found for referral services. These findings suggest that having some type of technical assistance offered by state programs plays a role in facilitating the delivery of more treatment services. Since this was not observed for referrals, it might be the case that referrals are part of a coordinated care process (e.g., with a school psychologist or 68 school nurse) outside of the center. More research is needed to understand which types of technical assistance work best in promoting services within SBHC sites. Moreover, states with a program unit to collect data reported more screening/assessment and medication management services—but lower number of substance use treatment or referral services. It is possible that there are outside services, such as prevention activities or coordinate care services, that are influencing the direction of these relationships. More research should consider the overlap of services offered through multiple staff located within school settings, such as school psychologist, school nurse, and SBHC staff. The State Policy Survey - Policy and Standards subscale. External policy and incentives refer to changes influenced by external forces, such as mandates, regulations, and guidelines (Damschroder et al., 2009). These external forces can often shape the delivery of services and foster access to health care (Doll et al., 2017; Sprigg et al., 2017). Results from the current study demonstrated that operating standards, as defined by state, were significantly associated with an increase in substance use treatment and referral services over time—but not screening/assessment or medication management services. Moreover, the criteria for these operating standards varied. For example, more referrals were reported by states with operating standards for all SBHCs regardless of funding source, but more substance use treatment services were reported by states with operating standards only for state-funded SBHCs. The role of variations within these standards remains unclear. State requirements that are mandatory for SBHCs funded by the state demonstrated more screening/assessment, medication management, substance use treatment, and referral services over time. These findings suggest that state defined regulations can help promote services by developing frameworks to meet service quality standards. 69 Limitations There are several limitations to the current project that relates to the nature of secondary data. First, the project is limited to data that has already been collected by a third-party organization. Assessments and measures are then limited in scope, with some circumstances compromising the quality of the data collected. Moreover, missing data cases may have influenced the outcomes. For example, administrative errors in data collection may have yielded the observed missing data cases. Second, multicollinearity of the variables was not controlled for in the main analyses. These relationships, however, were noted in unadjusted models (bivariate analyses) and considered throughout the analyses and interpretation of results. Third, results from this study are not able to identify or explain causal relationships between inner and outer contexts with the provision of mental health services delivered through SBHCs. Results can only suggest the direction of relationships between inner contexts, outer contexts, and mental health services in SBHCs over time. In an attempt to minimize all these limitations, the study attempted to remain as transparent in procedures as possible for reproducibility with the secondary data set. Conclusion There remains a need to identify key factors in state level support that contribute to the delivery of mental health (MH) services in SBHCs over time and to understand variation of MH services across states. Results from the current study support significant variations in SBHC MH services among the 41 eligible U.S. states over the four time points. Findings from the current study suggest that there are significant relationships between structural characteristics, networks, and state level support and the delivery of MH services. Given the variation in the frequency of reported screening/assessment, medication management, substance use and referral services, it is 70 recommended that future research explore how these contextual variables influence the types of services made available to students. Importantly, this project has several important and innovative contributions to scientific knowledge and policy. First, the project is innovative in its theoretical approach. Using a contextualist perspective can address several limitations found in prior studies by accounting for the role of unique contexts for each state. Utilizing the contextualist approach allowed us to explore patterns in the structural characteristics and networks of SBHCs that had not been included in previous literature. The study results clearly emphasize that specific inner setting variables were significantly and positively related to more MH services being reported from 2005 to 2014 but that the relations had differential impacts on which service was provided. Such patterns can be crucial to understand likelihood of specific mental health services within a given setting. Moreover, this project was innovative because, to our knowledge, it is the first study to include the State Policy Survey and evaluate associations between funding and policies at the state level with the number of mental health services delivered by SBHCs. Further, this study provides novel information by examining how these associations change over time. This information may be particularly useful for promoting facilitating factors that increase the use of specific types of mental health services to increase access to quality mental health treatment services, reducing the oft noted mental health disparities, and improve the lives of youth in the United States. 71 Figure 1: Stacked Bar Count (“Yes”) of School Characterization by Time 72 APPENDICES 73 APPENDIX A: National SBHC Census Survey SCHOOL-BASED HEALTH CENTER CENSUS SCHOOL YEAR 2013-2014 Welcome to the School-Based Health Alliance Census, School Year 2013-2014! Who should participate in the 2013-2014 School-Based Health Care Census? • Partnerships between schools and community health organizations that deliver health care to students within a fixed site on school campus [SCHOOL-BASED] • Health care programs that are formally or informally linked with schools to coordinate and promote health care for students on campus; clinical services are not provided on school site [SCHOOL-LINKED] • Programs without a fixed site that rotate a health care team through a number of schools [MOBILE] • Programs delivering school-based health services exclusively via telehealth technology [TELEHEALTH ONLY SITE]. For programs delivering some services by a provider who is physically onsite and some services via telehealth technology, respond to the census as a school-based, school-linked, or mobile health center and use the appropriate column in Section 3 to indicate which team members are accessible via telehealth. Who should complete the Census? The Census should be completed by the person who is most knowledgeable about the clinical care provided in the health center, such as the nurse practitioner or clinical director. Instructions: Please answer all questions. 1. If you are unable to complete every section, provide as much data as possible and return your incomplete form. We appreciate any amount of information you can provide. 2. If you are completing a paper version of the survey, please use a separate questionnaire for each fixed health center site you represent. For programs on campuses that serve several schools, complete one survey and provide information on all grades served within the immediate campus. Mobile programs may use one survey. 3. All questions refer to the school year 2013-2014, unless otherwise specified. If you are completing three or more censuses, the School-Based Health Alliance staff will offer you hands-on support. Please contact census@sbh4all.org and a member of the staff will schedule an appointment with you to complete the Censuses. You may use this printout to review the questions that are being asked beforehand to ensure collection of correct information. To complete the Census online, please visit: www.sbh4all.org/censussurvey. To complete a paper copy, please print and answer all items. Send a scanned copy or fax completed copy to: Mail: School- Based Health Alliance, 1010 Vermont Avenue NW, Suite 600, Washington, D.C. 20005 Email: census@sbh4all.org Fax: 202-638-5879 Announcement: For the first time ever, we will have a publically accessible map of SBHCs across the country that will include basic health center demographic information and characteristics drawn from the census. 74 1. HEALTH CENTER DEMOGRAPHICS A. The health center I represent is (select one): o o o In a school building On school property, but not in a school building Beyond school property, but has formal or informal links with one or more schools in the community o Mobile program serving several schools but with no fixed site o Telehealth only site (100% of services provided through telehealth technology) - If some services are provided by a provider onsite and some via telehealth technology, please select the location of the health center from the other options above. B. How many schools are served by the health center? (For campuses with more than one school, such as a middle and high school, include all schools in your count.) o One school (i.e., Only the school where health center is located) o Name of school: o More than one school o Names of schools: o o All schools in school district Do not know C. Does your health center serve individuals other than students enrolled in your school (For school-linked programs, answer not applicable)? o o o o Yes No Not applicable Do not know D. Which of the following populations are eligible to use the health center services (select all that apply) (For school-linked programs, describe populations served in “Other”)? Yes No Do not know Students from other schools Out-of-school youth Faculty/school personnel Family of students users (i.e., siblings, parents, or infants of students) Other people in the community Other, please specify: o o o o o o o o o o o o o o o o o o 75 E. Indicate the type of agency that serves as the primary administrator and/or sponsoring health care organization for the health center (select one). o Federally Qualified Health Center (FQHC) or Look-Alike (an organization that receives funding under the Health Center Program as authorized under section 330 of the Public Service Act or a FQHC look-alike organization, which meets all of the Health Center Program requirements but does not receive a Health Center Program grant) o o o o Community health center (non-FQHC) Hospital/medical center Local health department School system o Mental health agency o o o o Private, non-profit organization (not a community health center) University (i.e., school of medicine, nursing, public health) Tribal government Other, please specify: _ F. The geographic location of the community served by the health center is described primarily as (select one): o o o Urban Rural Suburban 2. HEALTH CENTER OPERATIONS A. In what year was the health center first established? B. During the 2013-14 school year, how many days each week was the health center open? C. During the 2013-14 school year, how many hours per week was the health center open? D. Indicate when the health center was open during the 2013-14 school year (select all that apply): Yes No Do not know Before the school day begins After the school day ends During school hours During school vacations/ holiday breaks (i.e., Thanksgiving, winter, spring break) During summer months o o o o o o o o o o o o o o o E. Does the health center have a prearranged source of after-hours care (i.e., on-call services provided by the sponsoring agency, health center staff, or external agency)? o Yes o No 76 3. HEALTH CENTER CARE TEAM A. This table is about your health center’s staffing. Include all staff, even those employed by other agencies. Do not include interns, volunteers, peer educators, etc. Total hours per week: For each staff person, write in the total clinical hours per week that person is physically at the health center or providing the service via telehealth technology. If a person serves many functions, select the position that describes the majority of their work at the health center. If more than one person fills a position, add together all the hours for that position. (For example, if two NPs each work 5 hours per week, write in 10 hours.) Total clinical hours per week Total clinical hours per week via on-site telehealth Nurse practitioner Physician Physician assistant Alcohol and drug counselor Licensed social worker/counselor/therapist Unlicensed social worker/counselor/therapist Psychiatric nurse practitioner Psychiatrist Psychologist Administrative assistant or receptionist Medical assistant or health aide Care coordinator Case manager/social services Dental assistant Dental hygienist Dentist Health educator Licensed practical or vocational nurse Nutritionist Ophthalmic tech Optometrist or ophthalmologist Outreach coordinator Registered dietician Registered nurse Other health center staff, please specify. Please add any relevant comments concerning the health center staffing: 77 B. The following staff are (select one): School nurse School behavioral health provider Not in the school In the school but separate from the health center In school and co- located with the health center Do not know o o o o o o o o 4. HEALTH CENTER PARTNERSHIPS A. Does the health center have a memorandum of understanding (MOU) that includes any of the following components with the school and/or school district (select all that apply)? Formal outline of partner roles and responsibilities Expectations for services provided by each agency (i.e., in-kind space, janitorial support, etc.) Data sharing protocols as it relates to HIPAA and FERPA Other, please specify: Yes No Do not know o o o o o o o o o o o o B. In what school teams/committees does your health center participate (select all that apply)? Curriculum development committee Crisis management or early intervention team Individuals with Disabilities Education Act (IDEA) team School improvement team School wellness committee (i.e., coordinated school health or other school wellness committee) School district wellness committee Other, please specify: 5. PRIMARY CARE Yes No Do not know o o o o o o o o o o o o o o o o o o o o o A. Do you provide primary care onsite, which includes comprehensive health assessments, diagnosis, and treatment of minor, acute, and chronic medical conditions, and referrals to, and follow-up for, specialty care? o o o Yes No Do not know 78 6. VISION SERVICES A. Does the health center offer vision services including screening, examination, and/or dispensing of eye glasses (select all that apply)? Screening Examination Dispensing of eye glasses Provide onsite Refer for services not provided at the health center Not provided or Do not know referred o o o o o o o o o o o o 7. CHILD AND ADOLESCENT IMMUNIZATIONS A. Do you provide any of the following immunizations (individually or in combination) to children or adolescents (select all that apply)? Provide onsite Refer for services not provided at the health center Not provided or referred Do not know Diphtheria/Tetanus/Acelluar Pertussis (DtaP, Tdap, or Td)) Haemophilus influenza tybe b (Hib) Hepatitis A Hepatitis B Inactivated Poliovirus (IPV) Influenza Human Papilloma Virus (HPV): male Human Papilloma Virus (HPV): female Measles-Mumps-Rubella (MMR) Meningococcal (MCV4) Pneumococcal (PCV, PPV) Rotavirus (Rota) Varicella (Varivax) Other, please specify: o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o 79 8. SEXUAL AND REPRODUCTIVE HEALTH A. Indicate which of the following sexual and reproductive health services are provided by the health center (select all that apply). Provide onsite Refer for services not provided at the health center Not provided or Do not know referred Abstinence counseling Chlamydia testing and treatment STD diagnosis and treatment Counseling for contraceptive services Contraceptive (prescriptive) services Gynecological examinations HIV testing and counseling Papanicolaou (Pap) test Pregnancy testing Prenatal care Relationship violence (counseling/intervention) Sexual orientation education and counseling Testicular examinations Other, please specify: o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o B. Indicate which of the following contraceptive/barrier methods are provided to individual students (select all that apply). Provide onsite Prescribe onsite Refer for services not provided at the health center Not provided or referred Do not know Barrier methods (i.e., male or female condoms, diaphragm) Hormonal methods (i.e., birth control pills, depo-provera, patch (OrthoEvra), ring (NuvaRing)) Implantable devices (i.e., implant (Nexplanon), intrauterine device (IUD)) Emergency contraception Other: 80 C. Is the dispensing of prescribed contraceptives prohibited in the health center? o o o Yes No Do not know D. By whom is this prohibition made (select all that apply)? State law/regulation State policy School district policy School policy Sponsor policy Health center policy Other, please specify: 9. BEHAVIORAL HEALTH Yes No Do not know A. Does the health center provide screening/assessment, referral, or treatment of any of the following behavioral health issues (select all that apply). Screening/ Assessment Treatment Refer for services not provided at the health center Not provided or referred Do not know Academic functioning Anxiety/nervousness/phobias Attention/concentration/ADD/ ADHD Depression/sadness Eating disorders Grief/loss/bereavement Identity issues Oppositional/defiant behavior/anger management Social skills/relationship issues/conflict (family, peers, partners) Strengths/resiliency factors Substance use (alcohol, tobacco and/or drugs) Suicidal ideation/attempt Trauma/abuse/exposure to violence/PTSD Other, please specify: o o o o o o o o o o o o o o o o o o o o o o o o o o o o 81 o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o B. Does the health center prescribe and/or manage behavioral health medications (select all that apply)? Prescribe Manage Co-manage 10. ORAL HEALTH Yes o o o No Do not know o o o o o o A. Indicate which of the following oral health services are provided to individuals by the health center (NOTE: Unless indicated, check off the services provided at your health center regardless of who provides the service) (select all that apply). Provide service onsite Provide service through mobile unit Refer for services not provided at the health center Not provided or referred Do not know Comprehensive dental examination and diagnosis by a dentist Comprehensive dental examination and diagnosis by a dental hygienist or therapist Dental screenings (i.e., visual inspection and assessment) Dental sealants Fluoride mouthrinse Fluoride varnish Fluoride supplements Fluoride gel/foam Dental cleaning General dental care (i.e., fillings, extractions) Oral health education Specialty care (i.e., orthodontics, oral surgery) Other, please specify: o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o 82 B. Which of the following oral health services do primary medical care professionals provide at the health center (select all that apply)? Dental screenings Education Guidance/referral Prescribe fluoride supplements Preventive services: fluoride treatments Preventive services: sealants Risk assessment Other, please specify: Yes No Do not know o o o o o o o o o o o o o o o o o o o o o o o o 83 Substance Use Prevention Tobacco, alcohol, drug use, and/or highly caffeinated beverages prevention) Injury and Violence Prevention General violence prevention (fighting, guns, gangs) Dating and intimate partner violence prevention Suicide prevention Sexual and Reproductive Health and Behaviors HIV/STD prevention Pregnancy prevention - abstinence only Pregnancy prevention - comprehensive Sexuality education (forming attitudes, values, and beliefs that support the sexual health of youth) Sexual orientation/gender identity differences (LGBTQ respect) Teen parenting classes Healthy Living Healthy eating/active living/weight management Chronic disease management (asthma, diabetes) Oral health education and promotion Interpersonal relationships (race relations, conflict resolution, healthy dating) Emotional health and well-being (social/emotional learning, stress management, hopefulness) School safety/climate Positive youth development (skills building, youth engagement, multiculturalism) 11. HEALTH PROMOTION AND DISEASE PREVENTION A. Indicate which of the following prevention/education activities are provided by the health center staff and to whom (select all that apply). Individually with a child/adolescent (1 on 1) With groups of children/adolescents in the clinic or classroom Not offered Do not know With parents or community members 84 Successful Learning Dropout prevention, school/academic performance intervention, and/or school attendance Other, please specify 12. HEALTH INFORMATION TECHNOLOGY A. Is the health center able to access students’ individual educational data (i.e., attendance records, discipline action, grades)? o o o Yes No Do not know B. Indicate whether the health center uses any of the following (select all that apply): Yes No Do not know Electronic health/medical record (EHR/EMR) Management information system (MIS)/Practice management system Electronic billing system Electronic prescribing o o o o o o o o o o o o C. Is there a common EHR/EMR used by primary care providers and behavioral health providers? o Yes o No o Do not know D. Has having an EHR/EMR allowed you to achieve any of the following stages of “meaningful use” as defined by the Centers for Medicare and Medicaid (CMS) (select all that apply): o o o o Stage one Stage two Health center does not have an EHR/EMR Do not know 85 13. BILLING AND REIMBURSEMENT FOR STUDENTS A. Does your health center bill any of the following entities for services provided to students (select all that apply)? Medicaid: State agency Medicaid: Managed Care Organization (MCO) Children’s Health Insurance Program (CHIP) Private/commercial insurance Tri-Care (military insurance program) State family planning programs State programs for the medically indigent Patients or families (self-pay) Other, please specify: Yes No Do not know o o o o o o o o o o o o o o o o o o o o o o o o o o o B. What types of health insurance payment does the health center receive (select all that apply)? Fee for service Monthly or annual capitated payments for primary care Monthly or annual capitated payments for care coordination Supplemental payments for meeting performance standards Other, please specify: Yes No Do not know o o o o o o o o o o o o o o o C. Who administers billing and collection for the health center (select all that apply)? SBHC Staff Medical sponsor staff Third-party billing service Other, please specify: Yes No Do not know D. Estimate the percent of your total operational expenses that are covered by billing revenue. % E. How does your health center assist in enrolling children/families in health insurance coverage (select all that apply)? o o o Enrollment completed onsite at health center Assistance completing forms provided by health center Referred to enrollment site outside of health center o No assistance 86 F. Which of the following describes the SBHC’s relationship with any managed care organization (select all that apply)? SBHC serves as a PCP/preferred provider/ medical home SBHC serves as a specialty care provider (i.e., behavioral health, family planning) SBHC is not recognized as preferred provider, but is reimbursed for some services Other, please specify: 14. NON-BILLING SOURCES OF REVENUE/FUNDING Yes No Do not know o o o o o o o o o o o o A. What are the sources of revenue/funding that support the health center (do not include in-kind donations or billing revenue) (select all that apply)? Yes No Do not know Federal government State government County/city government Tribal government Private foundations Corporations/businesses Hospital School/school district Managed care organization or private insurer (a grant or donation, not patient revenue related) State network/association School-Based Health Alliance (formerly the National Assembly on School-Based Health Care) Other, please specify: o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o 87 B. If you receive support from the federal government, indicate the funding sources for your health center during the 2013-14 school year (select all that apply): Federal Funding Sources: Yes No Do not know CDC Community Transformation grants CDC RFA-1308: Promoting Adolescent Health Through School-Based HIV/STD Prevention and School-Based Surveillance Indian Health Services Nurse-Managed Health Clinics T56 (Affordable Care Act) Substance Abuse and Mental Health Services Administration’s (SAMSHA) Centers for Substance Abuse Treatment and Prevention’s Safe School/Health Communities School-Based Health Center Capital Program (HRSA) Section 330 PHSA (community/migrant/rural health centers) TANF (Temporary Assistance to Needy Families) Teen Pregnancy Prevention programs Title I ESEA (Elementary Secondary Education Act) Title V SSA (Social Security Act-maternal and child health block grant) Title X PHSA (Public Health Service Act-family planning) Title XX SSA (social services block grant) Other, please specify: 15. HEALTH CENTER ACCESS POLICIES o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o A. Do parents have the ability to restrict access to specific services? (For example, parents can cross off specific services on the consent form.) o o o Yes No Do not know B. Most states allow minors to access sensitive services (i.e., reproductive health, behavioral health, and substance use prevention) without parental consent. Indicate which of the following best describes student access to sensitive services in your health center (select one). o In accordance with state law o More restrictive than state law o Do not know 88 16. HEALTHCARE QUALITY A. Indicate which of the following components of a quality assurance system are used by the health center (select all that apply): Yes No Do not know Chart audits Staff credential and training requirements Policies and procedures Standards for the physical environment Measures of patient knowledge Clinical Laboratory Improvement Act (CLIA) certification Measures of patient satisfaction Data reports from electronic medical record Review of claims data Other, please specify: o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o B. Does the health center collect quality outcomes data based on (select all that apply): Healthcare Effectiveness Data and Information Set (HEDIS) measures Yes No Do not know Recommended core set of child health quality measures (CHIPRA/Medicaid) State-defined tool/measures Sponsor-specific tool/measures SBHC-developed tool/measures School-Based Health Alliance CQI for SBHC Tool School-Based Health Alliance Mental Health Program Evaluation Template Other, please specify: o o o o o o o o o o o o o o o o o o o o o o o o C. Does the health center use any of the following risk assessment screening tools (select all that apply)? o o o o o o o Bright Futures The CRAFFT screening tool Guidelines for Adolescent Preventive Services (GAPS) H.E.A.D.S.S. Patient Health Questionnaire (PHQ7, PHQ9, or PHQ15) Rapid Assessment for Adolescent Preventive Services (RAAPS) Screen for Child Anxiety Related Disorders (SCARED) o Other, please specify: 89 D. Does the health center collect any data for quality improvement (i.e., % clients with BMI assessment; % clients with complete immunizations)? o Yes o No o Do not know E. Has your health center been accredited, directly or through your sponsoring agency, by any of the following (select all that apply): Yes No Plan to pursue in the next 12 months Do not know Joint Commission National Committee on Quality Assurance (NCQA) State certification (indicate type ) Other, please specify: o o o o o o o o o o o o o o o o F. Has your health center been recognized as a patient-centered medical home by any of the following (select all that apply)? Joint Commission National Committee on Quality Assurance (NCQA) State-specific program (indicate type ) Other patient-centered medical home recognition: Yes No Plan to pursue in the next 12 months Do not know o o o o o o o o o o o o o o o o 90 17. SCHOOL/CAMPUS DEMOGRAPHICS A. Indicate the grade levels served by the health center (select all that apply). (For campuses with more than one school served by the health center, include every grade. For school-linked programs, estimate grade levels served and describe in “Other”.) If you are uncertain about this answer, you can find the information at http://nces.ed.gov/globallocator/. o o o o o o o o o o o o o o o Pre-K K 1 2 3 4 5 6 7 8 9 10 11 12 Other, please specify: B. What is the 2013-2014 academic year official school enrollment for the school/campus in which the health center is located? If your health center serves more than one school, list the official total enrollment of all the schools served. For school -linked programs, answer not applicable. (If you are uncertain about this answer, you can find the information at http://nces.ed.gov/globallocator/): C. What is the total student health center enrollment for 2013-2014 academic year (students with consent to use or registered to use the health center, although they do not have to be seen)? For school-linked programs, answer not applicable. D. In the 2013-2014 academic year, the ethnic/racial profile of the student population at the school/campus in which the health center is located was: (If you are uncertain about this answer, you can find the information at http://nces.ed.gov/globallocator) (For school-linked programs, provide an estimate based on patients served.): o o o o o Hispanic or Latino of any race % American Indian or Alaskan Native % Asian % Black or African-American % Native Hawaiian or other Pacific Islander % o White o Two or more races % % TOTAL =100% E. In the 2013-2014 academic year, what percent of the student population at the school/campus in which the health center is located was eligible for the free- or reduced-price lunch program? (If you are uncertain about this answer, you can find the information at http://nces.ed.gov/globallocator/) (For school-linked programs, provide an estimate based on patients served.) % 91 F. Can the school in which you are located be characterized as any of the following? (If there is more than one school in the building in which you are located and you provide services to those students include that type of school in your selection.) (For school-linked programs, answer not applicable.) Select all that apply. Yes No Not applicable Do not know Title I School (receives funding from US Dept. of Education to meet needs of at-risk and low-income students. If you are uncertain about this answer, you can find the information at http://nces.ed.gov/globallocator/) Charter School (public school operated independently of the local school board, often with a curriculum and philosophy different from the rest of the district) Parochial/Private School (funded by a religious organization, individuals, or corporation) Alternative School (offers nontraditional educational ideals, methods of teaching, or curriculum) Vocational School (often on the secondary level and offers instruction and practical experience in skilled trades) Magnet School (public school with specialized curriculum and student body representing a cross section of the community) Traditional Public School (supported by public funds and providing free education for children of a community or district) Community School (school with a school site leadership team including school staff, families, community members, and partner organizations with a designated coordinator responsible for coordinating partnerships focused on results, i.e., Beacon school, full-service school) Other, please specify: 18. YOUTH AND PARENT/GUARDIAN INVOLVEMENT o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o A. Other than as patients, are students and parents/guardians involved in your health center in any of the following ways (select all that apply)? Students Parents/Guardians No involvement of students or parents/guardians Do not know Participate in organizing center-sponsored health education events (i.e., health fair) Participate in peer mentoring, counseling, or education Advocacy activities (local, state, or national) Participate in health center advisory council, committee, or board Participate in the design of health services Promote health services provided by health center Provide feedback to the health center o o o o o o o o o o o o o o o o o o o o o o o o o o o o 92 Other, please specify: o o o o B. Which of the following methods do you use to communicate with students and parents/guardians (select all that apply)? Students Parents/Guardians Do not use this method with students or parents/guardians Do not know Email Social media (i.e., Twitter, Facebook) SBHC website or school website Blog Phone Text message Written material (i.e., brochure, newsletter) Other, please specify: o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o THANK YOU FOR COMPLETING THE 2013-14 CENSUS! 93 APPENDIX B: State Policy Survey FY 2014 School-Based Health Care (SBHC) Policy Assessment - State School-Based Health Center (SBHC) Program Office Survey 1. 2. 3. 4. 5. The state school-based health care (SBHC) policy assessment should be completed by the person(s) most knowledgeable about state-level policies, funding, and program support related to SBHCs. For question #1, please provide the name and contact information for the primary respondent. Please provide the contact information for secondary respondents at the end of this survey. It should take approximately 30 minutes to complete this survey. Please review instructions on how to “save and continue” at the top of the online survey page. If you are unable to complete every question, please forward the word document version of this survey (emailed to you) to the most appropriate person. 6. All questions refer to fiscal year 2014. 7. Deadline: November 7th, 2014 Definitions. For the purpose of this survey, school-based health centers (SBHCs) are located in or near a school; are organized through school, community, and health provider relationships; are administered by a sponsoring facility such as a hospital, community health center, university, or public health department; and provide primary health care services to children in accordance with State and local law, including laws relating to licensure and certification. The term school-based health center does not refer to school health services identified in Individualized Education Plans (IEP) for special education students and delivered by Local Education Agencies (LEAs). About the Respondent 1. Respondent #1: Title: Org: Street: _______________________________________ City/State/Zip: Phone number: Email: What is the total number of SBHCs in your state? ___________ SBHCs in Your State 2. 3. Does your State define SBHCs in law or regulation? Do not know Yes No Other, please specify: __________________________________________ If yes, please provide a citation to this definition: ____________________________________________________________ 4. SBHC Funding 5. Does the state fund or sponsor a grant program dedicated specifically to SBHCs? a) Yes No Please indicate the source and amount of funds granted to SBHCs in fiscal year 2014. Grant Program Funding Sources Title V MCH State General Fund Amount $ $ 94 Tobacco Settlement Preventative Services Block Grants Other (please specify): Total $ $ $ $ b) What is the total number of SBHCs that were funded by the state-directed grants in fiscal year 2014? 6. Which state agency is responsible for administration of grant program or funds to state’s SBHC program? Check all that apply. 7. Public Health Mental Health Education Human Services Do not know Other, please specify: _______________________________ Is your state implementing patient-centered medical homes (PCMH) for their school-aged Medicaid populations? Yes No SBHC Oversight and Support). (skip to question 10 – SBHC Oversight and Support) Do not know (skip to question 10 – 8. 9. If yes, are SBHCs included and/or participating in the PCMH program? Yes, all SBHCs Yes, some SBHCs No Do not know Do SBHCs qualify for any of the following enhanced Medicaid payments offered to participating PCMH providers? Check all that apply. Per member per month (PMPM) care coordination payment Higher primary care payment levels Pay for performance payments or bonuses Other, please specify:__________ Do not know Not applicable SBHC Oversight and Support 10. Is there a state government program office or unit that is responsible for providing oversight and support to state-funded SBHCs? Yes No (skip to question 26 – Future Outlook for SBHCs) Do not know (skip to question 13) 11. How many Full Time Equivalents (FTEs) staff this state program office? 12. FTEs What are the top 3 most requested topics of technical assistance that this state program office provides? Please check 3 options only. Planning/implementing an SBHC Training for SBHC staff Program evaluation Financial sustainability Quality assurance Clinical guidelines Coding and billing Assistance with certification, licensing and accreditation None Do not know Other, please specify: Does the state government program collect data from SBHCs? Yes No (skip to question 18) Do not know (skip to question 18) If yes, what types of data are collected? Check all that apply. Operations data (i.e., staffing, hours, policies) Client/visit data Risk assessment data Quality improvement data Finance data Do not know Other, please specify: Describe state requirements for SBHC data collection. Check all that apply. Mandatory for SBHCs funded by state Mandatory for SBHCs not funded by state Voluntary for SBHCs funded by state 95 13. 14. 15. 16. Voluntary for SBHCs not funded by state Do not know Other, please specify: Which of the following performance indicators does the state government program use to assess SBHC performance/productivity? Check all that apply. Access measures (waiting times, etc.) Annual risk assessments Asthma care BMI assessment Chlamydia screening Patient satisfaction Depression screening Diabetes care HIV screening Immunization status Oral health assessment Physical exams/well child visits Pregnancy testing Provider productivity SBHC enrollment as percent of student body Poor school performance Substance use (alcohol and other drugs) screening Tobacco use screening Users as percent of SBHC enrollees No performance indicators are tracked Other, please specify: ______________________ Yes No Do not know Does the state government program review SBHC Medicaid or CHIP claims data as part of assessing SBHC performance? Yes, as a requirement for state grants Yes, as a requirement for Medicaid billing State does not certify or credential SBHCs Do not know 17. SBHC Policies and Standards 18. Does the state certify or credential SBHCs? Check all that apply. 19. Does the state require SBHCs to adhere to state-defined operating standards? Examples of state standards can be found on the School-Based Health Alliance website. 20. How does the state monitor compliance with SBHC standards? Check all that apply. 21. What is the state’s response to SBHCs found to be out of compliance with state standards? Check all that apply. 22. Which of the following represent barriers for SBHCs to improving patient revenue collection? Check all that apply. State funds are rescinded or revoked SBHC issued warning or placed on probationary period SBHC standards are not enforced Do not know Other, please specify: ____________________________________________ Yes, for state funded SBHCs only Yes, for all SBHCs regardless of funding source No (skip to question 22) Do not know (skip to question 22) Site review by state government representative Paper survey/report completed by site State does not monitor its SBHC standards Do not know Other, please specify: SBHCs lack administrative infrastructure to bill Medicaid SBHCs cannot cover costs associated with Medicaid billing SBHCs unable to manage complexity of Medicaid MCO contracts SBHCs lack support for SBHC billing and collection from sponsor organization SBHCs lack knowledge of proper Medicaid billing practices Medicaid reimbursement does not provide enough financial incentive for SBHC to bill SBHCs have inadequate information technology 96 There are no SBHC practice-level barriers to Medicaid reimbursement in my state Other, please specify in comments: _________________________ 23. Regarding Medicaid Managed Care organizations (MCOs) within your state, please indicate if any of the following represent barriers to SBHC Medicaid reimbursement. Check all that apply. There are no Medicaid MCOs in our state MCOs lack of knowledge of SBHC value Contracting with multiple MCOs is burdensome to SBHCs SBHCs inability to track MCO quality measures Limited capacity of SBHCs to demonstrate its value to MCOs SBHCs inability to meet MCO facility and/or provider requirements Limited number of MCO enrollees served by SBHCs SBHCs inability to exchange patient encounter information with MCOs There are no MCO practice-level barriers to SBHC Medicaid reimbursement Other, please specify: _________________________ 24. Does the state prohibit contraceptives from being dispensed in SBHCs? Yes No (skip to 26) Do not know (skip to 26) 25. If yes, what is the source of the prohibition? Check all that apply. State law State regulation Other, please specify: ___________________ Future Outlook for SBHCs 26. In the next three years, do you expect that state-level financial support for SBHCs will increase, decrease, or stay the same? 27. Of the issues below, which do you consider to be the top three concerns with regards to growing and sustaining SBHCs in your state? Please check 3 options only. Stay the same Do not know Increase Decrease Creating a sustainable financial model for SBHCs Ensuring continued support of SBHCs through public sector funding Establishing policies and mechanisms to maximize patient revenue streams Demonstrating the value and efficacy of SBHCs to health insurance and health care plans. Maintaining strong partnerships with schools and local health providers Effectively addressing complex health and behavioral health issues (mental health, substance abuse, school performance) of children and youth I am not concerned about growing or sustaining SBHCs in my state _____ Other, please specify:____________ 28. Please rate the degree to which the following strategies affect SBHC funding: Not effective Some-what effective Effective Very effective Do not know Grassroots advocacy by providers, community leaders, consumers Support from state health agency leadership/ staff Support from state education agency leadership/ staff Political support from executive branch Political support from legislative branch Strong data or evidence of SBHC’s effectiveness 97 REFERENCES 98 REFERENCES Adams, E. K., & Johnson, V. (2000). An elementary school-based health clinic: Can it reduce medicaid costs? Pediatrics, 105(4), 780-8. Retrieved from http://ezproxy.msu.edu.proxy2.cl.msu.edu/login?url=https://search-proquest- com.proxy2.cl.msu.edu/docview/228396063?accountid=12598 Agudelo-Suarez, A. A., Gil-Gonzalez, D., Vives-Cases, C., Love, J. G., Wimpenny, P., & Ronda-Perez, E. (2012). A metasynthesis of qualitative studies regarding opinions and perceptions about barriers and determinants of health services' accessibility in economic migrants. BMC Health Serv Res, 12, 461. doi:10.1186/1472-6963-12-461. Allison, M. A., Crane, L. A., Beaty, B. L., Davidson, A. J., Melinkovich, P., & Kempe, A. (2007). School-based health centers: Improving access and quality of care for low- income adolescents. Pediatrics, 120(4), e887-e894. doi: http://dx.doi.org/10.1542/peds.2006-2314 Anderson, J. K., Howarth, E., Vainre, M., Jones, P. B., & Humphrey, A. (2017). A scoping literature review of service-level barriers for access and engagement with mental health services for children and young people. Children and Youth Services Review, 77, 164- 176. doi:http://dx.doi.org.proxy1.cl.msu.edu/10.1016/j.childyouth.2017.04.017 Anyon, Y., Moore, M., Horevitz, E., Whitaker, K., Stone, S., & Shields, J. P. (2013). Health risks, race, and adolescents’ use of school-based health centers: Policy and service recommendations. Journal of Behavioral Health Services and Research, 40(4), 457–468. https://doi.org/10.1007/s11414-013-9356-9 Armbruster, P. (2002). The administration of school-based mental health services. Child & Adolescent Psychiatric Clinics of North America, 11(1), 23-41. doi: http://dx.doi.org/10.1016/S1056-4993(03)00059-2 Armbruster, P., Andrews, E., Couenhoven, J., & Blau, G. (1999). Collision or collaboration? School-based health services meet managed care. Clinical Psychology Review, 19(2), 221-237. doi: http://dx.doi.org/10.1016/S0272-7358(98)00074-9 Armbruster, P., & Lichtman, J. (1999). Are school based mental health services effective? Evidence from 36 inner city schools. Community Mental Health Journal, 35(6), 493-504. doi: http://dx.doi.org/10.1023/A:1018755100381 99 Bains, R. M., Cusson, R., White-Frese, J., & Walsh, S. (2017). Utilization of mental health services in school-based health centers. The Journal of School Health, 87(8), 584-592. doi:http://dx.doi.org.proxy2.cl.msu.edu/10.1111/josh.12528 Bains, R. M., & Diallo, A. F. (2016). Mental health services in school-based health centers: Systematic review. The Journal of School Nursing, 32(1), 8-19. doi: http://dx.doi.org/10.1177/1059840515590607 Baquiran, R. S., Webber, M. P., & Appel, D. K. (2002). Comparing frequent and average users of elementary school-based health centers in the Bronx, New York City. The Journal of School Health, 72(4), 133-137. Bauer, M. S., Damschroder, L., Hagedorn, H., Smith, J., & Kilbourne, A. M. (2015). An introduction to implementation science for the non-specialist. BMC Psychology, 3 Retrieved from http://ezproxy.msu.edu.proxy1.cl.msu.edu/login?url=https://search- proquest-com.proxy1.cl.msu.edu/docview/1772214189?accountid=12598 Bersamin, M., Garbers, S., Gold, M. A., Heitel, J., Martin, K., Fisher, D. A., & Santelli, J. (2016). Measuring success: Evaluation designs and approaches to assessing the impact of school-based health centers. Journal of Adolescent Health, 58(1), 3-10. doi: http://dx.doi.org/10.1016/j.jadohealth.2015.09.018 Brindis, C., Kapphahn, C., McCarter, V., & Wolfe, A. L. (1995). The impact of health insurance status on adolescents' utilization of school-based clinic services: Implications for health care reform. Journal of Adolescent Health, 16(1), 18-25. doi: http://dx.doi.org/10.1016/1054-139X(95)94069-K Brindis, C. D., Klein, J., Schlitt, J., Santelli, J., Juszczak, L., & Nystrom, R. J. (2003). School- based health centers: Accessibility and accountability. Journal of Adolescent Health, 32(Suppl6), 98-107. doi: http://dx.doi.org/10.1016/S1054-139X(03)00069-7 Brown, M. B., & Bolen, L. M. (2003). School-based health centers: Strategies for meeting the physical and mental health needs of children and families. Psychology in the Schools, 40(3), 279-287. Bloom, B., Jones L.I., Freeman G. (2013). Summary health statistics for U.S. children: National Health Interview Survey, 2012. Vital Health Stat, 10(258), 1-81. 100 Damschroeder, L.J., Aron, D.C., Keither, R.E., Kirsh, S.R. Alexander, J.A., & Lowery, J.C. (2009). Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Doll, B., Nastasi, B. K., Cornell, L., & Song, S. Y. (2017). School-based mental health services: Definitions and models of effective practice. Journal of Applied School Psychology, 33(3), 179-194. doi:http://dx.doi.org.proxy2.cl.msu.edu/10.1080/15377903.2017.1317143 Dowden, S. L., Calvert, R. D., Davis, L., & Gullotta, T. P. (1997). Improving Access to Health Care: School-Based Health Centers. In R.P. Weissberg & T.P. Gullotta (Eds.), Healthy children 2010: Establishing preventive services (pp. 154-182). Thousand Oaks, CA: Sage. Dryfoos, J. G. (1994). Full service schools: A revolution in health and social services for children, youth, and families. San Francisco: Jossey-Bass. Dryfoos, J. G. (1998). School-Based Health Centers in the Context of Education Reform. Journal of School Health, 68(10), 404-408. doi: 10.1111/j.1746-1561.1998.tb06317.x Flaherty, L. T., Weist, M. D., & Warner, B. S. (1996). School-based mental health services in the United States: History, current models and needs. Community Mental Health Journal, 32(4), 341-352. doi: http://dx.doi.org/10.1007/BF02249452 Greenhalgh, T., Robert, G., Macfarlane, F., Bate, P., & Kyriakidou, O. (2004). Diffusion of Innovations in service organizations: Systematic review and recommendations. The Milbank Quarterly, 82(4), 581–629. http://doi.org/10.1111/j.0887-378X.2004.00325.x Gulliver, A., Griffiths, K. M., & Christensen, H. (2010). Perceived barriers and facilitators to mental health help-seeking in young people: A systematic review. BMC Psychiatry, 10, 113. doi:http://dx.doi.org.proxy1.cl.msu.edu/10.1186/1471-244X-10-113 Gullotta TP., & Noyes, L. (1995). The changing paradigm of community health: the role of school-based health centers. Adolescence, 30(117), 107–115. Guo, J. J., Wade, T. J., & Keller, K. N. (2008). Impact of School-Based Health Centers on Students with Mental Health Problems. Public Health Reports, 123(6), 768-780. 101 Guo, J. J. P., Wade, T. J. P., Pan, W. P., & Keller, K. N. M. P. A. (2010). School-Based Health Centers: Cost-Benefit Analysis and Impact on Health Care Disparities. American Journal of Public Health, 100(9), 1617-1623. Hacker, K., & Wessel, G. L. (1998). School-based health centers and school nurses: Cementing the collaboration. The Journal of School Health, 68(10), 409-414. Harold, R. D., & Harold, N. B. (1993). School-based clinics: A response to the physical and mental health needs of adolescents. Health & Social Work, 18(1), 65-74. Hill, J., Ohmstede, T., & Mims, M. (2012). A look into mental health in the schools. International Journal of Psychology: A Biopsychosocial Approach, 11, 119-131. doi: http://dx.doi.org/10.7220/1941-7233.11.6 Institute for Digital Research (IDRE): UCLA Statistical Consulting Group. (2017). Introduction to linear mixed models. Retrieved from https://stats.idre.ucla.edu/other/mult- pkg/introduction-to-linear-mixed-models/. Jennings, J., Pearson, G., & Harris, M. (2000). Implementing and maintaining school-based mental health services in a large, urban school district. The Journal of School Health, 70(5), 201-205. Jensen, P. S., Goldman, E., Offord, D., Costello, E. J., Friedman, R., Huff, B., . . . Roberts, R. (2011). Overlooked and underserved: “Action signs” for identifying children with unmet mental health needs. Pediatrics, 128(5), 970-979. doi:http://dx.doi.org.proxy2.cl.msu.edu/10.1542/peds.2009-0367 Johnson, V., & Hutcherson, V. (2006). A Study of the utilization patterns of an elementary school-based health clinic over a 5-year period. The Journal of School Health, 76(7), 373-378. Juszczak, L., Melinkovich, P., & Kaplan, D. (2003). Use of health and mental health services by adolescents across multiple delivery sites. Journal of Adolescent Health, 32(6), 108-118. doi: http://dx.doi.org/10.1016/S1054-139X(03)00073-9 102 Kaplan, D. W., Calonge, B. N., Guernsey, B. P., & Hanrahan, M. B. (1998). Managed care and school-based health centers: Use of health services. Archives of Pediatrics & Adolescent Medicine, 152(1), 25-33. Kataoka, S. H., Zhang, L., & Wells, K. B. (2002). Unmet need for mental health care among U.S. children: Variation by ethnicity and insurance status. The American Journal of Psychiatry, 159(9), 1548-1555. doi:http://dx.doi.org.proxy2.cl.msu.edu/10.1176/appi.ajp.159.9.1548 Keeton, V., Soleimanpour, S., & Brindis, C. D. (2012). School-based health centers in an era of health care reform: Building on history. Current Problems in Pediatric and Adolescent Health Care, 42(6), 132–156. doi:10.1016/j.cppeds.2012.03.002 Kenward M.G., Roger, J.H. (1997). Small sample inference for fixed effects from restricted maximum likelihood. Biometrics, 53:14. Knopf, J. A., Finnie, R. K. C., Peng, Y., Hahn, R. A., Truman, B. I., Vernon-Smiley, M., . . . Fullilove, M. T. (2016). School-based health centers to advance health equity: A community guide systematic review. American Journal of Preventive Medicine, 51(1), 114-126. doi:http://dx.doi.org.proxy2.cl.msu.edu/10.1016/j.amepre.2016.01.009 Lai, K., Guo, S. Ijadi-Maghsoodi, R., Puffer, M., & Kataoka, S. H. (2016). Bringing wellness to schools: Opportunities for and challenges to mental health integration in school-based health centers. Psychiatric Services, 67(12), 1328. Langer, D. A., Wood, J. J., Wood, P. A., Garland, A. F., Landsverk, J., & Hough, R. L. (2015). Mental health service use in schools and non-school-based outpatient settings: Comparing predictors of service use. School Mental Health, 7(3), 161-173. doi: http://dx.doi.org/10.1007/s12310-015-9146-z Larson, S. A., & Chapman, S. A. (2013). Patient-centered medical home model: Do school-based health centers fit the model? Policy, Politics, & Nursing Practice, 14(3-4), 163-174. doi: http://dx.doi.org/10.1177/1527154414528246 Larson, S; Spetz, J; Brindis, CD; & Chapman, S. (2017). Characteristic differences between school-based health centers with and without mental health providers: A review of national trends. Journal of pediatric health care: Official publication of National 103 Association of Pediatric Nurse Associates & Practitioners, 31(4), 484 - 492. doi: 10.1016/j.pedhc.2016.12.007. Retrieved from: http://escholarship.org/uc/item/6fs6p2bg Liu, R. Y., Ramowski, S. K., & Nystrom, R. J. (2010). Health service integration in Oregon school-based health centers: Meeting the mental health needs of young people. Advances in School Mental Health Promotion, 3(2), 26-36. doi: http://dx.doi.org/10.1080/1754730X.2010.9715678 Lofink H, Kuebler J, & Juszczak L. (2013). 2010-2011 School-Based Health Alliance Census Report. Retrieved from http://www.sbh4all.org/wp- content/uploads/2015/02/CensusReport_2010-11CensusReport_7.13.pdf. Mason-Jones, A. J., Crisp, C., Momberg, M., Koech, J., De Koker, P., & Mathews, C. (2012). A systematic review of the role of school-based healthcare in adolescent sexual, reproductive, and mental health. Syst Rev, 1, 49. doi: 10.1186/2046-4053-1-49 Mcculloch, C. E. and Neuhaus, J. M. (2013). Generalized Linear Mixed Models. Encyclopedia of Environmetrics. 3. doi: 10.1002/9780470057339.vag009.pub2 McNall, M. A., Lichty, L. F., & Mavis, B. (2010). The impact of school-based health centers on the health outcomes of middle school and high school students. American Journal of Public Health, 100(9), 1604-1610. doi: http://dx.doi.org/10.2105/AJPH.2009.183590 Merikangas, K. R., He, J., Burstein, M., Swendsen, J., Avenevoli, S., Case, B., . . . Olfson, M. (2011). Service utilization for lifetime mental disorders in U.S. adolescents: Results of the national comorbidity survey-adolescent supplement (NCSA). Journal of the American Academy of Child & Adolescent Psychiatry, 50(1), 32-45. doi:http://dx.doi.org.proxy1.cl.msu.edu/10.1016/j.jaac.2010.10.006 Morone, J. A., Kilbreth, E. H., & Langwell, K. M. (2001). Back to school: A health care strategy for youth. Health Affairs, 20(1), 122-136 National School-Based Health Care Census. (2014). 2013-2014 census report of school-based health centers [Data file]. Retrieved from http://www.sbh4all.org/school-health- care/national-census-of-school-based-health-centers/. 104 O'Leary, S. T., Lee, M., Federico, S., Barnard, J., Lockhart, S., Albright, K., . . . Kempe, A. (2014). School-based health centers as patient-centered medical homes. Pediatrics, 134(5), 957-964. doi:http://dx.doi.org.proxy1.cl.msu.edu/10.1542/peds.2014-0296 Padilla-Frausto, D. I., Grant, D., Aydin, H., & Anguilar-Gaxiola, S. (2014). Three out of four children with mental health needs in California do not receive treatment despite having health care coverage. Center for Health Policy Research, Available from: https://www.researchgate.net/publication/320040250_Using_Children's_Literature_to_St rengthen_Social_and_Emotional_Learning?pag:3:mrect: Parasuraman SR, Shi L. (2014). Differences in access to care among students using school-based health centers. J Sch Nurs, 31(4):291-299. Pastore, D. R., Juszczak, L., Fisher, M. M., & Friedman, S. B. (1998). School-based health center utilization: A survey of users and nonusers. Arch Pediatr Adolesc Med, 152(8), 763-767. Penn State: Eberly College of Science. (2017). Applied multivariate statistical analysis: Mixed model analysis. Retrieved from https://onlinecourses.science.psu.edu/stat505/node/157 Pettigrew, A. M., Woodman, R. W., & Cameron, K. S. (2001). Studying organizational change and development: Challenges for future research. Academy of Management Journal, 44(4), 697-713. Retrieved from https://search-proquest- com.proxy2.cl.msu.edu/docview/199783433?accountid=12598 Price, O.A. (2016). School-centered approaches to improve community health: Lessons from school-based health centers. Economic Studies at Brookings, 5, 1-17. Ran, T., Chattopadhyay, S. K., & Hahn, R. A. (2016). Economic evaluation of school-based health centers: A community guide systematic review. Am J Prev Med, 51(1), 129-138. doi: 10.1016/j.amepre.2016.01.017 Richardson, J. W. (2007). Building bridges between school-based health clinics and schools. Journal of School Health, 77(7), 337-343. 105 Rones, M., & Hoagwood, K. (2000). School-based mental health services: A research review. Clinical Child and Family Psychology Review, 3(4), 223-241. doi: 10.1023/a:1026425104386 Rose, B. L., Mansour, M. and Kohake, K. (2005), Building a Partnership to Evaluate School‐ Linked Health Services: The Cincinnati School Health Demonstration Project. Journal of School Health, 75: 363-369. doi:10.1111/j.1746-1561.2005.tb06638.x Santor, D. A., Poulin, C., LeBlanc, J. C., & Kusumakar, V. (2006). Examining school health center utilization as a function of mood disturbance and mental health difficulties. Journal of Adolescent Health, 39(5), 729-735. doi: http://dx.doi.org/10.1016/j.jadohealth.2006.04.010 School Based Health Alliance. (2016). Core Competencies. Retrieved from http://www.sbh4all.org/resources/core-competencies/ Schlitt, J. J., Juszczak, L. J., & Eichner, N. H. (2008). Current status of state policies that support school-based health centers. Public Health Reports, 123(6), 731-750. Silberberg, M., & Cantor, J. C. (2008). Making the case for school-based health: Where do we stand? Journal of Health Politics, Policy and Law, 33(1), 3-37. doi: http://dx.doi.org/10.1215/03616878-2007-045 Simon, A. E., Pastor, P. N., Reuben, C. A., Huang, L. N., & Goldstrom, I. D. (2015). Use of mental health services by children ages six to 11 with emotional or behavioral difficulties. Psychiatric Services, 66(9), 930-937. doi:http://dx.doi.org.proxy2.cl.msu.edu/10.1176/appi.ps.201400342 Smith, B.L., (2013). Expanding school-based care. Early Career Psychology, 44(8), 44. Soleimanpour, S. M. P. H., Geierstanger, S. P. M. P. H., Kaller, S. M. P. H., McCarter, V. P., & Brindis, C. D. D. (2010). The role of school health centers in health care access and client outcomes. American Journal of Public Health, 100(9), 1597-1603. Sprigg, S. M., Wolgin, F., Chubinski, J., & Keller, K. (2017). School-based health centers: A funder's view of effective grant making. Health Affairs, 36(4), 768. doi: http://dx.doi.org/10.1377/hlthaff.2016.1234 106 Stone, S., Whitaker, K., Anyon, Y., & Shields, J. P. (2013). The relationship between use of school-based health centers and student-reported school assets. Journal of Adolescent Health, 53(4), 526-532. doi: http://dx.doi.org/10.1016/j.jadohealth.2013.05.011 Swider, S. M., & Valukas, A. (2004). Options for sustaining school-based health centers. The Journal of School Health, 74(4), 115-118. Szumilas, M., Kuthcer, S., & LeBlanc, J.C. (2010). Use of school-based health centers for mental health support in Cape Breton, Nova Scotia. The Canadian Journal of Psychiatry, 55(5), pp. 319-238. Wade, T. J., P.H.D., Mansour, M. E., M.D., Guo, J. J., P.H.D., Huentelman, T., M.A., Line, K., M.E.D., & Keller, K. N., M.P.A. (2008). Access and utilization patterns of school-based health centers at urban and rural elementary and middle schools. Public Health Reports, 123(6), 739. Retrieved from https://search-proquest- com.proxy1.cl.msu.edu/docview/201597552?accountid=12598 Walker, S. C., Kerns, S. E. U., Lyon, A. R., Bruns, E. J., & CosgroveW., T. (2010). Impact of school-based health center use on academic outcomes. Journal of Adolescent Health, 46(3), 251-257. doi:http://dx.doi.org.proxy1.cl.msu.edu/10.1016/j.jadohealth.2009.07.002 Weist, M. D., Paskewitz, D. A., Warner, B. S., & Flaherty, L. T. (1996). Treatment outcome of school-based mental health services for urban teenagers. Community Mental Health Journal, 32(2), 149-157. Winter, B. (2013). Linear models and linear mixed effects models in R with linguistic applications. Retrieved from https://arxiv.org/pdf/1308.5499.pdf. Wolk, L. I., & Kaplan, D. W. (1993). Frequent school-based clinic utilization: A comparative profile of problems and service needs. Journal of Adolescent Health, 14(6), 458-463. doi: http://dx.doi.org/10.1016/1054-139X(93)90118-9 Young, T. L., D'angelo, S. L. and Davis, J. (2001). Impact of a school-based health center on emergency department use by elementary school students. Journal of School Health, 71, 196–198. doi: 10.1111/j.1746-1561.2001.tb07316.x 107 Zhang, H., Lu, N., Feng, C., Thurston, S. W., Xia, Y., & Tu, X. M. (2011). On Fitting Generalized Linear Mixed-effects Models for Binary Responses using Different Statistical Packages. Statistics in Medicine, 30(20), 2562–2572. http://doi.org/10.1002/sim.4265 Zuur A.F., Ieno, E.N., Walker, N.J., Saveliev A.A., & Smith, G.M. (2009) GLMM and GAMM. In: Mixed effects models and extensions in ecology with R. Statistics for Biology and Health. Springer, New York, NY 108