UNIVERSAL PROGRAMMING FOR SOCIAL EMOTIONAL LEARNING AND
EFFECTS ON STUDENT COMPETENCE AND ACHIEVEMENT
By
Laura S. Benson

A DISSERTATION
Submitted to
Michigan State University
in partial fulfillment of requirements
for the degree of
School Psychology – Doctor of Philosophy
2017

ABSTRACT
UNIVERSAL PROGRAMMING FOR SOCIAL EMOTIONAL LEARNING AND
EFFECTS ON STUDENT COMPETENCE AND ACHIEVEMENT
By
Laura S. Benson
Many students have inadequate social-emotional skills, which, can negatively affect
academic performance, behavior, and overall well-being (Blum, Libbey, Bishop, & Bishop,
2004; Rimm-Kaufman, Pianta, & Cox, 2000; Zigler & Bishop-Josef, 2006). Schools, because of
their public educational role and the significant time that children spend there, can provide an
ideal context for social-emotional development and intervention (Rones & Hoagwood, 2000;
Weisz, Sandler, Durlak, & Anton, 2005). Students who have experienced high quality, schoolbased, social-emotional learning (SEL) programs have demonstrated improved academic
performance, attitudes toward school, and social-emotional skills as well as reduced conduct
problems, anxiety, and aggression (Durlak et al., 2011; Payton et al., 2008). Yet, school
infrastructures often fail to support the integration of SEL programming in ways that are
sustainable and embedded in the day-to-day functioning of students and educators.
The purpose of this study was to examine the potential benefits of integrating Second
Step (Committee for Children, 2012), a widely disseminated school-based SEL program, into an
existing tiered model of educational programming. As such, the study addressed the following
research questions: Controlling for baseline skills, do students receiving Second Step show
greater improvement in social-emotional competence and academic achievement than students in
a wait-list comparison group? If so, do improvements depend on student level of learning risk at
baseline? Are intervention effects moderated by gender?

The primary research questions are answered through secondary analysis of existing data
using a quasi-experimental wait-list comparison group design with pretest and posttest. The
dataset included teacher ratings of social-emotional competence and academic achievement data
collected at baseline and following SEL intervention for all students attending one elementary
school.
The hypothesis that all students would benefit similarly from Second Step received mixed
support. Statistically significant improvements for the intervention group were found only for
academic outcomes and not social-emotional competence. Post-hoc analysis revealed that grade
level moderated the effects of treatment, indicating that Second Step produced significantly
greater gains in social-emotional competence and academic achievement for lower elementary
students compared to upper elementary students. Lower elementary students’ empathy skills,
reading scores, and math scores significantly improved among those receiving Second Step
compared to lower elementary students in the comparison group. The positive effects of Second
Step participation on reading scores were specific to students demonstrating a moderate level of
learning risk at baseline., Overall, girls showed greater improvements in social-emotional
competence from pretest to posttest than boys, but gender did not moderate students’ response to
intervention as predicted.

TABLE OF CONTENTS

LIST OF TABLES ......................................................................................................................... vi	
LIST OF FIGURES ..................................................................................................................... viii	
CHAPTER 1 ................................................................................................................................... 1	
INTRODUCTION .......................................................................................................................... 1	
A Shift Toward Prevention ......................................................................................................... 1	
Problem Statement ...................................................................................................................... 4	
Social Emotional Learning and MTSS ....................................................................................... 4	
The Present Study ....................................................................................................................... 7	
CHAPTER 2 ................................................................................................................................... 9	
LITERATURE REVIEW ............................................................................................................... 9	
Social and Emotional Competence and Student Outcomes ........................................................ 9	
A Framework for School-Based Social Emotional Learning ................................................... 12	
School-Based SEL Intervention and Outcomes ........................................................................ 15	
Effect of School-Based SEL on Student Outcomes .......................................................... 15	
Interpreting the Effects of School-Based SEL .................................................................. 17	
Differential Effects: Who Reaps the Benefits? ................................................................. 18	
Delivery Matters: Teacher vs. Non-School Personnel ...................................................... 20	
Evidence-Based SEL: Second Step ........................................................................................... 21	
Measuring Social Emotional Competence in School................................................................ 28	
Strengths-Based Assessment ............................................................................................ 30	
Teacher Ratings of Social and Academic Behavior ......................................................... 31	
School-Based Universal Screening ................................................................................... 32	
Research Questions & Hypotheses ........................................................................................... 34	
CHAPTER 3 ................................................................................................................................. 38	
METHOD ..................................................................................................................................... 38	
Design ....................................................................................................................................... 38	
Setting ....................................................................................................................................... 39	
Classrooms and Teachers .......................................................................................................... 41	
Students ..................................................................................................................................... 42	
Second Step Training and Delivery........................................................................................... 44	
Measures ................................................................................................................................... 46	
Social-Emotional Competence Outcome Measure ........................................................... 46	
Academic Outcome Measure ............................................................................................ 49	
Social-Emotional Screening Measure ............................................................................... 50	
Academic Screening Measure........................................................................................... 52	
Learning Risk Measure ..................................................................................................... 53	
Data Collection ......................................................................................................................... 54	

iv

CHAPTER 4 ................................................................................................................................. 56	
RESULTS ..................................................................................................................................... 56	
Preliminary Analyses ................................................................................................................ 56	
Missing Data ..................................................................................................................... 56	
Screening for Outliers ....................................................................................................... 58	
Tests for Violations of Statistical Assumptions ................................................................ 58	
Equivalence of Intervention Groups at Baseline .............................................................. 62	
Primary Analysis....................................................................................................................... 64	
Effects of Second Step on Social-Emotional Competence ....................................................... 66	
Effects of Second Step on Academic Achievement .................................................................. 73	
CHAPTER 5 ................................................................................................................................. 83	
DISCUSSION ............................................................................................................................... 83	
Second Step and Social-Emotional Competence ...................................................................... 84	
Second Step and Academic Achievement ................................................................................. 90	
Limitations of the Study............................................................................................................ 93	
Future Directions and Implications for Practice ....................................................................... 97	
APPENDIX ................................................................................................................................. 100	
REFERENCES ........................................................................................................................... 102	

v

LIST OF TABLES

Table 1 Grade Level, Gender, and Ethnicity of Teachers (N = 17) by Intervention Group ......... 42	
Table 2 Grade, Gender, and Ethnicity of Students (N = 369) by Intervention Group .................. 43	
Table 3 Means and Standard Deviations for Dependent Variables and Covariates by Intervention
Group with Percent Missing ......................................................................................................... 57	
Table 4 Cronbach’s Alpha (a) Coefficients of Reliability for the DESSA Scales by Intervention
Group ............................................................................................................................................ 60	
Table 5 Pearson Correlations Between Social-Emotional Competence Variables for Second Step
Group ............................................................................................................................................ 61	
Table 6 Pearson Correlations Between Social-Emotional Competence Variables for Comparison
Group ............................................................................................................................................ 62	
Table 7 Learning Risk, Gender, and Grade of Students (N = 369) by Intervention Group.......... 63	
Table 8 Means and Standard Deviations for Dependent Variables and Covariates by Grade ..... 64	
Table 9 Research Question 1 and Data Analysis .......................................................................... 67	
Table 10 Descriptive Statistics for Social-Emotional Competence Variables by Intervention
Group, Learning Risk, Grade, and Gender ................................................................................... 67	
Table 11 Multivariate Analysis of Covariance Summary for Social-Emotional Competence by
Intervention Group, Learning Risk, Grade, and Gender ............................................................... 68	
Table 12 Univariate Analysis of Covariance Summary for Social-Emotional Competence by
Intervention Group, Learning Risk, Grade, and Gender for Measures ......................................... 69	
Table 13 Descriptive Statistics of Social-Emotional Competence by Intervention Group and
Grade Level ................................................................................................................................... 72	
Table 14 Descriptive Statistics of Social-Emotional Competence by Intervention Group and
Learning Risk ................................................................................................................................ 73	
Table 15 Descriptive Statistics of Social-Emotional Competence by Intervention Group and
Gender ........................................................................................................................................... 73
Table 16 Research Question 2 and Data Analysis ........................................................................ 74	

vi

Table 17 Descriptive Statistics for Academic Achievement by Intervention Group, Learning
Risk, Grade, and Gender ............................................................................................................... 75	
Table 18 Analysis of Covariance Summary for Reading Achievement by Intervention Group,
Learning Risk, Gender, and Grade ............................................................................................... 76	
Table 19 Analysis of Covariance Summary for Math Achievement by Intervention Group,
Learning Risk, Gender, and Grade ............................................................................................... 77	
Table 20 Comparisons of Mean Differences in Reading Achievement by Learning Risk and
Intervention Group ........................................................................................................................ 80	
Table 21 Comparisons of Mean Differences in Math Achievement by Learning Risk................ 81	
Table 22 Descriptive Statistics for Academic Achievement by Intervention Group and Grade .. 82	
Table 23 Descriptive Statistics for Academic Achievement by Intervention Group and Learning
Risk ............................................................................................................................................... 82	
Table 24 Descriptive Statistics for Academic Achievement by Intervention Group and Gender 82	

vii

LIST OF FIGURES

Figure 1. Multi-Tiered System of Supports (MTSS) instruction and intervention. ........................ 3	
Figure 2. Conceptual framework connecting school-based Social Emotional Learning (SEL)
intervention to student outcomes. ................................................................................................... 6	
Figure 3. Second Step implementation and data collection schedule. .......................................... 45	
Figure 4. Using DESSA-mini and DIBELS data to categorize student level of learning risk. .... 54	
Figure 5. Simple plot for interaction effects between intervention group by grade level on mean
empathy score ............................................................................................................................... 70	
Figure 6. Simple plot for interaction effects between intervention group by grade level on
reading achievement ..................................................................................................................... 77	
Figure 7. Simple plot for interaction effects between intervention group and grade level on math
achievement .................................................................................................................................. 78	
Figure 8. Simple plot for interaction effects between intervention group by learning risk on
reading achievement ..................................................................................................................... 79	

viii

CHAPTER 1
INTRODUCTION
There has been much debate about the relative importance of academic versus socialemotional skills for learning and achievement, but decades of research have clearly shown that
social, emotional, and academic skills are inextricably connected (Blum et al., 2004; Zigler &
Bishop-Josef, 2006). This large and growing body of literature demonstrates that social and
emotional competence equips children with the necessary skills to successfully manage their
emotions, relationships, and behaviors, all of which are important prerequisite skills for school
and life success (Jones, Greenberg, & Growley, 2015; Pianta, Hamre, & Allen, 2012; Raver &
Knitzer, 2002, Wang, Haertel, & Walberg, 1997; Valiente, Lemery-Chalfant, Swanson, &
Reiser, 2008). Deficits in these key areas, on the other hand, have been linked to a host of
learning and school-related problems, including lower levels of self-concept, school attendance,
and academic performance and higher incidence of internalizing symptoms, feelings of rejection,
and social isolation (Graham & Juvonen, 1998; Guerra & Bradshaw, 2008; Juvonen, Nishina, &
Graham, 2000; Masten & Coatsworth, 1998). Despite the growing empirical and practical
support for the use of a continuum of social-emotional interventions to promote school and life
outcomes for all students, integration of such programming with existing educational
frameworks has been largely unrealized (Zins & Elias, 2006).
A Shift Toward Prevention
School-based social and emotional programming has traditionally emphasized delivery of
discrete services that reactively focus on the diagnosis and treatment of those few students
exhibiting the most severe emotional and behavioral problems. One critique of this approach is
that it often results in a fragmented system of redundant and ineffective services that are

1

disconnected from the more central framework of academic programming. With disproportionate
efforts and resources devoted to intensive intervention targeting only those students exhibiting
the most profound difficulties, traditional approaches to social-emotional programming
deemphasize prevention and early intervention efforts. For this reason, approaches that tend to
withhold supports and only intervene after significant delays and difficulties have emerged are
often characterized as "wait to fail" initiatives.
An alternative to the "wait to fail" approach that has dominated K-12 education in recent
years is a multi-tiered practice framework, often referred to in schools as Response to
Intervention (RTI) or, more recently, the Multi-Tiered System of Supports (MTSS; Castillo et
al., 2010; Batsche, Castillo, Dixon, & Forde, 2008). These multi-tiered approaches to educational
programming have roots in the public health model of disease prevention that differentiates
primary, secondary, and tertiary levels of intervention depending on the degree of need and
response to intervention (Sugai & Horner, 2009). Schools operating on a multi-tiered model
expend resources and deliver services on a continuum with the goal of preventing problems from
developing and remediating existing problems before they can escalate (Gresham, 2005). The
tiered model relies on universal screening, evidence-based practices, and continual progress
monitoring to determine intervention effectiveness and inform educational decisions (Tilly,
Reschly, & Grimes, 1999).
In the past decade, the RTI framework has gained both practical and empirical support as
an effective model to promote core academic development (i.e., reading, math) for all students
and to identify students at risk for learning difficulties (Jenkins, Hudson, & Johnson, 2007;
Jimerson, Burns, & VanDerHeyden, 2007). This has not been the case, however, for schoolbased social-emotional and behavioral programming. Rather, these initiatives have traditionally

2

operated in fragmented and marginalized “silos”, cut off from the day-to-day functioning of the
formal education system (Sugai & Horner, 2009). In contrast to the more narrowly focused RTI
framework, MTSS is a comprehensive evidence-based model of education that employs databased problem solving to integrate programming into a unified system of tiered support (Gamm
et al., 2012). The MTSS approach capitalizes on the interconnectedness of academic, behavioral,
and social-emotional development to improve outcomes for all students (Lane & Menzies, 2003;
McIntosh, Chard, Boland, & Horner, 2006). This integrated system of support is provided to
students in tiers of increasing intensity according to student level of need. MTSS represents a
needs-driven model that aims to ensure instructional and intervention resources reach students at
suitable levels to promote outcomes for all students. The current study focused on universal SEL
programming (Tier I), but the three tiers typically associated with the MTSS framework are
provided in Figure 1 to illustrate the larger, multi-tiered context that the proposed initiative might
operate.

Tier	3:	FEW
Intensive
Individualized
In	addition	to	Tier	1	&	2
Weekly	monitoring
Tier	2:	SOME
Targeted
Small	group	intervention	setting
In	addition	to	Tier	1	instruction	
Monthly	progress	monitoring
Tier	1:	ALL
Universal
General	education	 classroom
Evidence-based	 core	academic	&	SEL	instruction
Quarterly	benchmark

Figure 1. Multi-Tiered System of Supports (MTSS) instruction and intervention.

3

Problem Statement
Limited social-emotional skills in the elementary years can have a negative effect on
academic performance, behavior, and overall well-being (Blum et al., 2004; Rimm-Kaufman et
al., 2000; Zigler & Bishop-Josef, 2006). Schools, because of their public educational role and the
significant time that children spend there, can provide an ideal context for social-emotional
development and intervention (Rones & Hoagwood, 2000; Weisz et al., 2005). A growing body
of research indicates that social-emotional competence and academic achievement are
interwoven and that integrated instruction and support in both areas can maximize student
success (Zins & Elias, 2007) and reduce the risk of maladjustment (Elias et al., 1997; Zins,
Bloodworth, Weissberg, & Walberg, 2004). However, school infrastructures often fail to
integrate social and emotional learning in ways that are sustainable and embedded in the day-today functioning of students and educators.
Social Emotional Learning and MTSS
In 1997, the term social emotional learning (SEL) was introduced to describe the process
by which individuals "learn to recognize and manage emotions, care about others, make good
decisions, behave ethically and responsibly, develop positive relationships, and avoid negative
behaviors" (Zins et al., 2004). SEL provides a unifying conceptual framework for the
organization and integration of school-based prevention programs. Students who have
experienced high quality, school-based SEL programs have demonstrated improved academic
performance, attitudes toward school, and social-emotional skills and reduced conduct problems,
anxiety, and aggression (Durlak et al., 2011; Payton et al., 2008).
In response to growing research and interest in SEL, a variety of commercialized
programs to help schools support the development of SEL have been developed (Merrell &

4

Gueldner, 2010). The Collaborative for Academic, Social, and Emotional Learning has identified
19 evidence-based SEL programs that have been found to promote positive academic, socialemotional, and behavioral outcomes in children and adolescents (CASEL, 2013). Second Step:
K-5 4th Edition (Committee for Children, 2012), is one school-based, universal program
recognized by CASEL for its established evidence base demonstrating effectiveness across grade
levels and school contexts on increasing school success and decreasing problem behaviors
through the promotion of social-emotional competence. Other national organizations have also
endorsed Second Step as an SEL program meeting the criteria for evidence-based practice. For
example, The U.S. Department of Education recommends Second Step as an exemplary program,
reporting that there is evidence of the program’s efficacy (Exemplary and promising safe,
disciplined, and drug-free schools program, 2002). Second Step is also included in the Substance
Abuse and Mental Health Services Administration National Registry of Evidence-based
Programs and Practice (NREPP, 2009) and the Office of Juvenile Justice and Delinquency
Prevention Model Programs Guide (OJJDP, n.d.).
Second Step is a universal, skills-focused SEL curriculum based on a cognitive problemsolving model with an emphasis on direct instruction of skills that strengthen the ability to learn,
have empathy, manage emotions, and solve problems. According to the Second Step logic model,
when students are provided direct instruction in social-emotional skills along with opportunities
for practice and reinforcement, they are likely to demonstrate improvements in intermediate
outcomes that lead to a cascade of positive distal outcomes (see Figure 2). The most recent
edition of Second Step (Committee for Children, 2012) includes separate curricula for each grade
to ensure developmentally appropriate and relevant instruction for all children. The program
includes a total of 22 lessons that are organized across four units: (a) Skills for Learning, (b)

5

Empathy, (c) Emotion Management, and (d) Problem Solving. Second Step was designed to be

Integration with academic
curriculum
Teacher instructional
practices

SEL skills acquisition
Improved attitudes about
self, others, schools
Improvement in
classroom climate
Improved teacherstudents relationships

Secondary Outcomes

Explicit SEL instruction

Immediate Outcomes

SEL Approaches

delivered by classroom teachers and integrated with everyday classroom routines and activities.

Positive social behaviors
Fewer conduct	problems
Less emotional distress
Academic success

Figure 2. Conceptual framework connecting school-based Social Emotional Learning (SEL)
intervention to student outcomes.
Despite growing empirical support and availability of programs like Second Step that are
designed to be implemented in the school setting, a wide gap exists between research and
practice in school-based SEL programming (Weisz et al., 2005). One key barrier to the
implementation of multi-tiered SEL programming is the commonly held assumption among
educators that the promotion of SEL in the classroom would necessarily compete with other high
priority objectives for which schools are held accountable (Seifer et al., 2004). It stands to reason
that, in the absence of immediate and clear connections between SEL programming and
academic outcomes, schools would likely be reluctant to endorse such initiatives and view SEL
as non-essential programming (Zins et al., 2004). Continued research demonstrating the link
between SEL and learning outcomes as well as the benefits of integrating SEL programming
with existing academic programming is needed to strengthen support for multi-tiered SEL among
6

educators and to increase the likelihood that evidence-based SEL programming will be fully
enmeshed in the everyday practices and routines of schools.
Another barrier to the advancement of multi-tiered SEL programming has been the lack
of normed and validated social-emotional assessment tools and procedures that are feasible and
sustainable for use in schools (Dishion & Tipsord, 2011, Merrell & Gueldner, 2012; Nese et al.,
2012; Tilly, 2008). Continued research is needed to better understand reliable, valid, and
practical assessment procedures to assist schools in determining student need for socialemotional support and evaluating the effectiveness of SEL programming to improve outcomes
for all students.
The Present Study
Schools can play an important role in preventing problems and promoting healthy
development by fostering not only academic skills but also social-emotional competence (Rones
& Hoagwood, 2000; Weisz et al., 2005). Research supports the use of the MTSS model to
effectively integrate school-based SEL intervention and assessment with preexisting academic
practices to maximize school success for all students (Cook, Burns, Browning-Wright, &
Gresham, 2010; Doll & Cummings, 2008; Domitrovich, Gest, Jones, Gill, & Derousie, 2010).
Although the challenges associated with the implementation and sustainability of evidence-based
practices in schools are well documented, there is little guidance in the literature on the
integration of SEL practices within a tiered model of service delivery (Dishion, 2011;
Domitrovich et al., 2010).
The purpose of this study was to address the need for more research examining the
potential benefits of integrating social-emotional screening and primary prevention into an
existing tiered model of educational programming. As such, this research examined whether the

7

addition of a universal social emotional learning (SEL) curriculum to a standard, academically
focused tiered model of service delivery improved elementary students' social-emotional
competence and academic performance.

8

CHAPTER 2
LITERATURE REVIEW
The purpose of this literature review is to establish the need for continued research
relating to the integration of social emotional learning (SEL) and assessment into routine school
practices. To that end, this chapter is subdivided into five sections. The first section defines
social-emotional competence and its association with developmental and educational outcomes.
The second section establishes the theoretical foundations that guide the research and support the
inclusion of universal SEL programming in the school setting. A practice framework that
informs the effective integration of SEL practices into the real-world context of classrooms and
schools is also presented. The third section of the review highlights key findings linking schoolbased SEL programs to the development of core competencies, learning outcomes, and indicators
of school success. Attention is devoted to examination of the research associated with Second
Step, a school-based SEL intervention program designed to promote social-emotional
competence. The fourth section presents key literature related to the valid and reliable
assessment of social-emotional competence, with emphasis on the use of universal, strengthsbased screening procedures within a tiered model of educational programming. The final section
summarizes the literature review and situates the proposed study within the existing body of
empirical literature relating to school-based SEL and its integration into existing frameworks of
educational programming.
Social and Emotional Competence and Student Outcomes
Children need a strong foundation in social-emotional competence to succeed in school
(Raver, 2002). In the context of school-based SEL, social-emotional competence represents a
student’s overall ability to meet the social and emotional demands of the learning environment

9

(McClelland, Acock, & Morrison, 2006; Merrell & Gueldner, 2012). Social-emotional
competence has been found to be reliable predictor of future academic performance and school
success, even more so than prior academic performance (Malecki & Elliot, 2002). Socialemotional skills are said to operate in conjunction with cognitive skills to promote school success
(Denham, Bassett, & Zinsser, 2012; DiPerna & Elliott, 1999; Flook, Repetti, & Ulman, 2005;
Malecki & Elliott, 2002; Welsh, Parke, Widaman, & O’Neil, 2001; Wentzel, 1993). The SEL
literature relies on a variety of overlapping social, behavioral, and affective constructs to
describe the skills, attitudes, and knowledge that enable the development of social-emotional
competence. The present study conceptualizes social-emotional skills according to the
framework created by the Committee for Children (2012) and used in the development of the
Second Step curriculum. From this framework, skills associated with social and emotional
competence are organized into four categories: skills for learning, empathy, emotion
management, and social problem solving. The four social-emotional skills of interest in the
current study are defined below along with empirical support linking the skill to student
outcomes and school success.
Skills for learning refers to the ability to apply executive function skills, specifically
attention, working memory, and inhibitory control, to enhance engagement in the learning
process and benefit from classroom instruction and experiences (Low, Cook, Smolkowski, &
Buntain-Ricklefs, 2015; McClelland et al., 2006). Attention in this case refers to the ability to
focus on a task or activity while ignoring extraneous factors that might inhibit engagement in the
learning process (Barkley, 1997; Low et al., 2015). Working memory refers to the ability to
remember and use information, such as a teacher’s directions or multi-step procedures for
solving a math problem (Demetriou, Christou, Spanoudis, & Platsidou, 2002; Low et al., 2015).

10

Inhibitory control, also referred to as effortful or impulse control, involves the ability to interrupt
automatic but inappropriate responses or actions and enlist appropriate behaviors instead (Blair,
2002; Eisenberg et al., 2005; Raver & Knitzer, 2002; Low et al., 2015; Rennie, Bull, &
Diamond, 2004). The proficient demonstration of skills for learning in elementary school has
been found to predict academic success and fewer behavior problems in the future (Blair &
Razza, 2007; Duncan et al., 2007; Else-Quest, Hyde, Goldsmith, & Van Hulle, 2006; Howse,
Lange, Farran, & Boyles, 2003; Kroesbergen, Van Luit, Van Lieshout, Van Loosbroek, & Vande
Rijit, 2009; McClelland et al., 2006; Ponitz, McClelland, Matthews, & Morrison, 2009; Vitaro,
Brendgen, Larose, & Tremblay, 2005).
Empathy refers to the ability to identify, understand, and respond in a caring way to how
someone else is feeling (Cohen & Strayer, 1996; Jolliffe & Farrington, 2004; Low et al., 2015).
The perspective-taking, interpersonal skills associated with empathy provide the critical
foundation for socially responsible behavior, friendships, cooperation, and social problem
solving (Mayer & Salovery, 1997; Saarni, 1997). Children who demonstrate higher levels of
empathy are more accepted by peers (Crick & Dodge, 1994; Fabes et al., 1994; Denham,
McKinley, Couchoud, & Holt, 1990; Izard et al., 2001), better prepared for school (Raver &
Knitzer, 2002), less likely to be aggressive towards peers (Arsenio, Cooperman, & Lover, 2000;
Kaukiainen et al., 1999), and enjoy greater academic success than students with lower skills in
this area (Katsurada & Sugawara, 1998; Wentzel, 1991, 1993).
Emotion management involves the ability to understand, monitor and manage emotions,
thoughts, and behaviors (Barkley, 2004; Eisenberg, Cumberland, & Spinrad, 1998; McClelland
et al., 2006; Raver & Knitzer, 2002). Specifically, children with strong emotion management
skills have the capacity to recognize strong feelings and use self-calming strategies to

11

successfully manage strong emotions. Emotion management is positively linked to young
children’s school transition, cognitive competence, and social functioning (Garner & Waajid,
2012; Raver & Knitzer, 2002). The ability to manage emotions effectively is also linked to
decreased levels of aggression and substance abuse (Brady, Myrick, & McElroy, 1998;
Underwood, Coie, & Herbsman, 1992; Vitaro, Ferland, Jacques, & Ladouceur, 1998) as well as
improvements in self-control and effective problem solving (Donohew et al., 2000; Greenberg,
Kusche, Cook, & Quamma, 1995; Simons, Carey, & Gaher, 2004). Students who struggle with
emotion management, on the other hand, are more likely to demonstrate socially unacceptable
behavior, lower academic self-concept, more internalizing behaviors, and gain less peer
acceptance than children who can successfully manage their emotions (Eisenberg, Fabes, &
Losoya, 1997; Flook et al., 2005).
Problem solving refers to the ability to effectively handle personal challenges and
interpersonal conflicts and make socially responsible decisions (Hawkins, Farrington, &
Catalano, 1998; Shure & Spivack, 1980, 1982; Tolan & Guerra, 1994). Problem solving skills
are a critical component of social-emotional competence, relying largely on communication
skills to generate and select potentially effective strategies for coping with problematic social
situations (Elias & Tobias, 1996). Effective problem solving skills are linked to a reduction in
impulsive behavior, improvements in social adjustment, and avoidance of violence and other
negative social behaviors (Flook et al., 2005).
A Framework for School-Based Social Emotional Learning
Social emotional learning (SEL) is defined as the process by which children "learn to
recognize and manage emotions, care about others, make good decisions, behave ethically and
responsibly, develop positive relationships, and avoid negative behaviors" (Zins et al., 2004).

12

SEL is a broad term that encompasses a variety of techniques to help individuals develop
competencies and acquire knowledge to optimize school and life success (CASEL, 2003; Elias,
Parker, & Rosenblatt, 2005; Greenberg et al., 2003). It has been posited that universal, schoolbased efforts to promote SEL represent a promising approach to preventing behavioral
difficulties and enhancing student success and overall well-being (Elias et al., 1997; Zins &
Elias, 2006). School-based SEL is based on the premise that social-emotional skills are malleable
and can be developed and reinforced through direct instruction and enhancement of the
classroom environment (Elias et al., 1997; Greenberg et al., 2003; Zins & Elias, 2006; Zins et al.,
2004).
The present study was guided by the SEL conceptual framework set forth by the
Collaborative for Academic, Social and Emotional Learning (CASEL). CASEL, a national
collaborative of consultants and experts in social and emotional development, was created in
1994 with the mission of making evidence-based SEL an integral part of all students’ formal
education through high school. The framework provided a useful perspective for examining the
potential benefits of fostering social-emotional competence in the school setting. The conceptual
model is grounded in developmental research and prevention science, relying heavily on the
work of Zins and colleagues (2004), CASEL (2013), and Rimm-Kaufman et al. (2014), to
describe how school-based SEL intervention improves social, emotional, and academic
performance. From the CASEL framework, school-based SEL programs involve two core
components that serve as the “active ingredients” in the promotion of cognitive, affective, and
behavioral competencies: 1) direct instruction in processing, integrating, and selectively
applying social-emotional skills; and 2) establishment of safe, caring learning environments.
According to the conceptual framework, school-based SEL programs provide students with

13

direct instruction in social-emotional skills and establish safe learning environment, leading to
immediate improvements in the classroom social environment (e.g., teacher shows greater
emotional responsiveness to students, enhanced classroom management) and students’ socialemotional skills (e.g., emotional, interpersonal, cognitive, self-skills). These core intervention
components, along with the immediate improvements associated with participation in the
intervention, are thought to be the mechanisms through which more distal, or long-term,
improvements in social, emotional, and academic performance are realized (Rimm-Kaufman &
Hulleman, 2015). From this conceptualization, short-term improvements in students’ socialemotional skills support the ability to pay attention and follow instructions, eventually leading to
more distal improvements in school success. Similarly, the more immediate development of
problem-solving and emotional-management skills leads to a reduction of disruptive behavior
and, thus, frees up time for academics and eventually leads to improvements in academic
performance. Explicit instruction in social-emotional skills, the intervention “active ingredient”,
initially ameliorates intense emotions (e.g., anxiety, anger) that can interfere with cognition,
ultimately leading to long-term improvements in both school and life success. Figure 2 illustrates
this conceptual framework.
As social-emotional competencies are considered foundational to healthy development
across the life span, many scholars posit that school-based SEL programming best serves
students when situated within a multi-tiered prevention and intervention framework based on the
public health model (Bradshaw, Bottiani, Osher, & Sugai, 2014; Nastasi, Moore, & Varjas, 2004;
Ysseldyke et al., 2006). From this approach, school-based SEL programming at the universal or
primary level involves teaching social-emotional skills to all students through universal
programming with attention to such ecological factors as effective instructional practices,

14

supportive relationships, high expectations, and safe learning environments (Osher et al., 2008).
For effective implementation of this model to occur, however, schools need access to evidencebased intervention and assessment tools that can be feasibly and reliably transported to the real
world setting of the classroom.
School-Based SEL Intervention and Outcomes
The mounting evidence linking social-emotional competence to learning outcomes, along
with the escalation of emotional and behavioral problems in the school setting, has prompted the
development and dissemination of numerous prevention and intervention programs aimed at
promoting school-based SEL. School-based SEL programs are designed to foster the
development of competencies that provide the foundation for positive social-emotional
adjustment and academic performance (CASEL, 2003). A large volume of published studies has
examined the connection between school-based SEL and improvements in student socialemotional competence, school success and overall well-being. Several meta-analyses and largescale reviews have synthesized this extensive body of literature, establishing the empirical
foundation that links school-based SEL intervention and positive social-emotional, behavioral,
and learning outcomes (e.g., Durlak et al., 2011; Durlak & Wells, 1997; Greenberg,
Domitrovich, & Bumbarger, 2001; Payton et al., 2008; Zins et al., 2004).
Effect of School-Based SEL on Student Outcomes
In a large-scale analysis and discussion of the SEL literature, Durlak et al., (2011)
investigated the effects of 213 school-based universal SEL programs on kindergarten through
high school students’ (n = 270,034) school outcomes. The meta-analysis included only published
studies that focused on SEL skills, targeted students between the ages of five and eighteen
without an identified disability, included a control group and provided enough information to

15

calculate effect sizes. Student outcomes were grouped into six categories: (1) social-emotional
skills, (2) attitudes toward self, school, and others, (3) positive, social behaviors, (4) conduct
problems, (5) emotional distress, and (6) school performance. Results of the meta-analysis
demonstrated significant improvements in all outcome measures of interest for students receiving
SEL intervention compared to control group peers not participating in such programming.
Intervention students were found to experience improvements in social-emotional skills (g =
0.57), attitudes towards themselves and others (g = 0.23), positive social behaviors (g = 0.24),
conduct problems (g = 0.22) and emotional distress (g = 0.24). A link between SEL program
participation and higher academic scores was also found (g = 0.27).
It is important to note that, although the social-emotional skills outcome category
included a broad range of related outcomes (e.g., emotional self-awareness, coping with stress,
resolving conflict, and resisting unwanted peer pressure), assessments were based on student,
teacher, parent, or independent ratings completed in structured or test situations. Positive social
behaviors, on the other hand, included emotion management, positive social interaction,
cooperation, leadership, problem-solving, and social assertiveness, as reflected in daily
classroom behavior rather than hypothetical or test situations (Durlak et al., 2011; Payton et al.,
2008). Therefore, the more robust effect size (g = 0.57) found for social-emotional skills reflects
demonstration of skills on a contrived task in a testing setting that likely varied significantly
from authentic classroom tasks in the actual classroom setting. When looking to understand the
benefits of participation in SEL intervention on students’ daily application of social-emotional
skills in the classroom as conceptualized in this study, the positive social behaviors outcome is
likely a better indicator.

16

In another large-scale review and analysis of prevention programs, Payton et al., (2008)
summarized the findings from three reviews of different types of SEL programs: (a) universal,
school-based SEL interventions, (b) intervention programming for students displaying early
signs of behavioral and emotional problems, and (c) after-school programs. Whereas Durlak et
al. (2011) included students between the ages of five and eighteen in their analysis, Payton and
colleagues (2008) included only studies targeting kindergarten through eighth-grade students (n
= 324,303). All other study inclusion criteria as well as student outcome categories were the
same across the two large-scale reviews. As with the Durlak and colleagues (2011) results,
Payton et al. (2008), found positive effects for students participating in school-based SEL
programs (n = 180) across the six outcomes of interest compared to control students. Students
receiving school-based SEL intervention demonstrated increased social-emotional skills (g =
0.60), higher academic performance (g = 0.28), enhanced attitudes towards self and others (g =
0.23) and increased positive social behavior (g = 0.24). SEL program participation was also
linked to decreases in conduct problems (g = 0.23) and emotional distress (g = 0.23) compared to
students not receiving intervention.
Interpreting the Effects of School-Based SEL
Although findings from the two large-scale analyses discussed above (Durkin et al.,
2011; Payton et al., 2008) indicate that SEL programs generally result in small effect sizes, the
authors argue that the application of Cohen’s (1988) standard conventions for determining the
magnitude of effects may not be appropriate in the study and evaluation of school-based SEL
intervention. Instead, the researchers advocate for interpretation within the context of previous,
related research with an emphasis on the practical value of indicated program effects (Durlak,
2009; Hill, Bloom, Black, & Lipsey, 2008). Interpreted in this way, the SEL programs included

17

in the Durlak et al. (2011) analysis were found to yield results similar to or, in some cases, higher
than those achieved by other psychosocial and educational interventions. To highlight this point,
the authors emphasize that the mean effect size for academic achievement (g = 0.27) found in
their analysis was comparable to the results of 76 meta-analyses of strictly educational
interventions (Hill et al., 2007). According to this line of reasoning, when placed in the context
of previous research, meta-analysis findings offers strong support that school-based SEL
programs are among the most successful interventions offered to school-aged youth.
Also, emphasizing the practical value of seemingly modest effect sizes, Payton and
colleagues (2008) translated program effects into improvement indices to show percentile gains
achieved by the average student in an SEL intervention class compared to the average student in
a control class. The authors reported improvement indices ranging from 9 to 10 percentile points
in positive attitudes and social behaviors, conduct problems, and emotional distress; 11
percentile points in academic performance; and 23 percentile points in social-emotional skills.
According to the authors, improvement indices provide a better indication of the practical value
of improvements in student outcomes than effect size alone (Payton et al., 2008).
Differential Effects: Who Reaps the Benefits?
One noted limitation in the empirical literature examining the effects of school-based
SEL intervention is the failure in many research studies to report student demographic data (e.g.,
race/ethnicity, SES, age/grade, gender; Durlak et al. 2011, Payton et al., 2008). In the review of
school-based SEL programs conducted by Payton et al., such data were so limited that the
authors were unable to report differential effects across participant demographics. Noting a
similar omission of demographic data, Durlak et al. (2011) analyzed the differential effects by
school’s geographic location, student ethnicity, and students’ mean age across a subset of studies

18

that included the information. Durlak and colleagues found significant effects for only one
demographic characteristic on one of the six outcome categories: students’ mean age
significantly and negatively related to social-emotional skills (rs = -.27). In other words,
students’ social-emotional skills tended to decrease with age. Again, it should be noted that the
negative effects of age were found on students’ demonstration of social-emotional skills on a
contrived task in a testing setting, as opposed to daily application of skills in the classroom.
Neither Durlak et al. (2011) or Payton and colleagues (2008) reported a moderating
effect of gender relating to SEL intervention. A review of the broader social-emotional
competence literature revealed, however, that several published studies found that girls
demonstrate more positive emotion, emotional regulation, emotion knowledge, and overall social
emotional competence than boys (Bosacki & Moore, 2004; Brown & Dunn, 1996; Else-Quest et
al., 2006; Garner & Waajid, 2012; Merrell, Cohn, & Tom, 2011). These social-emotional
advantages, however, were not consistently translated to girls realizing more benefit from
participation in SEL intervention compared to boys. For instance, SEL programs have been
found to be more effective for boys in decreasing indirect aggression, increasing prosocial
behavior, and interrupting a decline in social-emotional skills (Aber, Jones, Brown, Chaudry, &
Samples, 1998; Belfield, Nores, Barnett, & Schweinhart, 2006; Miller, Malone, & Dodge, 2010;
Muenning, Schweinhart, Montie, & Neidell, 2008; Frey, Nolen, Estrom, & Hirschstein, 2005).
Research suggests that, although girls may demonstrate better social-emotional competence than
boys, boys and girls do not similarly benefit from participation in SEL programming. It may be
the case that girls have less to gain from SEL intervention than boys, as they on average
demonstrate higher levels of social-emotional competence. Emerging research indicating that the

19

positive effects of SEL intervention are most pronounced for children with lower baseline
competencies seems to support this claim (e.g., Frey et al., 2005; Low et al., 2015).
Delivery Matters: Teacher vs. Non-School Personnel
Large-scale analysis and review of school-based SEL programs demonstrate that school
personnel can effectively implement SEL programs to improve student behavior, attitudes
toward school, and academic achievement (Durlak et al., 2011; Payton et al., 2008). In their
meta-analysis of universal SEL programs, Durlak and colleagues (2011) compared the
effectiveness of classroom-based intervention delivered by teachers versus non-school personnel
(e.g., researchers, community-based workers). Teacher-delivered SEL intervention was found to
result in positive mean effects in all six of the student outcomes categories: SEL skills (g = .62),
attitudes (g= .23), positive social behavior (g = .26), conduct problems (g = .20), emotional
distress (g = .25), and academic performance (g = .34). SEL interventions delivered by nonschool personnel, on the other hand, resulted in positive mean effects for only three of the
outcome measures: SEL skills (g = .87), attitudes (g= .14), and conduct problems (g = .17).
Payton et al. (2008) similarly compared the effectiveness of interventions’ delivery mode.
Results indicated even stronger main effects for teacher-delivered interventions than reported in
the Durlak et al., (2011) comparison, with teacher-delivered interventions again resulting in
positive mean effects for all outcome measures (SEL skills, g = .68; Attitudes, g = .24; Positive
social behavior, g = .27; Conduct problems, g = .21; Emotional distress, g = .23; Academic
performance, g = .43) while non-school personnel-delivered programs demonstrated
effectiveness for only two of the outcomes (SEL skills, g = .84; Conduct problems, g = .17). One
important finding to note in the two large-scale comparisons is that participation in classroombased SEL interventions resulted in significant improvements in academic performance only

20

when delivered by teachers as opposed to non-school personnel (Durlak et al., 2011). These
results provide strong evidence that, not only is it feasible for school staff to implement SEL
programs, teachers are more effective at delivering the programs than non-school personnel and
are the only implementers likely to generate improvements in student academic performance
(Durlak et al., 2011; Payton et al., 2008).
Evidence-Based SEL: Second Step
Among the many school-based SEL programs available, Second Step (Committee for
Children, 2012) is one of the most widely disseminated and nationally recognized. Several
research studies have evaluated the efficacy and effectiveness of Second Step on improving
elementary students’ social-emotional competence and reducing problem behaviors. One
seminal, large-scale, randomized controlled trial examined the effect of Second Step on secondand third-grade students’ (n=790) level of aggression and positive social behavior in six urban
schools (Grossman, 1997). Direct observation revealed a significant increase in prosocial
behavior and a significant reduction in physically violent behavior among students receiving
Second Step intervention. During the same time, students not participating in the program
showed increases in aggression at school and no appreciable changes in prosocial behavior.
These findings suggest that the skills students develop through participation in Second Step not
only improve observed prosocial behavior; the skills may also mitigate a normative increase in
observed aggressive behavior associated with age as students progress through the school year.
Change in teacher-rated aggression and prosocial behavior, on the other hand, indicated
no significant differences between intervention and control schools in targeted behaviors. Much
of the incongruence between teacher ratings and direct observation, however, may be attributed
to the way in which the direct behavior observation data were collected and coded. It is

21

important to note that observational data were collected in three different settings: classroom,
playground, and cafeteria. Significant differences in behavioral changes between intervention
and control schools were only observed on the playground and in the cafeteria; directly observed
classroom behaviors did not reveal significant differences. As teacher ratings of student behavior
are based primarily on classroom observations, direct observation (in classroom setting) and
teacher-ratings of student behavior were consistent; both sources indicated that students
receiving Second Step intervention did not demonstrate significant differences in prosocial and
aggressive behavior in the classroom setting compared to students not receiving the program.
Another important factor to consider when interpreting the results of the Grossman
(1997) study is the method used for coding prosocial behavior. Because observers had difficulty
distinguishing between prosocial and neutral behaviors in the field, the two types were collapsed
into one category (neutral/prosocial) for analysis and interpretation. As such, the reported gains
in prosocial behavior on the playground and in the cafeteria for students receiving Second Step
include an unknown proportion of neutral behavior. As neutral behavior is not defined in the
study report, it could be that reported increases in prosocial behavior included observation of
withdrawal, social isolation, disengagement, or passivity, all of which would not be considered a
developmental asset that would promote student success. Overall, these findings offer support for
Second Step’s effectiveness in reducing aggressive behavior in school settings that are less
structured and more prone to problem behaviors (i.e., playground, cafeteria). Results did not
reveal, however, that Second Step was effective in promoting prosocial skills or reducing
problem behaviors in the classroom setting, a context of importance in the proposed study due to
the close link between student classroom behavior and school outcomes.

22

A second randomized controlled trial of Second Step involving second- through fourthgrade students (n = 1,253) in fifteen elementary schools examined program effects on student
social competence and antisocial behavior (Frey et al., 2005). Unlike Grossman’s (1997)
findings, Frey and colleagues found significant differences in teacher-rated social behavior
among students participating in Second Step and control students. Students in the Second Step
group showed significant gains in social competence relative to students in the control group
(partial h2 = .20). Teacher ratings of antisocial behavior also significantly decreased for students
in the intervention group while antisocial behavior increased for control students during the same
time. Decreases in antisocial behavior were particularly meaningful for students rated by their
teachers as highly antisocial at baseline (one or more SD above the mean; h2 = .25). Intervention
students initially rated as lower in antisocial behavior (less than one SD above the mean) also
saw greater reductions in antisocial behavior compared to the control group, although these
differences were not as pronounced as students rated as highly antisocial (h2 = .17). Like the
observational results in the Grossman (1997) study, these findings suggest that the skills students
develop through participation in Second Step not only improve observed prosocial behavior; the
skills may also mitigate a normative increase in observed aggressive behavior associated with
age and the progression of the school year.
In addition to teacher ratings of student social behavior, Frey et al. (2005) used direct
observation during a structured conflict to measure student social behavior. The conflict
situations used in the study were designed to elicit competition or cooperation over the
distribution of resources. Students receiving Second Step were less likely to need adult assistance
to negotiate a resolution with peers, requiring 41% fewer adult interventions than students in the
control group. Second Step participants also demonstrated significantly less aggression when

23

negotiating for prizes than students in the control group (h2 = .14). Program effects were not
observed, however, for other social behaviors such as cooperation, motivation, or subtle coercive
behavior (e.g., “power plays”).
Overall, findings from the study are promising and suggest that Second Step can promote
social-emotional competence and reduce problem behavior in the classroom as rated by teachers.
Although observational results demonstrate potential program benefits, observed improvements
in student social competence occurred outside of an authentic classroom setting and are based on
behavioral response to a contrived social problem. As with Grossman’s (1997) findings
suggesting Second Step’s positive effects on observed playground and cafeteria behavior, it is
difficult to predict how the observed increases in social competence during a structured conflict
would transfer to the real world setting of the classroom environment. These results also align
with large-scale review of school-based SEL interventions indicating robust effects in the
promotion of social-emotional skills yet, to a lesser degree, of effectiveness in increasing
prosocial behaviors in the classroom (Durlak et al., 2011; Payton et al., 2008).
Taken together, these results illustrate a prominent critique of tiered models of service
delivery set forth in the SEL literature, that social-emotional and academic programming often
represent disconnected systems and the failure to integrate programming compromises the
degree to which students benefit from instruction and intervention (Lane & Menzies, 2003;
McIntosh et al., 2006; Zins et al., 2004). The benefits of such an integrated approach are
supported by findings from large-scale reviews indicating that, compared to non-school
personnel-delivered interventions, teacher-delivered interventions are significantly more
effective across a wider range of student outcomes (Durlak et al., 2011; Payton et al., 2008).
Presumably, the stronger effects of teacher-delivered interventions might be explained by the

24

greater likelihood that intervention content and skills are introduced, reinforced, and integrated
throughout the school day in the actual classroom setting during authentic learning activities.
Non-school personnel-delivered interventions, on the other hand, are more likely to be
disconnected from the everyday functioning of the classroom and curriculum with fewer
opportunities to apply emerging skills to authentic settings and learning activities resulting in
weaker effects on behavior in the classroom setting.
A small-scale study investigated the effect of one school’s implementation of Second
Step on third- through fifth-grade students’ (n = 54) social behavior relative to same-grade peers
attending a comparison school (n = 33) and not receiving SEL intervention (Taub, 2001). Using
the School Social Behavior Scales (SSBS; Merrell, 1993), students in the Second Step school
were rated by teachers as significantly lower on the social competence scale than students in the
comparison school prior to intervention. However, one year following intervention, students at
both schools were rated similarly as a result of significant improvements in social-emotional
competence for students receiving the Second Step intervention. These results suggest that, in
addition to serving a preventative role by bolstering social-emotional competence for all
students, Second Step may also be effective in remediating skills for students starting the year
behind peers in social-emotional competence.
Teacher ratings also indicated a significant time by school effect on the SSBS antisocial
behavior scale attributed to the combined effects of a slight decrease in antisocial behavior at the
Second Step school and an increase in antisocial behavior at the comparison school. These results
are consistent with other studies indicating the mitigating effect of Second Step on age-related
increases in negative school behavior (Frey et al., 2005; Grossman, 1997). Although results from
teacher ratings of antisocial behavior revealed a statistically significant group by time interaction

25

effect in favor of intervention students from baseline to one-year follow up, this change was
considerably smaller and less predictable than change in social-emotional competence ratings.
The differential effect of the intervention on social competence and antisocial behavior is
consistent with other research findings, lending further support for the effectiveness of SEL
intervention in developing new prosocial skills and, to a lesser degree, eliminating antisocial
behaviors. The Taub (2001) study also included individual observation of four prosocial
behaviors in the classroom setting: engages appropriately with peers, follows directions from
adults, follows classroom rules, bothers/annoys other students. Results revealed statistically
significant treatment effects for only one of the observed behaviors at the one-year follow up.
Overall, intervention students demonstrated greater improvement in follows directions from
adults from baseline to one-year follow up compared to control students. These results are
consistent with other Second Step studies that found weak or nonexistent effects in direct
observation of students’ classroom behavior (Frey et al., 2005; Low et al., 2015). It could be that
change in observed behaviors is more difficult to demonstrate due to the limited time observers
spend in the classroom in contrast to teachers who have daily interaction with students over
many months.
A recent large-scale, randomized-control trial investigated whether early elementary
students (n = 7,300) receiving the updated version of Second Step (Committee for Children,
2012) performed better on social and behavioral outcomes than students in comparison schools
(Low, Cook, Smolkowski, & Buntain-Ricklefs, 2015). Sixty-one schools within two different
states were matched on free and reduced lunch and percent of non-White students and included
in the study. Outcome measures included teacher ratings of student behavior using the Devereux
Student Strengths Assessment – Second Step Edition (DESSA-SSE; LeBuffe, Naglieri, &

26

Shapiro, 2011), teacher ratings of behavior using the Strengths Difficulties Questionnaire (SDQ;
Goodman, 2001), and direct observation of disruptive behavior in the classroom. In order to
evaluate the effectiveness of Second Step under real-world educational conditions, the study
maintained little research control. The intervention was delivered by classroom teachers and
relied on limited training and research involvement. Results suggest significant group differences
from pretest to posttest on only two of the eleven outcomes tested. Students in Second Step
schools showed greater improvements in teacher-rated skills for learning on the DESSA-SSE (g
= .11) and greater reduction in SDQ emotional problems (g = -.10) as compared to students in
control schools over a one-year period. Consistent with other studies, program effects on direct
observation of student disruptive behavior in the classroom did not reach statistical significance
(Frey et al., 2005; Taub, 2001).
The researchers also tested whether students’ response to Second Step varied by baseline
social-emotional competence scores. Tests of moderation indicated that Second Step produced
significant improvements in eight of the eleven social-emotional and behavioral outcomes for
students who started the school year with skill deficits relative to their peers. Specifically,
students demonstrating higher levels of problem behaviors at baseline showed a significant
decrease in teacher-rated conduct problems, hyperactivity, and peer problems following
intervention. Students demonstrating deficit social-emotional skills prior to intervention also
realized significant improvements in skills for learning, emotional management, problem
solving, and overall social-emotional competence following participation in Second Step
intervention. However, these effects were specific to children who were generally in the lower
half of their peers (50th percentile). Echoing Taub’s (2001) findings, results suggest that the
benefits of Second Step are most pronounced for children with lower baseline competencies at

27

the beginning of the school year. This pattern suggests that the expectation of main effects across
all students may be unrealistic and that a certain subgroup of participants may not demonstrate
the benefits of SEL programming, as they already possess a high level of social-emotional
competence at baseline. Analysis that relies solely on testing for main effects runs the risk of
masking differential effectiveness, suggesting the importance of moderation analyses to better
understand the types of students who are most responsive to school-based SEL intervention such
as Second Step. As with skills in the academic domain, it can be expected that students will begin
the school year with variable levels of social-emotional skills and will likely respond differently
to intervention. This pattern lends support for the integration of universal SEL screening to
identify those students in need of additional social-emotional support and to assess student
response to universal SEL intervention.
Overall, the research on Second Step reveals inconsistent improvements in student
outcomes that vary widely depending on measurement procedures, outcomes of interest, and
student characteristics. The most consistent pattern across studies indicates that the benefits of
Second Step are most pronounced for children beginning the school year with lower baseline
social-emotional competencies, suggesting that all children do not benefit similarly from
participation in the Second Step program. Differential effects of gender, grade, or ethnicity,
however, were not indicated.
Measuring Social Emotional Competence in School
Assessment of social-emotional competence is necessary in order for schools to make
data-based decisions regarding the effectiveness of SEL programming for individual students,
classrooms, and the student body as a whole. Assessment measures and procedures also play an
important role in the early identification of students in need of additional social-emotional

28

support and to determine the level or intensity of programming will most appropriately address
the identified needs (Merrell, Juskelis, Tran, & Buchanan, 2008). The development and
widespread dissemination of valid and reliable tools to support the school-based screening,
assessment, and progress monitoring of social and emotional competence, however, lags behind
assessment in academic domains. Often educators have been left with little choice but to rely on
traditional tools and procedures that are not necessarily aligned with current SEL theory and best
practice. For example, such misaligned assessment methods might include lengthy rating scales
designed for diagnostic purposes, office discipline referrals or school suspensions/expulsions that
focus on severe levels of maladaptive behavior, and teacher anecdotal reports that lack a
quantifiable and objective base from which to make defensible date-driven decisions. This
mismatch between social-emotional and behavioral assessment practices and the current theory
and practices that drive evidence-based prevention programs presents a significant barrier to the
transportability and effectiveness of SEL programming in schools.
In response to a growing interest in school-based SEL, a variety of packaged curricula
have been developed to support development of students’ social-emotional competence.
Unfortunately, the development of SEL curricula has outpaced the development of assessment
tools to measure social-emotional competence (Merrell & Gueldner, 2012). Amidst this
landscape of newly developed SEL programs, each with varying degrees of empirical evidence,
the challenges associated with the transportability and sustainability of evidence-based programs
in the real world context of classrooms and schools is well documented. Effective program
implementation within a multi-tiered framework is seen as a complex and iterative process that
relies heavily on a data-driven decision-making system to ensure that the system is meeting the
needs of all the students that it serves (Merrell & Gueldner, 2012). Until recently, a lack of

29

normed and validated social-emotional assessment tools posed a significant barrier to the
advancement of a multi-tiered framework for school mental health and SEL programming (Nese
et al., 2012; Tilly, 2008). Fortunately, several school-based assessments have recently been
developed and are beginning to make their way into schools and multi-tiered systems of program
delivery (e. g., Merrell et al., 2011; LeBuffe, Ross, Fleming, & Naglieri, 2013).
Strengths-Based Assessment
The importance of identifying and building upon strengths has been an important feature
of many educational models (Merrell, Ervin, & Peacock, 2012; Tilly, 2008). Yet, socialemotional assessment practices have traditionally focused on the identification of problems
rather than competencies and strengths (Merrell & Gueldner, 2010). The dual-factor model of
mental health posits that wellness is comprised of more than the mere absence of pathology, and
that students who demonstrate complete mental health (the absence of significant problem
symptoms and the presence of protective indicators) fare better on a variety of academic and life
outcomes (Adelman & Taylor, 2003; Suldo & Shaffer, 2008). Particularly relevant to strengthbased assessment is the notion that assessment informs intervention aimed at increasing students’
competencies (Batsche et al., 2008). Several features of strength-based assessment align with the
theoretical and empirical foundation that guides the integration of SEL programming into the
everyday practices and routines of schools. For example, strength-based assessment provides
information about students’ social-emotional competencies and skills, holding particular promise
for the evaluation of SEL interventions that target increases in skills and competencies (Batsche
et al, 2008). Strengths-based measures also focus on behaviors and skills associated with
resilience and have been shown to be predictive of later developmental outcomes (Hjemdal,
Friborg, Stiles, Rosenvinge, & Martinussen, 2006).

30

Teacher Ratings of Social and Academic Behavior
Although behavior-rating scales are one of the most commonly used measures of socialemotional behavior, such measures were traditionally developed for diagnostic purposes (i.e.,
measuring existing symptoms against a diagnosable disorder) rather than for identifying future
risk or response to intervention (Albers, Glover, & Kratochwill, 2007). In particular, behavior
rating scales have generally been comprised of negative items that provide limited information
about desirable behaviors and assets and even less utility when looking to monitor student
progress toward acquiring skills and competencies (Hosp, Howell, & Hosp, 2003). Social,
emotional, and behavioral assessment practices stand to benefit from the inclusion of positive
assets and other indicators of wellness. For social-emotional assessment practices to positively
affect students and educational systems, these assessments need to be integrally linked to the
creation and implementation of effective interventions.
One approach to social-emotional assessment that has generated increasing practical and
empirical support is the use of teacher ratings of student competence on a common set of socialemotional or behavioral criteria. Strengths-based teacher ratings of student social-emotional
competence have been found to be effective and represents an improvement over traditional
processes that rely on spontaneous referral of students demonstrating behavioral or socialemotional problems by general education teachers (Severson, Walker, Hope-Doolittle,
Kratochwill, & Gresham, 2007). The Devereux Student Strengths Assessment – Second Step
Edition (DESSA-SSE; LeBuffe et al., 2011) represents one newly developed strengths-based
measure that is integrally linked to SEL intervention. The DESSA-SSE is a 36-item,
standardized, norm-referenced behavior rating scale that assesses the social-emotional
competencies that serve as protective factors for children in kindergarten through the eighth

31

grade. The DESSA-SSE is organized into four conceptually derived scales that correspond to
skills taught in the Second Step program. Along with the overall social-emotional competence
composite (SEC; n = 36, α = 0.98), the DESSA-SSE scales include skills for learning (n = 9, α =
0.95), empathy (n = 9, α = 0.95), emotion management (n = 9, α = 0.91), and problem solving (n
= 9, α = .94). Given its strong psychometric characteristics, focus on student strengths,
reasonable demands on time, and integral connection to specific SEL learning objectives, the
DESSA-SSE represents a promising example of the shift in social-emotional assessment and the
potential that such a tool might have in supporting student success.
School-Based Universal Screening
School-based universal screening has been proposed as one method for early
identification of both academic (VanDerHeyden, Witt, & Gilbertson, 2007) and social-emotional
(Levitt, Saka, Romanelli, & Hoagwood, 2007) needs. The use of such procedures in the delivery
of prevention and intervention services in schools has been supported by federal legislation (e.g.,
No Child Left Behind Act of 2001, Individuals with Disabilities Education Act of 2004) and
endorsed by numerous professional organizations representing a variety of educational fields
(e.g., President's Commission on Excellence in Special Education, the Council for Exceptional
Children, the National Association of School Psychologists, the American Psychological
Association, the National Association of State Directors of Special Education; Cook, Volpe, &
Livanis, 2010).
Over the past two decades, the use of school-based universal screening for academic
difficulty as part of multi-tiered model of service delivery, particularly in early literacy
development, has become increasingly commonplace in elementary schools (Dowdy, Ritchey,
and Kamphaus, 2010). For example, the Dynamic Indicators of Basic Early Literacy Skills

32

(DIBELS; Good and Kaminski, 2003) is commonly used to screen all primary elementary school
students for early literacy problems and to monitor the development of early literacy skills
toward benchmark goals. The practice of systematic and universal screening for social and
emotional needs is rare in comparison to its prevalence in the academic domain, with less than
2% of schools estimated to routinely use such procedures to identify social-emotional needs and
monitor development in this area (Romer and McIntosh, 2005). Many factors are posited to
contribute to the lack of universal screening for social-emotional needs in schools. A key
consideration is the lack of reliable, cost-effective tools that are practical for use in schools and
among school professionals (Dowdy et al., 2010; Romer and McIntosh, 2005). To date, there are
few short-form versions of strength-based measures well-suited for universal screening (e.g.,
Merrell et al., 2011; Naglieri, LeBuffe, & Shapiro, 2011; Prince-Embury, 2007) and research is
needed to determine the utility of existing measures for universal screening purposes.
A myriad of considerations beyond the technical adequacy of an SEL screener affects a
school’s decision to adopt universal SEL screening procedures. One of the key issues in
developing universal screening procedures is the need for brief, easy to use, and relatively
inexpensive tools. The ease with which the screening procedures can be integrated into existing
models of educational programming is also an important consideration when looking to enhance
a system’s capacity to make informed educational decisions within a data-driven, multi-tiered
model (Weist, Rubin, Moore, Adelsheim, & Wrobel, 2007). Other pragmatic issues include the
length of the assessments, monetary cost, and disruption to classroom functioning and instruction
(Flanagan, Bierman, & Kam, 2003).
The strength-based Devereux Student Strengths Assessment-mini (DESSA-mini; Naglieri
et al., 2011) represents one promising SEL universal screening measure that has been recently

33

developed for use in schools. The DESSA-mini has demonstrated high internal consistency (α
= .92), parallel form reliability (α = .90-.93), test-retest reliability (α = .88 - .94), and interrater
reliability (α = .70-.81; Naglieri et al., 2011). The brief measure (8 items) requires limited
investment of time; a teacher can feasibly rate all students in a class during one 50-minute
planning period. The DESSA-mini produces a total scale score that is qualified by one of three
descriptive levels (strength, typical, need for instruction). The DESSA-mini generates
assessment data that are designed for ease of interpretation and integration with standard
academic methods that categorize students into level of risk within a tiered model of educational
programming. Given its psychometric soundness, ease of interpretation, progress monitoring
potential, and alignment with data-based decision making that occurs within a tiered model of
educational programming, the DESSA-mini is a promising tool to address the need for effective
and efficient universal screening for social and emotional development in the school setting.
Research Questions & Hypotheses
The purpose of this study was to take initial steps in exploring the potential benefits of
integrating school-based SEL intervention into a tiered approach to educational programming.
As such, this project evaluated the effects of a universal SEL intervention on elementary
students' social-emotional competence and academic performance. This study also aimed to
better understand for whom SEL programming produced immediate and secondary effects by
examining the moderating effect of student gender and baseline level of learning risk on response
to SEL intervention. The study addressed the following research questions and hypotheses:
RQ1: Controlling for baseline social-emotional competence, do students receiving
Second Step intervention show greater improvement in social-emotional competence
(skills for learning, empathy, emotion management, problem solving) than students in a

34

wait-list comparison group? If so, do improvements in social-emotional competence
depend on student level of learning risk at baseline? Are intervention effects moderated
by gender?
Hypothesis 1a: It was hypothesized that students receiving Second Step would
show greater improvements in the combination of social-emotional competencies
than students in a wait-list comparison group. This hypothesis is consistent with
prior research on the immediate positive effects of school-based SEL programming
and the anticipated outcomes produced by Second Step. This hypothesis was
predicated on the idea that social-emotional competence can be taught and
enhanced through integration of explicit SEL instruction with everyday classroom
practices and academic curriculum.
Hypothesis 1b: Students demonstrating a higher level of learning risk at baseline
would show greater improvements in social-emotional competence following
intervention than students demonstrating lower learning risk. This hypothesis was
based on prior research suggesting that the benefits of Second Step are greatest for
students with lower baseline competencies. Analysis tested the interdependence of
social-emotional competence and academic success and whether the level of
learning risk at baseline moderated the effects of the intervention. It was
hypothesized that students demonstrating higher levels of learner risk at baseline
would show greater benefits from Second Step participation.
Hypothesis 1c: The effects of Second Step would be moderated by gender, with
boys benefiting more from participation in SEL programming than girls. This
hypothesis is consistent with prior research suggesting that girls demonstrate higher

35

levels of social-emotional competence than boys and that the positive effects of
SEL intervention are most pronounced for students with lower baseline
competencies.
RQ2: Controlling for baseline achievement, do students receiving Second Step
intervention show greater improvement in academic achievement than students in a waitlist comparison group? If so, do improvements in reading achievement depend on student
level of learning risk at baseline? Are intervention effects moderated by gender?
Hypothesis 2a: Students receiving Second Step intervention would show greater
improvement in reading and math achievement than students in a wait-list
comparison group. This hypothesis is consistent with prior research on the positive
effects of teacher-delivered, school-based SEL programming on academic
achievement. As illustrated in Figure 2, this hypothesis was predicated on the idea
that social-emotional competence and academic achievement are interconnected
and SEL programming not only promotes social-emotional skills, it also improves
academic outcomes.
Hypothesis 2b: Students demonstrating a higher level of learning risk at baseline
would show greater improvements in reading achievement than students
demonstrating lower learning risk. This hypothesis is consistent with prior research
suggesting that the positive effects of SEL intervention are most pronounced for
students with lower baseline competencies. The hypothesis tested the
interdependence of social-emotional competence and academic success and
whether the level of learning risk at baseline moderated the effects of the
intervention on reading achievement.

36

Hypothesis 2c: The effects of Second Step on reading and math achievement will
be moderated by gender, with boys benefiting more from participation in SEL
programming than girls. This hypothesis is consistent with prior research
suggesting that girls demonstrate higher levels of social-emotional competence than
boys and that the positive effects of SEL intervention are most pronounced for
students with lower baseline competencies.

37

CHAPTER 3
METHOD
The purpose of this study was to evaluate the potential benefits of integrating SEL
primary prevention with a tiered approach to educational programming. The study involved a
secondary analysis of extant data to evaluate the effect of one school’s implementation of
universal SEL programming on elementary students’ social-emotional competence and academic
achievement. This chapter includes a description of the research design, characteristics of the
existing dataset, and analysis procedures. Additionally, an overview of the Second Step
curriculum is provided along with steps the school followed in delivery of the intervention and
data collection.
Design
This study is a retrospective investigation of one school’s efforts to incorporate SEL
primary prevention into an existing three-tiered model of academic programming during the
2014-2015 academic year. A nonequivalent group design with a pretest and posttest was used to
study the effects of universal Second Step programming on student social-emotional competence
and academic achievement. The Intervention condition represented a comprehensive approach to
educational programming that combined social-emotional and academic interventions into a
Multi-Tiered System of Support (MTSS) while the Wait-List Comparison condition represented
a standard, academically focused Response to Intervention (RTI) framework. Students assigned
to Wait-List Comparison classrooms received the Second Step curriculum during the second-half
of the 2014-2015 academic year, after the Intervention group completed the program.
This study relied on existing demographic and student outcome data provided by the
school district (program documentation, district database,) as well as public records of school

38

information (Public Schools Review, LLC, n.d.). The existing dataset included teacher ratings of
student social-emotional competence collected at baseline and again following intervention
implementation. Academic data collected during fall and winter benchmarking windows were
also included in the school-provided dataset. A Determination of Whether an Activity is Human
Subject Research Form was completed and submitted to the Michigan State University
Institutional Review Board (IRB) in February 2016. The board determined that the federal
regulations for the protection of human subjects did not apply to this research using existing data
and IRB approval was not needed to proceed. IRB documentation is included in the Appendix.
As the participating school district did not have a separate institutional review board, written
authorization to use existing data for this study was sought and granted from the school
administration in March 2016. After authorization was granted, the school research liaison
consolidated data associated with the variables of interest for this study, stripped the dataset of
all identifying information (e.g., name, birthdate), and provided the researcher with a deidentified dataset in Microsoft Excel format. School-provided demographic data included student
gender, grade level, ethnicity, and classroom assignment (Intervention or Wait-List Comparison).
Setting
This study involved secondary analysis of existing data from all students enrolled in one
public school academy (PSA; commonly referred to as charter school) serving kindergarten
through fifth-grade students. At the time of intervention implementation and data collection, the
participating school had been in existence for 20 years under a charter contract authorized by a
local public university. The school serves a neighborhood identified by city organizers as one of
four at-risk neighborhoods targeted for a multi-year collaborative initiative. Determination of
neighborhood risk was based on analysis of numerous early childhood indicators (e.g., maternal

39

and child health index, parent education, children living in poverty). Children residing in these
identified neighborhoods were found to be at significant risk for negative school and health
outcomes. At the time of data collection, the schools’ population represented a demographic
breakdown of 90.0% African American, 8.4% Hispanic, 1.1% White, and 0.5% American
Indian. In addition, over 90% of students met the federal criteria of economic disadvantage as
indicated by participation in the Free and Reduced Price School Meals Program. According to
2014-15 Michigan Student Test of Educational Progress (M-STEP) results, 19.1% of third
graders, 11.7% of fourth graders, and 15.9% of fifth graders demonstrated proficiency in reading
(kindergarten through second-grade students are not assessed on this measure; Michigan
Department of Education).
Prior to the 2014-2015 academic year, the participating school approached instruction
and service delivery from a standard, academic Response to Intervention (RTI) framework.
Programming was designed to prevent academic difficulties through universal reading and math
screening, regular monitoring of student academic progress, and a tiered system of increasingly
intensive academic interventions. Under this application of the RTI model, a systematic
framework for addressing the social and emotional needs of all students was not in place. Across
the 2014-2015 academic year, the participating district implemented the more comprehensive,
Multi-Tiered System of Supports (MTSS) by integrating universal SEL intervention with
existing educational programming. Because intervention materials were limited, implementation
of the SEL program took place in two phases across the 2014/2015 academic year. This study
involved secondary analysis of existing school-collected data from the first phase of
implementation.

40

Classrooms and Teachers
During the 2014-2015 academic year, the participating school housed eighteen
kindergarten through 5th-grade classrooms (three classrooms at each grade level), each staffed by
a fulltime certified teacher. Administration and teaching staff assigned students to classrooms
prior to the start of the school year per the school’s standard practices (e.g., prior teacher
recommendation, student strengths/needs). In August of 2014, after class rosters were finalized
and distributed to teachers, school administration assigned all 18 classrooms to one of two
conditions: Intervention or Wait-List Comparison. Teacher condition assignment was based on
administrator recommendation and teacher willingness to be the first to implement the SEL
program in his or her classroom (October – January). Initially, two classrooms from each grade
level were assigned to the Intervention condition (n = 12) and one classroom from each grade
level was assigned to the Wait-List Comparison condition (n = 6). However, during the fall data
collection window, one fourth-grade Intervention classroom teacher resigned and a substitute
teacher took over the classroom. The school principal decided that the classroom would receive
the SEL program later in the school year, during the second phase of implementation. Although
fall social-emotional competence data were eventually collected for these students after a longterm substitute teacher was in place for one month, intervention implementation had already
begun and baseline data from that classroom were not included in the proposed study. As a
result, data from 17 of the school’s 18 classrooms were included in this study. Table 1 presents
teacher gender, ethnicity, and grade level taught by condition.
During the first phase of implementation (October through January), the eleven
Intervention classrooms received SEL programming along with the existing tiered academic
programming. The Intervention condition, therefore, represented an integrated, Multi-tiered

41

System of Supports (MTSS) approach to service delivery. The Wait-List Comparison condition
represented the existing, academically focused RTI approach to educational programming.
Students assigned to Wait-List Comparison classrooms received the SEL intervention later in the
school year (February through May) during the second phase of implementation.
Table 1
Grade Level, Gender, and Ethnicity of Teachers (N = 17) by Intervention Group
Characteristic

Second Step Teachers
n (%)

Comparison Teachers
n (%)

Grade Level
Kindergarten
2 (18%)
1 (17%)
First
2 (18%)
1 (17%)
Second
2 (18%)
1 (17%)
Third
2 (18%)
1 (17%)
Fourth
1 (9%)
1 (17%)
Fifth
2 (18%)
1 (17%)
Total
11 (100%)
6 (100%)
Gender
Female
11 (100%)
4 (67%)
Male
0 (0%)
2 (33%)
Ethnicity
White
9 (82%)
2 (33%)
African American
2 (18%)
3 (50%)
Hispanic
0 (0%)
1 (17%)
Note. Totals of percentages are not 100 for every characteristic because of rounding.

Total Teachers
n (%)
3 (18%)
3 (18%)
3 (18%)
3 (18%)
2 (12%)
3 (18%)
17 (100%)
15 (88%)
2 (12%)
11 (65%)
5 (29%)
1 (6%)

Students
In total, 395 kindergarten through fifth-grade students were enrolled in the participating
charter school at the start of the 2014-2015 school year. Because baseline data for one classroom
were collected outside of the data collection window and ineligible for use in the study, data
from 369 students were included in this study. In total, eleven classrooms were assigned to the
Intervention condition and six classrooms were assigned to the Wait-List Comparison condition.
Table 2 summarizes the characteristics of the two intervention groups by grade level, gender, and
ethnicity.

42

Table 2
Grade, Gender, and Ethnicity of Students (N = 369) by Intervention Group
Characteristic

Second Step Students
n (%)

Comparison Students
n (%)

Grade
Kindergarten
46 (19%)
25 (20%)
First
48 (20%)
22 (17%)
Second
42 (17%)
21 (17%)
Third
43 (18%)
22 (17%)
Fourth
19 (8%)
16 (13%)
Fifth
44 (18%)
21 (17%)
Total
242 (100%)
127 (100%)
Gender
Female
127 (52%)
60 (47%)
Male
115 (48%)
67 (53%)
Ethnicity
African American
214 (88%)
118 (93%)
Hispanic
23 (10%)
8 (6%)
White
4 (2%)
0 (0%)
American Indian
1 (< 1%)
1 (1%)
Note. Totals of percentages are not 100 for every characteristic because of rounding.

Total Students
n (%)
71 (19%)
70 (19%)
63 (17%)
65 (18%)
35 (9%)
65 (18%)
369 (100%)
187 (51%)
182 (49%)
332 (90%)
31 (8%)
4 (1%)
2 (< 1%)

Statistical power analysis was used to determine if the available sample size from the
existing dataset was sufficient to detect the expected effect size for this study. G*Power 3 (Faul,
Erdfelder, Lang, & Buchner, 2007), a stand-alone power analysis program available free of
charge via the Internet (http://www.gpower.hhu.de/), was used to conduct this analysis. To
determine the likelihood that any observed differences between group means occurred randomly
or as a result of the intervention, the criterion for significance (alpha level of Type I error rate)
was set at 0.05. The test was one-tailed; meaning only an effect in the expected direction was
interpreted. With the existing sample size of 369 (Intervention = 242, Wait-List Comparison =
127), this study had a power of 80% to yield an effect size of .31 (minimum sample size = 358;
Intervention = 231, Wait-List Comparison = 126). It was determined that an effect size of this
magnitude was comparable to other studies in the field of SEL research (Durlak et al., 2011,
Payton et al., 2008) and that the existing dataset was sufficient to address the study’s primary
research questions. The primary goal of this study was to test the hypothesis that students
43

receiving Second Step programming would show greater improvement in social-emotional
competence and academic achievement than students assigned to wait-list comparison
classrooms.
Second Step Training and Delivery
During the 2014/15 academic year, the participating school district integrated universal
Second Step with its existing academically focused tiered model of assessment and support.
Second Step: K-5 4th Edition (Committee for Children, 2012) is a universal SEL program
designed to promote social competence and reduce social-emotional problems in elementary
students. The program builds on cognitive behavioral intervention models, predicated on the idea
that thoughts affect people’s social interactions (Crick & Dodge, 1994). The content of the
program is broken into four primary areas of instruction: (a) skills for learning; (b) empathy; (c)
emotion management; and (d) problem solving. Social-emotional skills are taught using direct,
explicit instruction. The program is designed to be user-friendly and implemented by various
school professionals in a variety of settings.
Teachers assigned to the Intervention condition delivered Second Step during the first
phase of program implementation (October through January) and teachers assigned to the WaitList Comparison condition delivered the intervention during the second phase of implementation
(February through May). At the time of classroom assignment, Intervention teachers were issued
a Second Step curriculum kit, a training schedule, and a lesson delivery timeline. The district
sponsored and the school social worker facilitated a two-hour Second-Step training for
Intervention teachers in September 2014. Wait-List Comparison teachers participated in a similar
training in January 2015, following completion of the first phase of intervention implementation
and data collection. The Second Step training focused on program core components and best

44

practices for teaching and reinforcing the lesson content. The web-based training included a
program overview, information about the links between social emotional learning and
academics, curriculum exploration, and program videos. The school social worker met with
Intervention classroom teachers one week after the Second Step training session to answer
questions, review materials, and reiterate the implementation schedule and expectations.

Intervention	Training	- September	2014
Intervention	teachers	- Second	Step	training

Wait-List	teachers	- standard	PD

Baseline	Data	Collection	- Early	October	2014
Intervention	teachers	completed	rating	scales

Wait-List	teachers	completed	rating	scales

Phase	I	Implementation	- October	2014	through	January	2015
Intervention	teachers	delivered	Second	Step

Wait-List	teachers	delivered	standard	programming

Post-Intervention	Data	Collection	- Early	February	2015	
Intervention	teachers	completed	rating	scales

Wait-List	teachers	completed	rating	scales

Figure 3. Second Step implementation and data collection schedule.
Figure 3 provides a summary of the intervention implementation and data collection
timeline. Trained Intervention classroom teachers delivered the twenty-two Second Step lessons
in 30- to 40-minute classroom sessions twice per week from October through January for a total
of approximately eleven weeks of instruction. During phase one of implementation (October
through January), students in Wait-List Comparison classrooms participated in the school’s
standard, academically focused RTI programming that did not include SEL instruction. In earlyFebruary, following completion of phase one of SEL program implementation and data
45

collection, Wait-List Comparison teachers participated in the Second Step web-based training.
Following training, Wait-List Comparison teachers delivered the Second Step curriculum to their
students during the second phase of implementation (February through May). It should be noted,
however, that the focus of the proposed study is on the first phase of SEL program
implementation (October through January).
Measures
To address the primary research questions, this study involved analysis of existing
student data from two areas of development: (1) social-emotional competence and (2) academic
achievement. A description of the outcome and screening measures used in compiling the
existing dataset used in this study follows.
Social-Emotional Competence Outcome Measure
The Devereux Student Strengths Assessment – Second Step Edition (DESSA-SSE;
LeBuffe et al, 2011) behavior scale, part of the Devereux Center for Resilient Children (DCRC)
assessment package, was included in the dataset considered in this study. All measures from the
DCRC assessment package are strength-based and designed to assess competencies shown to
serve as protective factors for children in kindergarten through eighth-grade. The 36-item
DESSA-SSE was derived from the lengthier 72-item Devereux Student Strengths Assessment
(DESSA; LeBuffe, Shapiro, & Naglieri, 2009) and developed based on the social-emotional
content covered in the Second Step curriculum (Committee for Children, 2012). The reduced
time requirement for completion and tighter alignment with Second Step were features that
persuaded the school to choose the DESSA-SSE over the full version of the DESSA as a
measure of social-emotional outcomes and student response to SEL intervention.

46

All teachers (Intervention and Wait-List Comparison) completed a web-based DESSASSE via the EVO Social/Emotional online assessment platform (Apperson, Cerritos, CA) for
each student in their classroom at pretest (October 2014) and again at posttest (January 2015). To
complete the scale for each student, the teacher read the stem: “During the past 4 weeks, how
often did the child …” and then rated questions using a 5-point Likert-type scale. The DESSASSE total raw scores range from 0 to 144, with high scores suggesting higher levels of social and
emotional competency. The web-based DESSA-SSE can be completed in less than 5 minutes per
student.
The DESSA-SSE generated four subscale scores and one composite score. The Social
Emotional Composite (SEC) provides an overall indication of the strength of the student’s social
and emotional competence. The four subscales include Skills for Learning (9 items), Empathy (9
items), Emotional Management (9 items), and Problem Solving (9 items). Raw scores on the
SEC and each subscale were converted to T-scores and provided an estimate of social and
emotional competence based on a comparison to national norms. The normative sample (n =
1,250) closely approximated the kindergarten through eighth-grade population of the United
States with respect to age, gender, geographic region of residence, race, ethnicity, and
socioeconomic status (LeBuffe et al., 2011). The authors of the DESSA rating scales recommend
that the SEC and subscale T-score value of 40 (one SD below the normative mean) be used as the
cutoff score that indicates a need for social-emotional instruction. Alternatively, the authors
suggest T-scores from 41 to 44 can be used to develop an “at-risk” category to align with tiered
frameworks for intervention (Naglieri et al., 2011). That is, DESSA scores at or above 45
categorized as low risk, sores of 41 to 44 categorized as moderate risk, and scores at or below 40
categorized as high risk.

47

Evidence reported in the technical manual indicates that the DESSA-SSE can be used
with confidence as a reliable assessment of students’ social-emotional competence (LeBuffe et
al., 2011). The internal reliability coefficients of the four DESSA-SSE subscales (.90 to .93) as
well as the composite (.98) exceed the commonly accepted .90 minimum (e.g., Salvia, Bolt, &
Ysseldyke, 2010). The measure has also demonstrated strong test-retest reliability, with
correlations coefficients ranging from .90 (Empathy) to .94 (Skills for Learning and Problem
Solving). As noted in the manual, this similarity in scores over time in the absence of targeted
social-emotional instruction is an important measurement characteristic to consider when
evaluating SEL intervention outcomes (as is the case in this research study). To date, only one
published study in the professional literature has utilized the DESSA-SSE as an outcome
measure for determining the effects of Second Step on student social-emotional competence
(Low et al., 2015). Low and colleagues reported reliability similar to those reported in the
DESSA-SSE technical manual, with coefficients for the four subscales ranging from .91
(Emotion Management subscale) to .98 for the composite score (SEC). The alpha coefficients
calculated for the current sample indicate similarly high internal reliability as that found in the
Low et al., study (2015) and in the technical manual. The following Cronbach alpha coefficients
obtained for the DESSA-SSE subscales and composite scores with the current sample also
indicated good internal consistency: Skills for Learning = .96, Empathy = .94, Emotion
Management = .93, Problem Solving = .95, and Social Emotional Composite = .98.
Evidence of criterion-related validity cited in the technical manual demonstrate that the
DESSA-SSE scores differentiated between groups of children with and without known socialemotional problems. Compared to typically developed children, students already identified as
having substantial social-emotional problems were rated significantly lower on the DESSA-SSE

48

(d-ratios ranging from .90 to 1.5 with large effect sizes). Although these findings suggest that the
DESSA-SSE can serve as a reliable and valid tool for measuring social-emotional outcomes,
there is a need to further examine the utility of the measure in the real-world context of a tiered
model of educational programming.
Academic Outcome Measure
Classroom teachers administered the Northwest Evaluation Association (NWEA)
Measures of Academic Progress (MAP; Northwest Evaluation Association, 2009) mathematics
and reading tests three times across the school year as part of the districtwide evaluation
program. MAP assessments are norm-referenced, computer-administered achievement tests
designed to assess content typically taught in specific grade bands and measure student academic
achievement and growth. The content of the math subtest is designed to assess number sense,
estimation and computation, geometry, algebra, measurement, and statistics and probability. The
reading subtest measures word recognition and vocabulary, reading comprehension, and
literature. The MAP technical manual provides evidence of strong reliability and concurrent
validity for both the math and reading assessments (Northwest Evaluation Association, 2009).
Scores on the MAP are reported as Rasch Unit (RIT) scores, percentiles, and analyses of
progress. The RIT scale is an equal interval scale with a range of 150 to 250. The RIT score
provides an estimate of student achievement based on the difficulty of individual items. Using
this scale, results of the MAP also can be reported as improvement scores, which represent the
number of RIT points gained by a student since the previous assessment and the extent to which
a student exceeds or falls short of average growth. The students’ Reading and Math RIT scores
from the fall and winter administration of the MAP were used for the analyses in this study.

49

Social-Emotional Screening Measure
The Devereux Student Strengths Assessment Mini (DESSA-mini; Naglieri et al., 2011),
another measure included in the DCRC assessment package, was used as a brief screener of
social-emotional competence in this study. Based on the 72-item Devereux Student Strengths
Assessment (DESSA; LeBuffe, Shapiro, & Naglieri, 2009), the web-based DESSA-mini is
comprised of four 8-item parallel forms designed to be a technically sound, universal screening
tools that can be completed in one to two minutes per student (Naglieri et al., 2011). As the
participating school was interested in exploring feasible methods for collecting and utilizing
universal SEL data within a tiered model, the DESSA-mini was chosen because of its minimal
time requirements and potential for frequent and reliable assessment of students’ overall socialemotional competence and response to intervention. All teachers (Intervention and Wait-List
Comparison) completed Form 1 of the web-based DESSA-mini for all students assigned to their
classroom during the two weeks prior to the Second Step implementation (October 2014). As
with the DESSA-SSE, teachers read a sentence stem: “During the past four weeks, how often did
the child …” and then rated questions using a 5-point Likert-type scale. The total raw score
generated by the DESSA-mini ranges from 0 to 32, with high scores suggesting higher levels of
social-emotional competence. The DESSA-mini yields a single T-score, the Social Emotional
Total (SET), which provides an estimate of the strength of the student’s overall social-emotional
competence based on a comparison to national norms. As with the DESSA-SSE, the DESSAmini SET scores can be used to categorize students’ as demonstrating low risk (at or above 45),
moderate risk (41 to 44), or high risk (at or below 40) of experiencing social-emotional
difficulty.

50

Reliability evidence reported in the DESSA-mini technical manual indicates that the
measure can be used with confidence as a screener for social and emotional competence
(Naglieri, LeBuffe, & Shapiro, 2011). Reported internal reliability coefficients for the four
DESSA-mini forms range from 0.91 (DESSA-mini 1) to 0.92 (DESSA-mini 3), exceeding
the .90 alpha standard for a total score suggested by Bracken (1987) and the .80 standard of
reliability recommended by Salvia, Bolt, and Ysseldyke (2010) for screening decisions. The
alpha coefficient calculated for the DESSA-mini (Form 1) with the current sample indicates
similarly high internal reliability (.95) as that reported in the technical manual. Reliability
evidence also indicates that each of the DESSA-mini forms has excellent test-retest reliability
(coefficients ranging from .88 to .94), providing increased confidence that observed differences
between pretest and posttest scores are less likely to be attributable to error variance or the
simple passage of time. The stability of a measurement tool across time has relevance in the
context of this study’s proposed tiered model of programming that relies heavily on a measure’s
ability to reliably detect changes in student competencies and determine response to intervention.
The DESSA-mini technical manual cites additional validity evidence, lending further
support for the utility of the measure as a universal measure of social and emotional competence.
The DESSA-mini has been found to be a strong predictor of overall competence scores obtained
on the full 72-item DESSA, with correlations ranging from .95 to .96 across the four DESSAmini forms. The DESSA-mini has also been found to differentiate groups of children with and
without known social-emotional problems, with large and significant differences between ratings
of the two groups (d-ratios ranging from 1.17 to 1.39). Additionally, the four DESSA-mini forms
and the full DESSA have been found to identify the same children as needing SEL instruction
94.5 to 95.3% of the time (based on a cutoff score of 40). Consistency between the DESSA-mini

51

and full DESSA in identification of students in need of instruction has also been shown to hold
across racial and ethnic groups, with the two measures demonstrating agreement 85.1% of the
time when identifying African American students, 80.7% of the time when identifying Hispanic
students, and 83.3% of the time when identifying White students. Given the high proportion of
African American students included in the current sample, the demonstrated consistency of the
DESSA-mini with the more comprehensive DESSA across diverse racial and ethnic groups lends
further support for the legitimacy of its use as a social emotional screener in this study.
The accuracy and efficiency of the DESSA-mini compare favorably with other
commonly used screening instruments in schools. For example, Kamphaus and Reynolds (2007)
reported that the 27-item Behavioral and Emotional Screening System (BESS) demonstrated
internal consistency reliability coefficients from .90 to .96 and test-retest reliability coefficients
from .80 to .91. Adjusted correlation coefficients obtained between the BESS and the more
comprehensive Behavior Assessment System for Children, Second Edition (BASC-2) ranged
from .62 (Internalizing Problems) to .90 (Behavioral Symptoms Index). Taken together,
reliability and validity data indicate preliminary evidence that the DESSA-mini is a promising
universal tool for screening student social-emotional competence in the school setting (Naglieri
et al., 2011). As with the DESSA-SSE, further examination of the utility of the DESSA-mini
within a tiered framework of educational programming is warranted.
Academic Screening Measure
Teachers administered the Dynamic Indicators of Basic Early Literacy Skills (DIBELS;
Good & Kaminiski, 1996) measures at three benchmarking windows (fall, winter, spring) across
the academic year. DIBELS are a set of procedures and tests for assessing the acquisition of
skills from kindergarten through sixth grade in each of the key literacy areas: phonemic

52

awareness, alphabetic principle, accuracy and fluency with connected text, reading
comprehension, and vocabulary. The screening measures were designed for use in identifying
children having trouble developing early literacy skills and are commonly used in schools
operating from an academically focused, RTI approach to literacy development. The DIBELS
program of research builds on assessment procedures from Curriculum-Based Measurement
(CBM; e.g., Deno & Fuchs, 1987) and General Outcome Measurement (GOM; Fuchs & Deno,
1991). Ongoing research since the late 1980s has documented DIBELS’s reliability, validity, and
sensitivity to change in literacy development. Additional information and technical reports can
be found on the Dynamic Management Group website at https://dibels.org.
The DIBELS Next assessment provides two types of scores at each benchmark
assessment period: a raw score for each individual measure and a composite score. The DIBELS
Composite Score is a combination of multiple scores and provides the best overall estimate of
students’ early literacy skills and reading proficiency. Each score is interpreted relative to
benchmark goals and cut points for risk to describe students’ level of reading development as at
or above benchmark, below benchmark, and well below benchmark. Fall DIBELS Next
Composite scores and associated reading performance levels were used as an academic screening
measure in this study.
Learning Risk Measure
The DESSA-mini and DIBELS categorical risk levels were dichotomized into two
categories (at risk, not at risk) and used to create a more comprehensive screening measure for
student level of learning risk. Specifically, student DESSA-mini descriptive qualifiers (low risk,
moderate risk, high risk) were converted to a dichotomized indicator of socio-emotional risk
(moderate or high risk = socio-emotionally at risk, low risk = socio-emotionally not at risk).

53

Similarly, student DIBELS descriptive qualifiers (at or above benchmark, below benchmark,
well below benchmark) were collapsed into a dichotomized indicator of academic risk (below
benchmark or well below benchmark = academically at risk, at or above benchmark =
academically not at risk). Three learning risk categories were derived from all possible
combinations of socio-emotional risk and academic risk: low learning risk, moderate learning
risk, high learning risk. Figure 4 illustrates the development of the three learning risk categories.

• DESSA-mini
• low	risk
• DIBELS
• at/above	
benchmark

• DESSA-mini
• low	risk
• DIBELS
• below/well-below	
benchmark		

Low	Risk

Moderate	
Risk

High	Risk

Moderate	
Risk

• DESSA-mini
• moderste/high	risk
• DIBELS
• below/well-below	
benchmark	

• DESSA-mini
• moderste/high	risk
• DIBELS
• at/above	
benchmark

Figure 4. Using DESSA-mini and DIBELS data to categorize student level of learning risk.
Data Collection
This study involved secondary analysis of preexisting data collected by the participating
school during the 2014-1015 academic year. Outcome data of interest included Pretest and
Posttest measurement of student social-emotional competence and academic achievement. All
teachers (Second Step and Wait-List Comparison) completed the web-based DESSA rating
scales prior to the first phase of universal Second Step implementation and again immediately

54

following completion of the first phase of the intervention. Academic achievement data were
collected during the MAP fall (October) and winter (January) benchmark windows.

55

CHAPTER 4
RESULTS
This chapter presents results of the secondary analysis of extant data to investigate one
elementary school’s efforts to integrate Second Step into an existing tiered model of academic
programming. Results from investigation of the relation between student level of learning risk
and intervention effectiveness are also presented. A quasi-experimental wait-list comparison
group design with pretest and posttest was used for this study. The dataset included teacher
ratings of student social-emotional competence and academic achievement data collected at
baseline and following intervention. Data were analyzed using IBM SPSS Statistics version 24.
Preliminary Analyses
To ensure the analysis plan would adequately address the primary research questions,
data were examined to identify potential sources of bias and ensure the assumptions underlying
the statistical methods were satisfied. Preliminary analyses relied on numerical and statistical
testing as well as visual inspection of data graphs and charts. Each variable was screened for
missing data, normality, and univariate and multivariate outliers for each of the two conditions
(Second Step, Wait-list comparison). The relations between key variables were also explored and
the social-emotional competence scales were tested for reliability within the two groups.
Missing Data
Data were examined for missing values on all outcome variables and covariates. The
percentage of missing values across the six variables varied between 0.0% and 10.8%. Out of
369 cases, 59 (16.0%) were missing data on at least one outcome variable or covariate, totaling
145 missing values out of a possible 2,214 (6.5%). Most instances of missingness (73%)
occurred on posttest measures and were attributable to 32 instances of student disenrollment

56

from the school following pretest data collection. The remaining 27 cases of incomplete data
were attributable to teacher failure to complete a social-emotional competence rating scale,
student nonparticipation in achievement testing due to absence, or clerical error. Attrition rates
did not significantly differ between Second Step and comparison groups nor were differential
attrition effects found on the dependent variables for gender or grade level. Table 3 presents the
means and standard deviations for dependent variables and covariates by group with percentage
of missing data for each measure at pretest and posttest.
Table 3
Means and Standard Deviations for Dependent Variables and Covariates by Intervention Group
with Percent Missing
Variable
Social-Emotional
Pretest
Posttest
Reading
Pretest
Posttest
Math
Pretest
Posttest

M

Intervention
SD

n

M

Comparison
SD

44.51
6.79

10.68
10.65

242
210

47.72
48.89

167.16
177.76

24.29
21.85

234
210

169.07
179.56

26.37
21.98

230
213

Total
SD

N

%
Missing

10.17
10.55

369
329

0.0
10.8

168.351
77.21

24.21
22.40

352
331

4.6
10.3

169.871
79.26

26.49
22.73

351
337

4.9
8.7

n

M

8.77
10.28

127
119

45.62
47.55

170.71
176.26

2.21
2.13

118
121

171.39
178.73

2.43
2.16

121
124

Multiple imputation was used to create and analyze five multiply imputed datasets to
improve the accuracy and statistical power of the analyses and results. Incomplete variables were
imputed under fully conditional specification (van Buuren, 2012). Calculations were done in
IBM SPSS using the Missing Values extension. Model parameters were estimated with multiple
regression applied to each imputed dataset separately. These estimates and their standard errors
were pooled using Rubin’s rules. For comparison, the analysis was also performed on the subset
of 310 complete cases.

57

Screening for Outliers
As part of the preliminary analysis, data were also screened for univariate and
multivariate outliers. Box plots and stem and leaf diagrams were generated and inspected to
identify univariate outliers in the data. Extreme outliers were not identified on the socialemotional competence scales or academic achievement scores. Mahalnobis Distance methods did
not detect multivariate outliers on the social-emotional competence scales.
Tests for Violations of Statistical Assumptions
Preliminary analyses included testing the assumptions associated with the statistical
techniques used in the study. This included testing for normality, linearity, homogeneity of
variances and covariances, homogeneity of regression slopes, reliability of covariate measures,
and multicollinearity. First, data were screened for univariate normality for the two groups using
measures of skewness, kurtosis, histograms, and the Kolmogorov-Smirnov statistic. Although
skewness and kurtosis were not found to be extreme, moderate departures from the normal
distribution were noted in each of the dependent variables. Visual inspection of the reading and
math score histograms indicated a somewhat flat distribution, suggesting a negative kurtosis
relative to normal distribution. Examination of the four social-emotional subscale score
histograms also indicated positively skewed distributions of scores. Kolmogorov-Smirnov test
statistics were also significant for all dependent variables, suggesting non-normal distributions of
scores. Skewness and kurtosis index values, however, for all dependent variables fell between
-.868 and .554, suggesting that the distributions did not substantially depart from normality.
Although preliminary analyses indicated a violation of the assumption of normal distribution at
the univariate level, Tabachnick and Fidell (2013) note that in relatively large samples (200 or
more cases), small deviations from normality can be deemed significant by the Kolmogorov-

58

Smirnov test and skewness will not “make a substantive difference in the analysis” (p. 80). In
such cases, Tabachnick and Fidel (2013) recommend against transformation as it often hinders
interpretation. As a partial check on multivariate normality, a matrix of bivariate scatterplots for
all pairs of social-emotional competence variables was constructed. Visual examination of the
plots indicated roughly linear relationships between pairs of social-emotional subscale scores,
lending support for the multivariate normality of the distribution of social-emotional competence
variables.
The assumption of homogeneity of variance and covariance across each group was also
tested. Insignificant results from Levene’s Test for Equality of Variances (p > .05) suggested that
the Second Step and comparison groups were relatively equal in degree of variance and
covariance on dependent variables. At the multivariate level, insignificant Box’s M Test results
similarly indicated satisfaction of the homogeneity of variance and covariance assumption.
Homogeneity of regression slopes were also evaluated to ensure that an interaction
between covariates and intervention group did not exist. To test this assumption, scatterplot
matrices between the dependent variables and covariates were examined. Visual inspection of
the scatterplots suggested minimally dissimilar slopes. This assumption was also assessed
statistically by checking whether there was a significant interaction between condition group and
the covariates. Results from tests of between-subjects interaction effects for group by baseline
social-emotional competence scores, F(1, 365)=1.012, p = .315, group by baseline reading
scores, F(1, 365)=1.704, p = .193, and group by baseline math scores, F(1, 365)=3.246, p = .072,
were not significant at an alpha level of .05. These nonsignificant results indicated that there was
no interaction between the covariates and group, suggesting that the assumption of homogeneity
of regression slopes was satisfied. The assumption of linearity holds that all pairs of dependent

59

variables and covariates demonstrate a linear relationship. Visual inspection of scatterplots for all
variables for each condition group indicated approximate linearity, suggesting that the
assumption of linearity was also satisfied.
Evidence of high internal consistency for all measures treated as covariates (MAP
reading composite, MAP math composite, DESSA-SSE social-emotional competence composite)
has been reported in the literature. Per the NWEA technical manual, the MAP reading and math
composite scales have demonstrated marginal reliabilities between .89 and .96 (Northwest
Evaluation Association, 2009). Similarly, the DESSA-SSE technical manual reports Cronbach
alpha coefficients between .96 and .98 for the social-emotional composite scale.
Table 4
Cronbach’s Alpha (a) Coefficients of Reliability for the DESSA Scales by Intervention Group
Scale

Second Step

Comparison

.98

.98

Baseline DESSA-SSE Scores
Social-emotional composite (SEC)

Skills for learning
.96
.97
Empathy
.94
.93
Emotion management
.93
.92
Problem solving
.95
.96
Baseline DESSA-Mini Scores
Social-emotional total (SET)
.95
.96
Note. DESSA-SSE = Devereux Student Strengths Assessment: Second Step Edition. DESSA-Mini = Devereux
Student Strengths Assessment: mini.

Although evidence of high reliability for the DESSA measures is reported in the
literature, it was necessary to confirm reliable use of the scales with the present sample.
Reliability analyses of the DESSA-SSE and DESSA-mini were carried out using Cronbach’s
alpha criterion (a). Results from these analyses are shown in Table 4. Cronbach’s alpha ranged
from .93 to .98 on the DESSA-SSE and DESSA-mini. Results indicated that the scales had
excellent reliability for both the Second Step and wait-list comparison groups (Tabachnick &
60

Fidell, 2013). Overall, the evidence suggests that the tools used to measure the covariates in the
present study were psychometrically sound and satisfied the assumption concerning the
reliability of the covariates.
The assumptions of MANCOVA include linearity between each pair of dependent
variables for each cell of the research design. To test this assumption, a scatterplot matrix was
plotted and examined. The matrix demonstrated a linear relationship between the dependent
variables, suggesting that the assumption was satisfied. MANCOVA also assumes a moderate
correlation (r < 0.9) between each dependent variable to ensure the absence of multicollinearity.
To detect relationships that were too strongly correlated, Pearson correlation coefficients were
calculated between the four DESSA-SSE subscale scores for each cell of the design. As shown in
Tables 5 and 6, correlations ranged from r = 0.86 to r = 0.95, indicating the presence of
multicollinearity and a violation of the assumption. To address the issue of multicollinearity, the
problem solving subscale was identified as demonstrating the highest correlation with the other
dependent variables and was removed in all subsequent analyses.
Table 5
Pearson Correlations Between Social-Emotional Competence Variables for Second Step Group
Social-Emotional Competence
Variables

Skills for
Empathy
Learning
Skills for Learning
1.00
Empathy
.87**
1.00
Emotion Mng
.87**
.87**
Problem Solving
.91**
.88**
** Correlation is significant at the 0.01 level (2-tailed)

61

Emotion
Mng

Problem
Solving

1.00
.94**

1.00

Table 6
Pearson Correlations Between Social-Emotional Competence Variables for Comparison Group
Social-Emotional Competence
Variables

Skills for
Empathy
Learning
Skills for Learning
1.00
Empathy
.86**
1.00
Emotion Mng
.89**
.92**
Problem Solving
.95**
.92**
** Correlation is significant at the 0.01 level (2-tailed)

Emotion
Mng

Problem
Solving

1.00
.94**

1.00

Equivalence of Intervention Groups at Baseline

Prior to proceeding with the primary analyses, the degree to which Second Step and
comparison groups were equivalent on demographic characteristics and baseline measures of
social-emotional competence and academic achievement was evaluated. Table 7 describes the
groups in terms of level of learning risk, grade, and gender. Although analysis of variance
(ANOVA) revealed statistically significant differences between the two condition groups on
baseline measurement of social-emotional competence, the actual mean differences were small
(Cohen’s d = .28) and favored the comparison classrooms. Inspection of the mean scores
indicated students in comparison classrooms were rated by their teachers as higher in socialemotional competence than students in Second Step classrooms. Statistically significant
differences in reading achievement and math achievement between the groups at baseline were
not found.

62

Table 7
Learning Risk, Gender, and Grade of Students (N = 369) by Intervention Group
Characteristic
Learning Risk
Low Risk
Moderate Risk
High Risk
Gender
Male
Female
Grade Level
Kindergarten
First
Second
Third
Fourth
Fifth

Second Step
n (%)

Wait-List Comparison
n (%)

Total
n (%)

57 (23.9%)
92 (38.7%)
89 (37.4%)

44 (34.9%)
51 (40.5%)
31 (24.6%)

101 (27.7%)
143 (39.3%)
120 (33.0%)

127 (52.5%)
115 (47.5%)

60 (47.2%)
67 (52.8%)

187 (50.7%)
182 (49.3%)

46 (19.0%)
48 (19.8%)
42 (17.4%)
43 (17.8%)
19 (7.9%)
44 (18.2%)

25 (19.7%)
22 (17.3%)
21 (16.5%)
22 (17.3%)
16 (12.6%)
21 (16.5%)

71 (19.2%)
70 (19.0%)
63 (17.1%)
65 (17.6%)
35 (9.5%)
65 (17.6%)

ANOVA was also performed to evaluate whether there were pretest differences in socialemotional competence and academic achievement by student gender and grade level. Results
indicated statistically significant differences between boys and girls in baseline social-emotional
competence. Mean differences by gender neared a medium effect size with Cohen’s d
calculations at .45. Inspection of the mean scores indicated that teachers rated girls higher in
social-emotional competence than boys. Statistically significant differences by grade level in
social-emotional competence were not found. Although significant differences in academic
achievement were indicated across grade level in the expected direction, significant achievement
differences by gender were not found. To statistically reduce the confounding influence of preexisting group differences when testing for effects of intervention group on student outcomes,
baseline measurement of social-emotional competence and academic achievement were included
in subsequent analyses as covariates.

63

Primary Analysis
To examine the effect of Second Step on student outcomes, a quasi-experimental design
adjusting for pretest scores was utilized. Between-groups ANCOVA and MANCOVA analyses
were conducted with group (Second Step, Wait-list comparison), student level of learning risk
(low risk, moderate risk, high risk), and gender as independent variables and baseline socialemotional competence and academic achievement as covariates. The dependent variables were
social-emotional competence (skills for learning, empathy, emotion management), reading
achievement, and math achievement at posttest. Means and standard deviations of the dependent
variables and covariates by grade are presented in Table 8.
Table 8
Means and Standard Deviations for Dependent Variables and Covariates by Grade
Grade
Kindergarten (n = 71)
Pretest
Posttest
First-Grade (n = 70)
Pretest
Posttest
Second-Grade (n = 63)
Pretest
Posttest
Third-Grade (n = 65)
Pretest
Posttest
Fourth-Grade (n = 35)
Pretest
Posttest
Fifth-Grade (n = 65)
Pretest
Posttest
Total (N = 369)
Pretest
Posttest

Social-Emotional
M

Reading

SD

M

SD

Math
M

SD

46.48
47.15

8.29
9.32

136.55
150.56

8.54
10.74

133.71
150.35

10.64
13.76

48.17
46.65

9.43
10.89

153.83
162.97

9.70
11.09

152.99
165.70

11.97
10.71

44.81
45.21

13.61
14.79

165.21
174.91

12.37
13.00

170.25
179.24

12.02
10.09

44.45
48.67

11.72
10.00

178.63
187.42

14.37
12.68

184.48
190.66

11.28
10.60

44.57
45.11

9.24
8.88

191.63
196.22

14.28
12.94

195.83
199.86

10.85
9.29

44.45
48.82

7.04
7.81

196.29
200.54

11.64
10.63

198.09
202.93

10.98
11.96

45.62
47.09

10.17
10.62

167.88
176.70

24.09
21.49

169.78
179.25

25.99
22.04

64

Although grade was originally included in the analysis as a control and differential
effects were not predicted for this variable, examination of outcome means revealed a pattern
that merited further investigation of the moderated effects of condition by student grade. The
imputed means and standard deviations of dependent variables and covariates by individual
grade are presented in Table 8. Analysis by individual grade (kindergarten through fifth totaling
six levels) was considered, but produced inadequate cell sizes to detect statistical significance.
Instead, individual grades were grouped into a dichotomous variable, lower elementary
(kindergarten, first, second) and upper elementary (third, fourth, fifth), and included in the
analysis.
For all analyses, the null hypotheses were rejected when p < 0.05. When statistically
significant differences were found, the practical importance of the differences was evaluated
using the standardized mean reported as Cohen’s d. This statistic expresses the mean difference
between intervention and comparison groups on outcome measures (social-emotional
competence, math achievement, reading achievement) in standard deviation units. The following
standard interpretation offered by Cohen was used evaluate the magnitude of the effect: small
= .2, moderate = .5, large = .8 (1988). Given the questions that have arisen in the literature
regarding the appropriateness of applying such standards when assessing the magnitude of
school-based SEL intervention effects (e.g., Durlak et al., 2011; Payton et al., 2008), effect size
was also interpreted within the context of previous research by comparing the size of effect to
that achieved by other psychosocial and academic interventions. In keeping with Payton and
colleagues’ (2008) interpretation of the practical value of improvements in student outcomes
resulting from SEL intervention, program effects were also translated into improvement indices

65

using Cohen’s U3 to show percentile gains achieved by the average student in the Second Step
group compared to the average student in the wait-list comparison group (Cohen, 1988).
Effects of Second Step on Social-Emotional Competence
Multivariate analysis of covariance (MANCOVA) was conducted to examine the effect
of universal Second Step programming on elementary students’ social-emotional competence and
answer the first research question.
RQ1: Controlling for baseline social-emotional competence, do students receiving Second
Step intervention show greater improvement in social-emotional competence (skills for
learning, empathy, emotion management, problem solving) than students in a wait-list
comparison group? If so, do improvements in social-emotional competence depend on
student level of learning risk at baseline? Are intervention effects moderated by gender?
Hypothesis 1a: Students receiving Second Step would show greater improvements in the
combination of social-emotional competencies than students in a wait-list comparison
group.
Hypothesis 1b: Students demonstrating a higher level of learning risk at baseline would
show greater improvements in social-emotional competence students demonstrating lower
learning risk.
Hypothesis 1c: The effects of Second Step would be moderated by gender, with boys
benefiting more from participation in SEL programming than girls.
Table 9 summarizes the data analysis procedures to test the first research question and
hypotheses: Analysis included two independent variables (intervention group and learning risk)
and three dependent variables (skills for learning, empathy, emotion management) measures of
social-emotional competence. Gender and grade level were included as control variables and

66

baseline social-emotional competence was included as a covariate. Descriptive statistics for the
three social-emotional competence outcome variables by intervention group, learning risk,
gender, and grade are presented in Table 10.
Table 9
Research Question 1 and Data Analysis
Research Question

Variables

Measures

Data Analysis

Do students receiving
Second Step show greater
improvement in socialemotional competence than
students in a wait-list
comparison group? If so,
do improvements in socialemotional competence
depend on student level of
learning risk at baseline?
Are intervention effects
moderated by gender?

IV: Group
IV: Category of Risk

DESSA-SSE
DESSA-mini
DIBELS

MANCOVA

COV: Pretest SEC
MODERATOR: Gender
CONTROL: Grade
DV: Posttest Scores
Skills for Learning
Empathy
Emotion Mng
Problem Solving

Table 10
Descriptive Statistics for Social-Emotional Competence Variables by Intervention Group,
Learning Risk, Grade, and Gender
Variable
Group

Elementary Grade

Learning Risk

Gender

Level

n

Second Step
Comparison

238
126

Social Emotional Competence
SLT
EPT
EMT
Adj M
SD
Adj M
SD
Adj M
SD
47.80 10.38
47.29 10.56
48.32 10.75
47.10 10.40
47.85 9.76
48.60 10.07

Lower
Upper

200
164

45.40
49.49

11.31
9.05

46.60
48.55

11.61
8.67

47.20
49.73

11.91
8.71

Low
Moderate
High

101
143
120

51.88
45.88
44.58

10.02
8.24
7.74

49.16
47.00
46.55

10.06
9.21
8.25

50.84
47.53
47.03

10.41
9.41
7.23

Male
Female

186
178

47.10
47.80

10.22
10.12

46.25
48.89

9.85
10.08

47.88
49.05

10.23
10.38

Note. SLT = Skills for Learning Total. EPT = Empathy Total. EMT = Emotion Management Total. Adjusted mean
based upon Baseline SLT, EPT, and EMT scores.

67

The MANCOVA did not yield a significant main effect for intervention group, F(3,
337)=1.10, p>.05, as predicted. Results revealed a main effect for grade level, F(3, 337)=7.61,
p<.01, d =.52., that was qualified by a significant interaction with intervention group on the
combined dependent variables F(3, 337)=4.88, p<.05, d =.42. Second Step students in the lower
elementary grades showed greater improvements on the combined social-emotional competence
scores (skills for learning, empathy, emotion management) from pretest to posttest than lower
elementary students in comparison classrooms. Multivariate results adjusted for baseline socialemotional competence scores are presented in Table 11.
Table 11
Multivariate Analysis of Covariance Summary for Social-Emotional Competence by Intervention
Group, Learning Risk, Grade, and Gender
Source

Pillai’s Trace

df

Error

F

Cohen’s d

Covariate

.324

3

337

53.90*

1.38

Intervention Group

.010

3

337

1.10

.20

Learning Risk

.118

6

676

7.04*

.50

Grade

.063

3

337

7.61*

.52

Gender

.032

3

337

3.75*

.36

Group * Risk

.021

6

676

1.17

.20

Group * Grade

.042

3

337

4.88*

.42

Group * Gender

.012

3

337

1.40

.22

Note. Multivariate F ratios were generated from Pillai’s statistic.
*p < .05.

Cohen’s U3 index was used to translate effect sizes into a percentile rank for the average
student in the Second Step group compared to the average comparison student who ranks at the
50th percentile. The mean effect size on measures of lower elementary students’ social-emotional
competence translates into a percentile difference of 16%. In other words, the average lower

68

elementary student in Second Step classrooms showed a 16-percentile-point gain in socialemotional competence compared to the average lower elementary student in a comparison
classrooms. Improvements in social-emotional competence in the upper elementary grades were
not significantly different for Second Step students and comparison students.
Table 12
Univariate Analysis of Covariance Summary for Social-Emotional Competence by Intervention
Group, Learning Risk, Grade, and Gender for Measures
Source

Skills for Learning

Empathy

Emotion Management

df

F

d

df

F

d

df

F

d

Covariate

1

103.28*

1.11

1

121.21*

1.19

1

160.63*

1.38

Group

1

.62

.09

1

.37

.06

1

.10

.00

Learning Risk

2

18.78*

.67

2

2.23

.23

2

5.50*

.36

Grade

1

21.20*

.50

1

4.47*

.23

1

8.14*

.31

Gender

1

.62

.09

1

8.04*

.31

1

1.73

.14

Group * Risk

2

2.07

.22

2

.75

.13

2

1.04

.16

Group * Grade

1

1.25

.13

1

6.22*

.27

1

.132

.00

Group * Gender

1

1.46

.13

1

.42

.06

1

2.80

.18

Note. d = Cohen’s d.
*p < .05.

Given the significance of the overall test, the univariate interaction effects for
intervention group by grade were examined. There was a significant univariate interaction effect
for intervention group and grade on improvement in empathy skills, F(1, 300)=6.22, p=.013, d
=.27. Univariate interaction effects for skills for learning, F(1, 300)=1.248, p=.265, and emotion
management, F(1, 300)=0.132, p=.716, were not significant. Second Step students in the lower
grades showed greater improvement in empathy skills than lower elementary students in
comparison classrooms. In the upper elementary grades, however, students in comparison

69

classrooms showed greater gains in empathy skills than peers in Second Step classrooms.
Univariate results adjusted for baseline social-emotional competence scores are presented in
Table 12.

Empathy Score

Lower Elementary
54
52
50
48
46
44
42
40

Second Step
Comparison

Pretest Mean

Posttest Mean

Empathy Score

Upper Elementary
54
52
50
48
46
44
42
40

Second Step
Comparison

Pretest Mean

Posttest Mean

Figure 5. Simple plot for interaction effects between intervention group by grade level on mean
empathy score
The hypothesis that students receiving Second Step would show greater improvements in
social-emotional competencies than students in a comparison group (Hypothesis 1a) was
partially supported by these findings. Although greater gains in social-emotional competence
skills were observed for Second Step students in the lower elementary, intervention effects were
not uniformly found across all grade levels. Second Step students showed greater improvement in
empathy skills than peers in comparison classrooms. The opposite effect was observed in the
upper grades with students in comparison classrooms showing greater gains than peers in Second
70

Step classrooms. The significant interaction effect for intervention by group level on
improvements in empathy skills is illustrated in Figure 5.
MANCOVA results also yielded a significant main effect for level of learning risk (low,
moderate, high) on the improvement in social-emotional competence scores, F(6,676)=7.04,
p<.05, d=.50. Students demonstrating low learning risk at baseline showed greater overall
improvements in social-emotional competence from pretest to posttest than students
demonstrating moderate and high learning risk. Results did not yield the anticipated significant
multivariate interaction effect for learning risk and intervention group on improvements in
social-emotional competence, F(6, 676)=1.17, p>.05. As illustrated in Table 11, examination of
the univariate main effect for learning risk revealed significantly greater improvement in skills
for learning, F(2, 300)=18.78, p<.001, d =.67, and emotion management, F(2, 300)=5.498,
p=.035, d =.36. A significant univariate effect for learning risk on improvement in empathy skills
was not found, F(2, 300)=2.228, p = .109. Significant pairwise mean differences found low
learning risk students, compared to moderate and high learning risk students, made greater gains
in skills for learning and emotion management scores from pretest to posttest. Overall, lower
learning risk students were found to demonstrate greater gains in social-emotional competence
from pretest to posttest than moderate and high learning risk students. Contrary to expectations,
the effect of Second Step participation on improvements on social-emotional competence was not
significantly moderated by student level of learning risk at baseline. Results from the study failed
to support the hypothesis that students demonstrating higher levels of learning risk would benefit
more from Second Step participation than lower learning risk students (Hypothesis 1b).
MANCOVA results yielded a significant main effect for gender on improvements in the
combined social-emotional subscale scores, F(3, 337)=3.75, p<.05, d =.36. Overall, girls

71

demonstrated greater gains in social-emotional competence from pretest to posttest compared to
boys. Gender did not moderate the effect of Second Step on improvements in social-emotional
competence as predicted, F(3, 337)=1.40, p>.05. Examination of the univariate main effects for
gender indicated that girls showed greater improvement in empathy skills from pretest to posttest
compared to boys, F(1, 300)=8.04, p=.005, d =.31. Significant differences between boys and
girls on improvements in skills for learning, F(1, 300)=0.619, p = .432, and emotion
management, F(1, 300)=1.73, p = .189, were not found. Descriptive statistics disaggregated by
intervention group for performance on the three social-emotional competence outcome by grade,
learning risk, and gender are presented in Tables 13, 14, and 15 respectively.
Table 13
Descriptive Statistics of Social-Emotional Competence by Intervention Group and Grade Level
Social Emotional Competence
SLT
EPT
EMT
Adj M SD
Adj M SD
Adj M SD

Group

Elementary Grade

n

Second Step

Lower
Upper

133
105

46.25 11.05
49.34 9.38

47.46
47.12

11.70
8.94

47.22 11.96
49.43 8.97

Comparison

Lower
Upper

67
59

44.56 11.85
49.63 8.40

45.73
49.98

11.45
7.29

47.18 11.60
50.03 8.09

Note. SLT = Skills for Learning Total. EPT = Empathy Total. EMT = Emotion Management Total. Adjusted mean
based upon Baseline SLT, EPT, and EMT scores.

72

Table 14
Descriptive Statistics of Social-Emotional Competence by Intervention Group and Learning Risk
Group

Learning Risk

n

Social Emotional Competence
SLT
EPT
EMT
Adj M SD
Adj M SD
Adj M SD

Second Step

Low
Moderate
High

57
92
89

50.95
46.84
45.59

9.63
8.86
8.36

48.26
46.52
47.10

10.02
10.11
8.72

49.93
47.26
47.78

10.63
10.14
7.78

Comparison

Low
Moderate
High

44
51
31

52.81
44.91
43.57

10.58
7.06
5.65

50.07
49.49
46.01

10.10
7.17
6.84

51.74
47.79
46.27

10.18
7.82
5.48

Note. SLT = Skills for Learning Total. EPT = Empathy Total. EMT = Emotion Management Total. Adjusted mean
based upon Baseline SLT, EPT, and EMT scores.

Table 15
Descriptive Statistics of Social-Emotional Competence by Intervention Group and Gender
Social Emotional Competence
SLT
EPT
EMT
Adj M SD
Adj M SD
Adj M SD

Group

Gender

n

Second Step

Male
Female

126
112

46.91
48.68

10.31
10.38

45.67 10.38
48.91 10.05

47.00
49.65

10.54
10.42

Comparison

Male
Female

60
66

47.28
46.91

10.63
10.23

46.84
48.87

48.76
48.45

9.40
10.16

8.51
9.90

Note. SLT = Skills for Learning Total. EPT = Empathy Total. EMT = Emotion Management Total. Adjusted mean
based upon Baseline SLT, EPT, and EMT scores.

Effects of Second Step on Academic Achievement
Separate analyses of covariance (ANCOVA) were conducted to examine the
effectiveness of Second Step programming in promoting elementary students’ reading
achievement and math achievement and to answer research question 2.
RQ2: Controlling for baseline achievement, do students receiving Second Step intervention
show greater improvement in academic achievement than students in a wait-list
comparison group? If so, do improvements in reading achievement depend on student level
of learning risk at baseline? Are intervention effects moderated by gender?
73

Hypothesis 2a: Students receiving Second Step intervention would show greater
improvement in reading and math achievement than students in a wait-list comparison
group.
Hypothesis 2b: Students demonstrating a higher level of learning risk at baseline would
show greater improvements in reading achievement than students demonstrating lower
learning risk.
Hypothesis 2c: The effects of Second Step on reading and math achievement will be
moderated by gender, with boys benefiting more from participation in SEL programming
than girls.
Table 16 summarizes the procedures used to test research question 2 and associated
hypotheses. The independent variables were group (Second Step, comparison) and learning risk
at pretest (low, moderate, high). Baseline measures of academic achievement were included as
covariates. Student gender and grade were also included as control variables in the analyses.
Descriptive statistics for reading and math achievement by intervention group, learning risk,
gender, and grade are presented in Table 17.
Table 16
Research Question 2 and Data Analysis
Research Question

Variables

Measures

Data Analysis

Do students receiving
Second Step show greater
improvement in
academic achievement
than students in a waitlist comparison group? If
so, do improvements in
academic achievement
depend on student level
of learning risk at
baseline? Are
intervention effects
moderated by gender?

IV: Group
IV: Category of Risk

MAP
DESSA-mini
DIBELS

ANCOVA

COV: Pretest Achievement
MODERATOR: Gender
CONTROL: Grade
DV: Posttest Achievement

74

Table 17
Descriptive Statistics for Academic Achievement by Intervention Group, Learning Risk, Grade,
and Gender
Variable

Level

n

Second Step 238
Comparison 126

Group
Elementary Grade
Learning Risk
Gender

Reading Achievement Math Achievement
Adj M
SD
Adj M
SD
178.01
20.85
180.03
21.28
175.66
23.09
178.15
23.84

Lower
Upper

200
164

175.59
182.59

15.24
13.26

175.37
182.86

16.61
12.16

Low
Moderate
High

101
143
120

178.21
175.60
176.70

21.32
21.88
19.73

181.32
178.83
177.12

20.74
23.47
20.81

Male
Female

186
178

176.50
177.19

21.32
21.96

178.98
179.26

22.36
22.04

Note. Adjusted mean based upon Baseline Reading and Math Achievement scores.

Adjusting for baseline scores, significant main effects for group were found on
improvements in reading, F(1, 351)=5.56, p=.019, d =.26, and math, F(1, 351)=5.36, p=.021, d
=.25. The overall effect of group, however, was qualified by a significant interaction with grade
level on reading, F(1, 351)=5.79, p<.001, d =.26, and math, F(1, 351)=7.34, p<.001, d =.35.
Lower elementary students in Second Step classrooms demonstrated greater gains in reading and
math achievement compared to lower elementary students in comparison classrooms. Using
Cohen’s U3 to translate achievement into a percentile rank for the average student in the Second
Step group compared to the average comparison student who ranks at the 50th percentile, results
indicated a percentile difference of 10% in reading achievement and 11% in math achievement
for lower elementary students in Second Step classrooms. In other words, the average lower
elementary student in comparison classrooms would demonstrate a 10-percentile-point (reading)
to 11-percentile-point (math) gain in achievement if they had participated in Second Step

75

programming. Similar differences between Second Step and comparison students in the upper
elementary grades were not found.
ANCOVA results for improvements in academic achievement scores from pretest to
posttest are presented in Table 18 (reading) and Table 19 (math). The significant interactions for
intervention group by grade on improvements in academic achievement scores are illustrated in
Figure 6 (reading) and Figure 7 (math). These findings indicate that the positive impact of
participation in Second Step on academic achievement was not uniformly observed and
significantly depended on grade level.

Table 18
Analysis of Covariance Summary for Reading Achievement by Intervention Group, Learning
Risk, Gender, and Grade
Source
Intervention Group
Learning Risk
Gender
Grade Level
Intervention Group x Learning Risk
Intervention Group x Gender
Intervention Group x Grade Level
Error

SS

df

MS

F

Cohen’s d

377.67
378.10
14.04
947.81
458.37
1.15
392.87
23823.80

1
2
1
1
2
1
1
351

377.67
189.05
14.04
947.81
229.19
1.15
392.87
67.87

5.56*
2.79
.20
13.96*
3.38*
.02
5.79*

.26
.26
.06
.40
.28
.06
.26

Note: R2 = .860, adj. R2 = .855. Other possible two-way and three-way interactions were not significant at the .05
level.
* p < .05

76

Table 19
Analysis of Covariance Summary for Math Achievement by Intervention Group, Learning Risk,
Gender, and Grade
Source
Intervention Group
Learning Risk
Gender
Grade Level
Intervention Group x Learning Risk
Intervention Group x Gender
Intervention Group x Grade Level
Error

SS

df

MS

F

Cohen’s d

448.30
778.01
5.34
1494.87
223.33
14.20
614.14
29370.69

1
2
1
1
2
1
1
351

448.30
389.01
5.34
1494.87
111.66
14.20
614.14
83.68

5.36*
4.65*
.064
17.87*
1.33
.170
7.34*

.25
.33
.00
.45
.18
.06
.35

Note: R2 = .835, adj. R2 = .830. Other possible two-way and three-way interactions were not significant at the .05
level.
* p < .05.

Lower Elementary

Reading Score

180
170

Second Step

160

Comparison

150
140

Pretest Mean Posttest Mean
Upper	Elementary

Reading Score

200
190
180
170

Second Step

160

Comparison

150
140

Pretest Mean Posttest Mean

Figure 6. Simple plot for interaction effects between intervention group by grade level on
reading achievement

77

Lower Elementary

Math Score

180
170

Second Step

160

Comparison

150
140

Pretest Mean Posttest Mean
Upper Elementary

Math Score

200
190
180
170

Second Step

160

Comparison

150
140

Pretest Mean Posttest Mean

Figure 7. Simple plot for interaction effects between intervention group and grade level on math
achievement
Overall, improvement in reading achievement did not significantly differ across the three
levels of learning risk, F(2, 351)=2.79, p>.05. Learning risk did, however, moderate the effect of
intervention group on improvement in reading scores from pretest to posttest, F(2, 351)=3.38, p
= .035, d =.28. Results from simple effects tests of the significant intervention group by learning
risk interaction are presented in Table 20. As anticipated, moderate learning risk students in
Second Step classrooms showed greater improvements in reading achievement than moderate
learning risk students in comparison classrooms and low learning risk students in Second Step
classrooms did not demonstrate greater gains in reading scores than counterparts in comparison
classrooms. Contrary to expectations, students in Second Step classrooms demonstrating the
highest level of learning risk did not demonstrate significantly greater improvements in reading

78

achievement than high learning risk students in comparison classrooms. The significant
interaction effect of intervention group by learning risk on improvements in reading achievement
is illustrated in Figure 8.

Low Risk

Reading Score

190
180
170

Second Step

160

Comparison

150
140

Pretest Mean Posttest Mean
Moderate Risk

Reading Score

190
180
170

Second Step

160

Comparison

150
140

Pretest Mean Posttest Mean
High Risk

Reading Score

190
180
170

Second Step

160

Comparison

150
140

Pretest Mean Posttest Mean

Figure 8. Simple plot for interaction effects between intervention group by learning risk on
reading achievement

79

Table 20
Comparisons of Mean Differences in Reading Achievement by Learning Risk and Intervention
Group
Intervention Group Comparison by
Learning Risk
Low Learning Risk
Second Step vs. Comparison
Moderate Learning Risk
Second Step vs. Comparison
High Learning Risk
Second Step vs. Comparison

Estimated
Mean
Difference

Standard Error of
Difference

Bonferroni
Adjusted 95% CI

.148

1.69

-3.18, 3.47

5.35*

1.47

2.46, 8.25

1.37

1.76

-4.83, 2.09

Note. Comparisons based upon ANCOVA adjusted means controlling for Baseline Reading Achievement mean of
167.90.
* p < .05, where p-values are adjusted using the Bonferroni method.

A significant main effect for learning risk on math achievement was also found, F(2,
351)=4.65, p = .035, d =.33. Pairwise comparisons of overall differences in math improvement
from pretest to posttest across the three levels of learning risk are presented in Table 21. In
contrast to reading achievement scores, student level of learning risk at baseline did not
significantly moderate the effect of Second Step on improvements in math achievement scores
from pretest to posttest as expected, F(2, 351)=1.33, p>.05. Results revealed significant mean
differences for only one pair; low learning risk students showed significantly greater
improvement in math scores than students demonstrating high learning risk. Significant mean
differences between low and moderate learning risk students or moderate and high learning risk
students were not found.

80

Table 21
Comparisons of Mean Differences in Math Achievement by Learning Risk
Learning Risk Comparison
Low vs. Moderate
Low vs. High
Moderate vs. High

Estimated Mean
Difference
1.37
2.92*
1.55

Standard Error
of Difference
1.28
1.42
1.31

Bonferroni Adjusted
95% CI
-1.13, 3.88
.121, 5.72
-1.02, 4.11

Note. Comparisons based upon ANCOVA adjusted means controlling for Baseline Math Achievement mean of
169.79.
* p < .05, where p-values are adjusted using the Bonferroni method* p < .05, where p-values are adjusted using the
Bonferroni method.

These findings partially support the hypothesis that students demonstrating a higher level
of learning risk at baseline would benefit more from Second Step intervention than students
demonstrating lower learning risk (Hypothesis 2b). Among students in Second Step classrooms,
those students categorized as demonstrating a moderate level of learning risk showed greater
improvements in reading achievement than both low learning risk and high learning risk
students. Students demonstrating the highest level of learning risk, however, did not show greater
reading gains than students demonstrating lower levels of learning risk (low, moderate). For
math, level of learning risk did not moderate the impact of intervention group on improvements
in math achievement from pretest to posttest as hypothesized.
ANCOVA results did not yield a significant main effect for gender on improvements in
reading achievement, F(1,351)=.20, p>.05, or math achievement, F(1,351)=0.064, p>.05.
Furthermore, gender did not moderate the effect of intervention group on improvements in
reading scores, F(1,351)=0.02, p>.05, or math scores, F(1,351)=0.170, p>.05, from pretest to
posttest. The hypothesis that the benefit of Second Step participation on improvements in math
and reading scores would be greater for boys than girls (Hypothesis 2c) was not supported by
these findings. Descriptive statistics disaggregated by intervention group for reading and math

81

achievement by grade, learning risk, and gender are presented in Tables 22, 23, and 24
respectively.
Table 22
Descriptive Statistics for Academic Achievement by Intervention Group and Grade
Group
Second Step
Comparison

Elementary Grade

n

Lower
Upper
Lower
Upper

133
105
67
59

Reading Achievement
Adj M
SD
176.57
14.73
179.45
13.30
171.55
15.98
179.88
13.29

Math Achievement
Adj M
SD
177.59
15.99
182.48
12.78
173.59
17.47
182.71
11.00

Note. Adjusted mean based upon Baseline Reading and Math Achievement scores.

Table 23
Descriptive Statistics for Academic Achievement by Intervention Group and Learning Risk
Reading Achievement
Adj M
SD

Math Achievement
Adj M
SD

Group

Learning Risk

n

Second Step

Low
Moderate
High

57
92
89

178.39
178.17
177.46

20.27
20.39
20.22

181.55
181.03
177.48

18.69
21.16
21.84

Comparison

Low
Moderate
High

44
51
31

178.03
173.03
175.03

22.83
23.79
18.51

181.04
176.64
176.77

23.33
26.83
17.58

Note. Adjusted mean based upon Baseline Reading and Math Achievement scores.

Table 24
Descriptive Statistics for Academic Achievement by Intervention Group and Gender
Social Emotional Competence
SLT
EPT
EMT
Adj M SD
Adj M SD
Adj M SD

Group

Gender

n

Second Step

Male
Female

126
112

46.91
48.68

10.31
10.38

45.67
48.91

10.38
10.05

47.00
49.65

10.54
10.42

Comparison

Male
Female

60
66

47.28
46.91

10.63
10.23

46.84
48.87

8.51
9.90

48.76
48.45

9.40
10.16

Note. Adjusted mean based upon Baseline Reading and Math Achievement scores.

82

CHAPTER 5
DISCUSSION
A growing body of research indicates that social-emotional competence and academic
achievement are interwoven and that integrated instruction and support in both areas can
maximize student success (Zins & Elias, 2007) and reduce the risk of maladjustment (Elias et al.,
1997; Zins et al., 2004). This investigation was guided by the SEL conceptual framework for the
organization and integration of school-based prevention programs. The framework holds that
school-based SEL programs can lead to immediate improvements in the classroom environment
and student social-emotional competence, establishing the mechanisms through which more
distal improvements in academic performance are realized. Full implementation of the model of
integrated educational programming as conceptualized in the SEL framework and applied
through MTSS practices was beyond the scope of the present project. Rather, Second Step
programming and social-emotional assessment were confined to Tier I and were minimally
integrated with existing academic RTI practices. The study represents initial steps in examining
the potential benefit of integrating universal SEL programming into an existing tiered approach
to elementary education.
This study tested the hypotheses that students participating in Second Step, a universal,
skills-focused SEL curriculum, would show greater improvements in social-emotional
competence and academic achievement than students not receiving the intervention. The
hypothesis that student response to Second Step would differ by level of baseline learning risk
and gender was also tested. This chapter interprets key findings in the context of prior research
and the theoretical framework guiding the investigation. Limitations of the study, as well as
implications for practice and future research, are also discussed.

83

Second Step and Social-Emotional Competence
Study results did not support the hypothesis that Second Step participation would result in
greater gains in social-emotional competence for all students. Post-hoc analysis, however,
revealed differential benefits when examined by grade. Lower elementary students in Second
Step classrooms showed significantly greater improvements in social-emotional competence than
peers in comparison classrooms (Cohen’s d = .4). Although the effect size might seem small, in
the context of prior research, the present results for the lower elementary grades are comparable
to outcomes from meta-analyses of school-based, universal SEL interventions (Durlak et al,
2011; Payton et al., 2008). Translated into a percentile rank for the average Second Step student
to the average comparison student, the effect size for social-emotional competence suggests
meaningful benefits for lower elementary students following Second Step participation. The
average lower elementary student in Second Step classrooms showed a 16-percentile-point gain
in social-emotional competence compared to the average lower elementary student in a
comparison classrooms. From a practical standpoint, the effects observed in the present study
reflect a level of improvement in social-emotional competence that likely generalizes into
valuable and noticeable differences in lower elementary classroom settings.
Similar intervention benefits were not observed in the upper grades. This finding was not
predicted and is inconsistent with large-scale reviews of the broader SEL literature reporting no
link between student response to intervention and grade level (Durlak et al., 2011; Payton et al.,
2008). The discrepancy in findings may be related to the highly selective sample from which the
present study data were drawn. The differential effects of SEL intervention by grade level were
observed in a markedly homogeneous student population embedded within a common school
culture and exposed to similar community factors. In contrast, meta-analyses results were based

84

on comparison across multiple studies representing varying geographical settings, school
cultures, student demographics, intervention programs, and implementation methods (Durlak et
al., 2011; Payton et al., 2008). Although meta-analysis affords greater statistical power and
ability to extrapolate to the general population, the pooling of multiple studies may result in a
more heterogeneous study populations and variables that mask sub group differences such as the
differential response to intervention by grade observed in the present study. The current findings
add to the research base regarding the differential benefits that lower and upper elementary
students from socioeconomically disadvantaged communities might derive from participation in
universal SEL intervention like Second Step.
Notably, the differential effects of Second Step participation in the lower grades were
specific to the development of empathy skills. Improvements in lower elementary students’ skills
for learning and emotion management did not significantly differ between treatment groups.
Thus, not only did students differ in their response to Second Step depending on grade level, the
intervention also differentially targeted key social-emotional competencies. This finding is not
surprising given the scope and sequence of skill development inherent in the Second Step
curriculum and, by extension, reflected in the DESSA-SSE and the present study. Second Step
defines empathy skills as the ability to identify emotions in self and others, label those emotions,
and take on others’ perspectives (Committee for Children, 2011). These skills are considered
foundational to the development of emotional literacy and critical interpersonal competencies,
two important factors in the development of more complex social-emotional skills such as
emotion management and problem solving. Compared to other competencies targeted by the
Second Step curriculum (i.e., skills for learning, emotion management), the development of
empathy skills relies heavily on the acquisition of content knowledge and language skills and,

85

much like social studies or science, are most readily demonstrated and measured through
traditional classroom practices (i.e., class discussion, question/response, partner share).
Competencies such as skills for learning and emotion management, on the other hand, represent
a more complex subset of competencies that require the integration of multiple skills and demand
more time and opportunity for practice before they can be competently generalized to the school
setting.
Considering the SEL conceptual framework, the current findings suggest that empathy
skills may represent an immediate outcome that lays the groundwork for other, more distally
situated social-emotional competencies such as skills for learning and emotion management.
Findings from a largescale review of the school-based, SEL intervention research indicating
more robust effects on improvements in content-related knowledge and skills compared to higher
level competencies (e.g., emotion management, problem solving) lend support to this argument
(Durlak et al., 2011; Peyton et al., 2008). These findings suggest that social-emotional
competence skills targeted by SEL intervention may not develop simultaneously as suggested by
the Second Step logic model and SEL conceptual framework. Rather, foundational skills that rely
heavily on the acquisition of content knowledge may develop more quickly and be more readily
observable to teachers in the classroom setting and more conducive to rating scale measurement.
Competencies that demand the integration of foundational and newly acquired skills may
constitute more distal, secondary outcomes that will develop over time as they are generalized
across school settings and situations.
Differential effects of intervention by grade have not been reported in large-scale reviews
of SEL intervention research or individual Second Step studies. However, the observed benefit
for younger students is perhaps not that surprising given what is known about early intervention,

86

particularly in the context of school communities characterized by multiple risk indicators. A
foundational tenet of the SEL conceptual framework holds that social-emotional competence
skills are malleable and can be developed and reinforced through direct SEL instruction and
enhancement of classroom social environment (Elias et al., 1997; Greenberg et al., 2003; Zins &
Elias, 2006; Zins et al., 2004). However, educational histories marked by chronic socioeconomic
disadvantage and exposure to multiple neighborhood and family risk factors may set students on
behavioral, social, and academic trajectories that become more firmly established and less
amenable to the positive changes associated with SEL intervention as they progress through the
elementary grades (Roeser, Eccles, & Freedman-Doan, 1999). The present findings suggest that
the malleability of social-emotional and behavioral skills may have unique developmental
implications within the context of socioeconomic disadvantage. It could be that, for upper
elementary students facing multiple neighborhood and school risk factors, universally delivered
SEL intervention may be inadequate to remediate skills that have grown resistant to change over
time. Older students from communities characterized by a high level of risk might require more
intense and targeted Tier II intervention (e.g., increased frequency and duration delivered in the
small group setting) to show similar improvements in social-emotional competence skills as their
younger schoolmates.
Although differential benefits by grade were not predicted, it was hypothesized that
students demonstrating a higher level of learning risk at baseline would have the most to gain
from Second Step participation, resulting in greater gains in social emotional competence
following intervention compared to peers at less risk of learning difficulty. Contrary to
hypothesized outcomes and previous research (Frey et al., 2005; Low et al., 2015; Taub, 2001),
Second Step did not produce larger gains in social-emotional competence among students at

87

higher levels of learning risk. This unexpected result might be due to differences in the number
and quality of risk factors represented in the current sample compared to previous studies. The
present study involved a highly selective sample, drawn from one school community
characterized by neighborhood, family, and student factors historically linked to weaker social,
emotional, and behavioral outcomes. By comparison, previous Second Step studies indicating
greater benefits for students initially lower in social-emotional skills (Frey et al., 2005; Low et
al., 2015; Taub, 2001) reported predominantly White, non-urban, and socioeconomically diverse
samples, a demographic profile generally found to be at less risk for negative school outcomes. It
could be that the anticipated differential intervention benefits for students at greater risk for
learning difficulty were overwhelmed by the multitude of additional neighborhood and school
risks faced by students in the current sample.
The lack of differential benefits of Second Step by level of learning risk may also be
explained by differences in the current study’s conceptualization and measurement of learning
risk compared to previous SEL intervention studies. The present study incorporated academic
and social-emotional indicators in categorizing students’ level of learning risk. This expanded
approach likely captured a greater breadth and intensity of potential learning difficulties than
previous studies that relied solely on measures of social-emotional competence (e.g., Frey et al.,
2005; Low et al., 2015; Taub, 2001). Consideration of the reciprocal nature of academic and
social-emotional development may be particularly relevant for student learning outcomes in the
context of high risk communities. For example, children from high-poverty, urban communities,
are at greater risk for language delays (e.g., vocabulary, comprehension, pragmatics) and
cognitive difficulties (e.g., memory, visual-spatial skills, attention) that can interfere with the
ability to engage with SEL instruction, comprehend and recall program content, and successfully

88

demonstrate acquired skills in the classroom setting (e.g., Norman & Farah, 2005; Hart & Risley,
1995). Youth from socioeconomically disadvantaged communities face a multitude of
developmental risk factors that can interfere with the ability to access and engage with classroom
instruction and negatively impact academic development. Given the mutually reinforcing
relationship between social-emotional and academic development, more intensive Tier II SEL
intervention, in addition to universal programming, may be necessary to address the needs of
students at greatest risk for negative school outcomes in high-poverty, urban school settings.
Boys, on average, have been found to demonstrate lower levels of social-emotional
competence than girls (Bosacki & Moore, 2004; Brown & Dunn, 1996; Else-Quest et al., 2006;
Garner & Waajid, 2012; Merrell et a., 2011). Because boys appear to have more difficulty
developing social-emotional skills and are at greater risk for deficits in this area, it was
hypothesized that boys would benefit more from SEL participation than their more socialemotionally competent female peers. Contrary to predicted outcomes and findings from earlier
SEL studies (Aber et al., 1998; Belfield et al., 2006; Miller et al., 2010; Muenning et al., 2008;
Frey et al., 2005), boys did not show greater improvements than girls in social-emotional
competence skills. As with the absence of greater intervention benefits for students
demonstrating a higher level of learning risk, the failure to produce greater improvements in
social-emotional competence skills for boys following SEL program participation may be related
to the multiple neighborhood, family, and school risks faced by students in the current sample.
Although boys may have more to gain from SEL intervention given their lower social-emotional
skills at baseline, the multiple risks faced by boys from socioeconomically disadvantaged
neighborhoods and families may prove too detrimental to the ability to access SEL instruction
and demonstrate improvements that exceed those of girls. Although this finding is contrary to

89

prediction and past research, the result seems to confirm recent findings from meta-analysis of
school-based SEL research concluding that, although girls demonstrate higher social-emotional
competence than boys, boys and girls benefit similarly from participation in SEL programming
(Durlak et al., 2011; Payton et al. 2008).
Second Step and Academic Achievement
Overall, students in Second Step classrooms showed significantly greater improvements
in reading and math scores from pretest to posttest than students in comparison classrooms. This
main effect for intervention group, however, significantly depended on student grade level. As
with social-emotional outcomes, lower elementary students in Second Step classrooms showed
greater gains in reading (Cohen’s d = .26) achievement and math achievement (Cohen’s d = .29)
than comparison students. Although these effect sizes are small according Cohen’s (1988)
conventions, when interpreted in the context of prior research, the results for lower elementary
reading and math achievement are comparable to the results of meta-analyses of universal SEL
interventions (Durlak et al, 2011; Payton et al., 2008) as well educational interventions (Hill et
al., 2007). In terms of practical value, results suggest that Second Step participation in the lower
elementary grades can generate meaningful improvement in students’ reading and math
achievement. When translated into a percentile rank, the average lower elementary student in
Second Step classrooms showed a 10-percentile-point gain in reading achievement and an 11percentile-point gain in math achievement compared to the average lower elementary student in
a comparison classrooms. Alternatively, the average lower elementary student in comparison
classrooms would demonstrate a 10-percentile-point (reading) to 11-percentile-point (math) gain
in achievement test scores if they had participated in Second Step programming. From a practical
standpoint, therefore, the seemingly small effect sizes yielded in the lower elementary grades for

90

translate into valuable improvements in reading and math performance in the classroom setting.
Similar effects in the upper elementary grades for academic achievement, however, were not
observed.
The simultaneous improvements observed in both social-emotional competence and
academic achievement for lower elementary students in Second Step classrooms support the SEL
theoretical framework that describes the mechanisms by which the acquisition of socialemotional competence skills might influence academic performance. For example, preparing for
and performing well on tests requires social-emotional skills such as self-control, cooperative
interaction, and appropriate assertiveness and problem solving. The social-emotional skills
supported by SEL intervention allow students to better focus on academic tasks despite learning
difficulties or other risk factors (Masten, 1994). This result also extends the Second Step research
to include improvements in academic achievement scores as an anticipated outcome of the
program; to date, published research has not reported improvements in reading and math scores
as a benefit of Second Step participation. This finding is consistent with predictions and confirms
prior literature indicating the positive effects of SEL curricula on gains in academic performance
(Durlak et al., 2011; Payton et al. 2008).
Although this finding aligns with empirical and theoretical expectations, the greater
benefit for lower elementary students on improvements in academic achievement was not
predicted. As with the moderating effect of grade on improvements in social-emotional
competence skills, the differential benefit of Second Step for younger students suggests that the
mechanisms of school-based SEL intervention through which improvements in academic
outcomes are realized may have unique developmental considerations within the context of
socioeconomic disadvantage. The present findings suggest that when universal SEL

91

programming is implemented in school communities characterized by socioeconomic
disadvantage, academic skills in the lower grades are significantly more amenable to
improvements than academic skills in the upper grades. The differential effects of intervention
by grade might be the result of the cumulative risks associated with socioeconomic disadvantage
that, over time, place students on increasingly persistent pathways of social-emotional,
behavioral, and learning difficulties that grow progressively resistant to change as students
progress through the grade (Roeser et al., 1999). As with the development of social-emotional
competence skills, not all students will respond similarly to school-wide SEL programming.
These results suggest that universal SEL programming may not be sufficient to meet the socialemotional, behavioral, and academic needs of students facing multiple risks to learning and
school success. In such cases, more intensive Tier II intervention may be needed to set to set the
most at-risk student on a path toward positive learning outcomes and school success.
Alternatively, the lack of improvement in academic outcomes for upper elementary
students may have more to do with time than the ineffectiveness of the intervention with this
subgroup of students. From the SEL conceptual framework, academic achievement is considered
a secondary, or more distal, outcome of school-based intervention. In the present study, growth
was measured over the course of a 16-week period. It could be the case that upper elementary
students did benefit from Second Step participation but adequate time was not allowed for the
intervention effects to translate into significant improvements in the secondary outcomes such as
academic achievement scores.
As hypothesized, the positive impact of Second Step on improvements in achievement
scores significantly depended on the level of learning risk that students demonstrated at the
beginning of the school year. Second Step students demonstrating moderate learning risk showed

92

significantly greater gains in reading achievement scores than counterparts in comparison
classrooms. Conversely, Second Step students categorized as low in learning risk showed similar
improvements in reading scores as low risk students in comparison classrooms as predicted.
Significant differences were not found, however, between Second Step students categorized as
demonstrating the highest level of learning risk and similar students in comparison classrooms.
As posited in the previous discussion of the effect of learning risk on social-emotional outcomes
in the context of socioeconomic disadvantage, it could be that students categorized as high in
learning risk in the present sample exceeded a critical threshold of risk, preventing them from
realizing the differential effects of SEL intervention enjoyed by moderately at-risk peers. This
finding suggests that universal SEL may not be adequate to address the needs of students at
greatest risk for negative school outcomes due to socioeconomic disadvantage. As with other
subgroups demonstrating a lack of response to Tier I SEL intervention in the present study,
students categorized as highest in learning risk may require more intensive SEL intervention to
show meaningful improvement in academic achievement. These findings extend previous
Second Step research indicating the moderating effect of learning risk on improvements in
social-emotional competence skills by including academic achievement as an outcome measure.
These results are relatively consistent with previous research suggesting that students
demonstrating moderate levels of learning risk at the start of the school year experience greater
benefit from SEL intervention than more proficient peers (Frey et al., 2005; Low et al., 2015;
Taub, 2001).
Limitations of the Study
Although the findings reported here are encouraging, several limitations should be noted
while evaluating the results. One limitation stems from the use of a quasi-experimental design

93

rather than a randomized controlled trial. Teachers were not randomly assigned to intervention
classrooms; instead, they volunteered or were nominated by administration to implement Second
Step in their classrooms. This limitation makes causal inferences between intervention group and
improvements in student outcomes difficult to make. The observed effects may be due to specific
dimensions of the Second Step curriculum, teacher characteristics, quality of classroom
environment, effectiveness of behavior management, quality of intervention implementation or
some combination of these and other factors. Nonetheless, there are several reasons for
confidence in the findings. First, students were not assigned to classrooms based on the Second
Step program. Thus, there was no reason to expect that children who received the program were
systematically different from peers in the wait-list comparison classrooms. Although statistical
differences between groups on baseline social-emotional competence scores were indicated, the
actual mean differences between the groups were small and favored comparison classrooms.
Significant baseline differences in academic achievement scores and demographic factors
between the groups were not found. Data analysis relied on ANCOVA/MANCOVA techniques
that included baseline measurement of outcome variables as covariates to statistically adjust the
posttest means for differences among the intervention groups at pretest.
Secondly, because quality of implementation and performance feedback data were not
collected, it is difficult to determine to what degree Second Step was delivered with fidelity and
improvements in student outcomes can be reasonably attributed to the intervention. There is
ample research linking the effectiveness of universal SEL programs to important implementation
features such as adherence, dosage, preparedness of interventionist, responsiveness of
participants, and degree of content differentiation (Durlak et al., 2011; Low et al., 2015; Low,
Smolkowski, & Cook, 2016). The failure to consider the quality of intervention implementation

94

makes it difficult to arrive at accurate conclusions about the effectiveness of Second Step or to
replicate this study and gain similar results (Gresham, MacMillan, Beebe-Frankenberger, &
Bocian, 2000). The quality of intervention implementation is particularly relevant in the context
of the MTSS framework that assumes intervention integrity when determining student progress
and the need for adjustments in programming. It is important that future research include both
direct and indirect measures of treatment integrity to ensure that student progress can be reliably
traced back to intervention.
Third, interpretation of key findings is limited due to the study’s reliance on existing data
and the inability to consider multiple sources when measuring outcome variables. Determination
of changes in social-emotional competence from pretest to posttest relied solely on teacherratings of individual student functioning. While teacher perceptions of gains in student
competencies are critical to understanding the effect of SEL intervention, use of behavioral
rating scales like the DESSA-SSE have been criticized for potential introduction of non-random
measurement error in assessment scores attributable to rater bias (Elliott, Frey, & Davies, 2015;
Hoyt & Kerns, 1999). Future research should consider multiple sources of information such as
behavioral observation, direct behavior rating, student self-report, peer reports, or parent
indicators to gain a more comprehensive and reliable understanding of social-emotional benefits
of participation in Second Step.
As with limitations related to measurement of SEL outcomes, the current study also
relied solely on student NWEA Measure of Academic Progress (MAP) data when determining
gains in academic achievement following Second Step participation. Although the MAP is
regarded as a valid measure of student academic achievement and is widely used in
benchmarking assessment practices across the country, reliance on the assessment as the sole

95

outcome measure in evaluating the effects SEL intervention across the elementary grades may be
problematic. For example, the benefits associated with Second Step may be more applicable to
the fundamental, rote reading development associated with letter recognition and memorization
skills prevalent in the early elementary grades rather than the more complex, higher level skills
encountered in the upper elementary grades. It may be that the function that the MAP was
designed to serve in educational programming explains the differential gains observed in the
lower grades rather than intervention effects. To avoid misinterpretation of study results, future
research should consider multiple measures of academic achievement such as curriculum-based
measurement, classroom performance, individual achievement testing, or report card grades.
Lastly, the present study considers the impact of SEL intervention on individual students
nested within individual classrooms in one elementary school. Because classmates share the
same teacher and classroom environment, they are likely to be more comparable to each other in
certain areas than to students in other classrooms. This clustering of students within classrooms
can result in non-independence of subjects, a violation of a key assumption of analytic methods
such as ANCOVA. This violation can bias the statistical tests used to identify intervention
effects and lead to incorrect conclusions about the statistical significance of observed
relationships. When analyzing nested data like that in the present study, hierarchical linear
modeling (HLM) is recommended to address the issues associated with non-independence of
subjects by controlling for confounding factors such as teacher and classroom effects
(Raudenbush & Bryk, 2002). Unfortunately, the small number of classrooms under examination
in the present study did not provide sufficient statistical power to utilize HLM methods as
recommended. Therefore, analyses were conducted at the individual level even though students

96

were nested within classroom. For this reason, study results should be interpreted with caution
given the potential for bias.
Future Directions and Implications for Practice
Despite the above-mentioned limitations, results of this study both replicate and extend
what is known about the impact of school-based, SEL programs on student outcomes in several
specific ways. First, the findings extend support for SEL programs beyond highly-controlled
trials to implementation in an authentic practice setting. Few studies have focused on evaluating
the effectiveness of SEL programs under real-world educational conditions (Merrell & Gueldner,
2010). The findings confirm previous research demonstrating the feasibility of teacher-delivered
SEL programs resulting in improvement in student outcomes, particularly academic gains
(Durlak et al., 2011; Payton et al., 2008). Although not a randomized controlled trial, the study is
high in ecological validity and is particularly relevant for urban school settings with limited
access to resources and supports to address the social-emotional and academic needs of general
education students. Future research should continue to explore the transportability of effective
SEL programming to contexts that reflect conditions typically found in school communities.
These efforts should focus on examination of feasible methods for implementing SEL
intervention, measuring implementation fidelity, and monitoring student progress toward
outcome goals.
Secondly, this study responds to the call for SEL research to go beyond examination of
simple main effects to identify factors that might moderate the effect of intervention on student
outcomes (Durlak et al. 2011, Payton et al., 2008). This research provides preliminary
exploration into a person-centered approach to defining risk status through the integration of
academic and social-emotional data. Such practices within an MTSS model of educational

97

programming might produce “profiles” of student risk to assist teams in designing prevention
and intervention programs in a more effective and strategic way. For example, a student
presenting with a profile indicating both social-emotional and academic risk may be at greater
risk and require more intensive intervention based on level of need. Future research should
continue to investigate ways that schools might capitalize on the extensive research base linking
academic performance and social-emotional competence to improve the predictive validity of
universal screening practices and enhance the capacity to effectively match student need to
intervention.
This research also contributes to the growing literature suggesting that, in addition to
serving a preventative role by promoting social-emotional competence for all students, SEL
programs can also promote protective factors for students already demonstrating school
difficulties or in jeopardy of negative outcomes due to school and community risk factors. With
the potential to be implemented across grades and schools within a district, SEL programs may
be particularly effective at ameliorating environmental risk factors, such as limited staff SEL
development or negative school climate, that can interfere with academic performance and
social-emotional development in high risk communities. Much of the SEL research has focused
on student-level outcomes such as behavior rating scales, achievement scores, and direct
observation of classroom behavior. Less attention has been paid to investigating valid and
feasible measurement of the environmental mechanisms that are hypothesized to link SEL
programs to positive school and learning outcomes. Future research focused on the identification
and measurement of environmental protective factors that link SEL programming, and positive
learning outcomes may prove to be particularly beneficial as schools prepare to implement
system-wide SEL programing that is currently being proposed at the state and federal levels.

98

In conclusion, the present study is important in that it provides support for the SEL
framework and the idea that students participating in school-based SEL programs such as Second
Step can demonstrate significant improvements in academic performance and social-emotional
skills (Durlak et al., 2011; Payton et al., 2008). The study also contributes to the growing body of
literature indicating that social-emotional competence and academic achievement are interwoven
and that integrated instruction and support in both areas can maximize student success (Zins &
Elias, 2007). This research links SEL programming to improvements in academic achievement,
providing the necessary justification for evidence-based SEL programming to be fully enmeshed
in the everyday practices and routines of schools.

99

APPENDIX

100

APPENDIX

101

REFERENCES

102

REFERENCES

Aber, J. L., Jones, S. M., Brown, J. L., Chaudry, N., & Samples, F. (1998). Resolving conflict
creatively: Evaluating the developmental effects of a school-based violence prevention
program in neighborhood and classroom context. Development and Psychopathology,
10(02), 187-213. doi:10.1017/S0954579498001576
Adelman, H. S., & Taylor, L. (2003). Toward a comprehensive policy vision for mental health in
schools. In M. D. Weist, S. W. Evans, & N. A. Lever (Eds.), Handbook of school mental
health advancing practice and research (pp. 23-43). New York: Kluwer
Academic/Plenum.
Albers, C. A., Glover, T. A., & Kratochwill, T. R. (2007). Where are we, and where do we go
now?: Universal screening for enhanced educational and mental health
outcomes. Journal of School Psychology, 45(2), 257-263. doi:10.1016/j.jsp.2006.12.003
Arsenio, W. F., Cooperman, S., & Lover, A. (2000). Affective predictors of preschoolers’
aggression and peer acceptance: Direct and indirect effects. Developmental Psychology,
36, 438-448.
Barkley, R. A. (1997). ADHD and the nature of self-control. New York: Guilford Publications.
Barkley, R. A. (2004). Attention-deficit/hyperactivity disorder and self-regulation: Taking an
evolutionary perspective on executive functioning. In R. F. Baumeister & K. D. Vohs
(Eds.), Handbook of self-regulation: Research, theory, and applications (pp. 301-323).
New York: Guilford Press.
Batsche, G. M., Castillo, J. M., Dixon, D. N., & Forde, S. (2008). Best practices in designing,
implementing, and evaluating quality interventions. In A. Thomas & J. Grimes (Eds.),
Best practices in school psychology V, (pp. 177-193). Bethesda, MD: National
Association of School Psychologists.
Belfield, C. R., Nores, M., Barnett, S. W., & Schweinhart, L. (2006). Cost-benefit analysis of the
High Scope Perry Preschool Program using age 40 follow-up data. Journal of Human
Resources, 41, 283-301.
Blair, C. (2002). School readiness: Integrating cognition and emotion in a neurobiological
conceptualization of children’s functioning at school entry. American Psychologist, 57,
111-117.
Blair, C., & Razza, R. P. (2007). Relating effortful control, executive function, and false belief
understanding to emerging math and literacy ability in kindergarten. Child Development,
78(2), 647-663.

103

Blum, R. W., Libbey, H. P., Bishop, J. H., & Bishop, M. (2004). School connectedness–
strengthening health and education outcomes for teenagers. Journal of School
Health, 74(7), 231-235. doi:10.1111/j.1746-1561.2004.tb08278.x
Bosacki, S. L., & Moore, C. (2004). Preschoolers' understanding of simple and complex
emotions: Links with gender and language. Sex Roles, 50(9-10), 659-675.
doi:10.1023/B:SERS.0000027568.26966.27
Bradshaw, C. P., Bottiani, J. H., Osher, D., & Sugai, G. (2014). The integration of positive
behavioral interventions and supports and social and emotional learning. In Handbook of
school mental health (pp. 101-118). New York: Springer. doi:10.1007/978-1-4614-76245_8
Brady, K. T., Myrick, H., & McElroy, S. (1998). The relationship between substance use
disorders, impulse control disorders, and pathological aggression. American Journal of
Addictions, 7, 221-230.
Brown, J. R., & Dunn, J. (1996). Continuities in emotion understanding from three to six
years. Child Development, 67(3), 789-802. doi:10.2307/1131861
Castillo, J. M., Batche, G. M., Curtis, M. J., Stockslager, K., March, A., & Minch, D. (2010).
Problem Solving/Response to Intervention Evaluation Tool Technical Assistance
Manual. Retrieved March 29, 2014, from
http://floridarti.usf.edu/resources/tools/ta_manual/index.heml
Cohen, D., & Strayer, J. (1996). Empathy in conduct-disordered and comparison youth.
Developmental Psychology, 32, 988-998.
Cohen J, 1988. Statistical power analysis for the behavioral sciences. Hillsdale, New Jersey:
Erlbaum.
Collaborative for Academic, Social, and Emotional Learning. (2013). CASEL guide: Effective
social and emotional learning programs. Chicago, IL: Author.
Collaborative for Academic, Social, and Emotional Learning. (2003). Safe and sound: An
educational leader’s guide to evidence-based social and emotional learning
programs. Chicago, IL: Author.
Cook, B. G., Burns, M., Browning-Wright, D., & Gresham, F. M. (2010). Transforming school
psychology in the RTI era: A guide for administrators and school psychologists. Palm
Beach: LRP Publications.
Cook, C. R., Volpe, R. J., & Livanis, A. (2010). Constructing a roadmap for future universal
screening research beyond academics. Assessment for Effective Intervention, 35(4), 197205. doi:10.1177/1534508410379842

104

Crick, N. R., & Dodge, K. A. (1994). A review and reformulation of social informationprocessing mechanisms in children's social adjustment. Psychological Bulletin, 115(1),
74. doi:10.1037/0033-2909.115.1.74
Demetriou, A., Christou, C., Spanoudis, G., & Platsidou, M. (2002). Monographs of the Society
for Research in Child Development, 67(1), 1-38.
Denham, S. A., Bassett, H. H., & Zinsser, K. (2012). Early childhood teachers as socializers of
young children’s emotional competence. Early Childhood Education Journal, 40, 137143.
Denham, S. A., McKinley, M., Couchoud, E. A. & Holt, R. (1990), Emotional and behavioral
predictors of preschool peer ratings. Child Development, 61, 1145–1152.
doi:10.1111/j.1467-8624.1990.tb02848.x
Deno, S. L., & Fuchs, L. S. (1987). Developing curriculum-based measurement systems for databased special education problem solving. Focus on Exceptional Children, 19(8), 1-16.
DiPerna, J. C., & Elliot, S. N. (1999). Development and validation of the academic competence
evaluation scales. Journal of Psychoeducational Assessment, 17, 207-255.
Dishion, T. J., & Tipsord, J. M. (2011). Peer contagion in child and adolescent social and
emotional development. Annual Review of Psychology, 62(1), 189-214.
doi:10.1146/annurev.psych.093008.100412
Doll, B., & Cummings, J. (2008). Transforming school mental health services. Bethesda, MD:
National Association of School Psychologists.
Domitrovich, C. E., Gest, S. D., Jones, D., Gill, S., & Derousie, R. M. (2010). Implementation
quality: Lessons learned in the context of the Head Start REDI trial. Early Childhood
Research Quarterly, 25(3), 284-298. doi:10.1016/j.ecresq.2010.04.001
Donohew, L., Zimmerman, R., Cupp, P. S., Novak, S., Colon, S., & Abell, R. (2000). Sensation
seeking, impulsive decision-making, and risky sex: Implications for risk-taking and
design of intervention. Personality and Individual Differences, 28, 1079-1091.
Dowdy, E., Ritchey, K., & Kamphaus, R. W. (2010). School-based screening: A populationbased approach to inform and monitor children’s mental health needs. School Mental
Health, 2(4), 166-176. doi:10.1007/s12310-010-9036-3
Duncan, G. J., Dowsett, C. J., Classens, A., Magnuson, K., Huston, A. C., Klebanov, P., et al.
(2007). School readiness and late achievement. Developmental Psychology 43(6), 14281446.
Durlak, J. A., (2009). How to select, calculate, and interpret effect sizes. Journal of Pediatric
Psychology, 34, 917-928.

105

Durlak, J. A., Weissberg, R. P., Dymnicki, A. B., Taylor, R. D., & Schellinger, K. B. (2011). The
impact of enhancing students’ social and emotional learning: A meta‐analysis of school‐
based universal interventions. Child Development, 82(1), 405-432. doi:10.1111/j.14678624.2010.01564.x
Durlak, J. A., & Wells, A. M. (1997). Primary prevention mental health programs for children
and adolescents: A meta‐analytic review. American Journal of Community
Psychology, 25(2), 115-152. doi:10.1023/A:1024654026646
Eisenberg, N., Cumberland, A., & Spinrad, T. L. (1998). Parental socialization of
emotion. Psychological Inquiry, 9(4), 241-273. doi:10.1207/s15327965pli0904_1
Eisenberg, N., Fabes, R. A., & Losoya, S. (1997). Emotional responding: Regulation, social
correlates, and socialization. In P. Salovey and D. J. Sluyter (Eds.), Emotional
development and emotional intelligence: Educational implications (pp. 129-163). New
York: Basic Books.
Eisenberg, N., Sadovsky, A., Spinrad, T. L., Fabes, R. A., Losoya, S. H., Valiente, C., et al.
(2005). The relations of problem behavior status to children’s negative emotionality,
effortful control, and impulsivity: Concurrent relations and prediction of change.
Developmental Psychology, 41, 193-211.
Elias, M. J., Parker, S., & Rosenblatt, J. L. (2005). Building educational opportunity. In S.
Goldstein & R. B. Brooks (Eds.), Handbook of resilience in children (pp. 315-336). New
York: Kluwer Academic/Plenum. doi:10.1007/0-306-48572-9_19
Elias, M. J., & Tobias, S. E. (1996). Social problem solving interventions in the schools. NY:
Guilford.
Elias, M. J., Zins, J.E., Weissberg, R.P., Frey, K.S., Greenberg, M.T., Haynes, N.M., Kessler, R.,
Schwab-Stone, M.E., & Shriver, T.P. (1997). Promoting social and emotional learning:
Guidelines for educators. Alexandria, VA: ASCD.
Elliott, S. N., Frey, J. R. & Davies, M. (2015). Systems for assessing and improving students’
social skills to achieve academic competence. In J. A. Durlak, C. E. Domitrovich, R. P.
Weissberg, T. P. Gullotta (Eds.), Handbook of social emotional learning: Research and
practice (pp. 301-309). New York, NY: Guilford.
Else-Quest, N. M., Hyde, J. S., Goldsmith, H. H., & Van Hulle, C. A. (2006). Gender differences
in temperament: a meta-analysis. Psychological Bulletin, 132(1), 33. doi:10.1037/00332909.132.1.33
Exemplary & promising safe, disciplined, and drug-free schools programs, 2001 (2002).
Washington, D.C.: U.S. Dept. of Education, Office of Educational Research and
Improvement, Office of Elementary and Secondary Education. Retrieved
from http://www.ed.gov/admins/lead/safety/exemplary01/sddsguid.pdf
106

Fabes, R. A., Eisenberg, N., Karbon, M., Bernzweig, J., Speer, A. L., & Carlo, G. (1994).
Socialization of children’s vicarious emotional responding and prosocial behavior:
Relations with mothers’ perceptions of children’s emotional reactivity. Developmental
Psychology, 30, 44-55.
Faul, F., Erdfelder, E., Lang, A. G., & Buchner, A. (2007). G*Power 3: A flexible statistical
power analysis program for the social, behavioral, and biomedical sciences. Behavior
Research Methods, 39 (2), 175-191. doi: 10.3758/BF03193146
Flanagan, K. S., Bierman, K. L., & Kam, C. M. (2003). Identifying at-risk children at school
entry: The usefulness of multibehavioral problem profiles. Journal of Clinical Child and
Adolescent psychology, 32(3), 396-407. doi:10.1207/S15374424JCCP3203_08
Flook, L., Repetti, R. L., & Ullman, J. B. (2005). Classroom social experiences as predictors of
academic performance. Developmental Psychology, 41(2), 319. doi:10.1037/00121649.41.2.319
Frey, K. S., Nolen, S. B., Edstrom, L. V. S., & Hirschstein, M. K. (2005). Effects of a schoolbased social–emotional competence program: Linking children's goals, attributions, and
behavior. Journal of Applied Developmental Psychology, 26(2), 171-200.
doi:10.1016/j.appdev.2004.12.002
Fuchs, L. S., & Deno, S. L. (1991). Paradigmatic distinctions between instructionally relevant
measurement models. Exceptional Children, 57, 488-501.
Gamm, S., Elliott, J., Halbert, J. W., Price-Baugh, R., Hall, R., Walston, D., Uro, G., & Casserly,
M. (2012). Common core state standards and diverse urban students: Using multi-tiered
systems of support. Washington, DC: Council of Great City Schools.
Garner, P. W., & Waajid, B. (2012). Emotion knowledge and self-regulation as predictors of
preschoolers’ cognitive ability, classroom behavior, and social competence. Journal of
Psychoeducational Assessment, 30(4), 330-343. doi:10.1177/0734282912449441
Good, R. H., & Kaminski, R. A. (2003). Dynamic indicators of basic early literacy skills.
Longmont, CO: Sopris West.
Goodman, R. (2001). Psychometric properties of the strengths and difficulties
questionnaire. Journal of the American Academy of Child and Adolescent Psychiatry,
40(11), 1337-1345. doi:10.1097/00004583-200111000-00015
Graham, S., & Juvonen, J. (1998). Self-blame and peer victimization in middle school: An
attributional analysis. Developmental Psychology, 34(3), 587. doi:10.1037/00121649.34.3.587

107

Greenberg, M. T., Domitrovich, C., & Bumbarger, B. (2001). The prevention of mental disorders
in school-aged children: Current state of the field. Prevention and Treatment, 4(1).
doi:10.1037/1522-3736.4.1.41a
Greenberg, M. T., Kusche, C. A., Cook, E. T., & Quamma, J. P. (1995). Promoting emotional
competence in school-aged children: The effects of the PATHS Curriculum.
Development and Psychopathology, 7, 117-136.
Greenberg, M. T., Weissberg, R. P., O'Brien, M. U., Zins, J. E., Fredericks, L., Resnik, H., &
Elias, M. J. (2003). Enhancing school-based prevention and youth development through
coordinated social, emotional, and academic learning. American Psychologist, 58(6-7),
466. doi:10.1037/0003-066X.58.6-7.466
Gresham, F. M. (2005). Response to intervention: An alternative means of identifying students
as emotionally disturbed. Education and Treatment of Children, 328-344.
Gresham, F. M., MacMillan, D. L., Beebe-Frankenberger, M. E., & Bocian, K. M. (2000).
Treatment integrity in learning disabilities intervention research: Do we really know how
treatments are implemented? Learning Disabilities Research & Practice, 15, 198-205.
Grossman, D. C. (1997). Effectiveness of a violence prevention curriculum among children in
elementary school. The Journal of the American Medical Association, 277(20), 1605.
doi:10.1001/jama.1997.03540440039030
Guerra, N. G., & Bradshaw, C. P. (2008). Linking the prevention of problem behaviors and
positive youth development: Core competencies for positive youth development and risk
prevention. New Directions for Child and Adolescent Development, 2(122), 1-17.
doi:10.1002/cd.225
Hart, B., & Risley, T. R. (1995). Meaningful differences in the everyday experience of young
American children. Baltimore, MD: Paul H. Brookes Publishing Company.
Hawkins, J. D., Farrington, D. P., & Catalano, R. F. (1998). Reducing violence through the
schools, In D. S. Elliot, B. A. Hamburg, & K. R. Williams (Eds.) Violence in American
schools (pp. 188-216). New York, NY: Cambridge University Press.
Hill, C. J., Bloom, H. S., Black, A. R., & Lipsey, M. W. (2007). Empirical benchmarks for
interpreting effect sizes in research. Child Development Perspectives, 2(3), 172-177.
doi:10.1111/j.1750-8606.2008.00061.x
Hjemdal, O., Friborg, O., Stiles, T. C., Rosenvinge, J. H., & Martinussen, M. (2006). Resilience
predicting psychiatric symptoms: A prospective study of protective factors and their role
in adjustment to stressful life events. Clinical Psychology and Psychotherapy, 13(3), 194201. doi:10.1002/cpp.488

108

Hosp, J. L., Howell, K. W., & Hosp, M. K. (2003). Characteristics of behavior rating scales
implications for practice in assessment and behavioral support. Journal of Positive
Behavior Interventions, 5(4), 201-208. doi:10.1177/10983007030050040301
Howse, R. B., Lange, G., Farran, D. C., & Boyles, C. D. (2003). Motivation and self-regulation
as predictors of achievement in economically disadvantaged young children. Journal of
Experimental Education, 71(2), 151-174.
Hoyt, W. T., & Kerns, M. D. (1999). Magnitude and moderators of bias in observer ratings: A
meta-analysis. Psychological Methods, 5(1), 64-86.
Izard, C., Fine, S., Schultz, D., Mostow, A., Ackerman, B., & Youngstrom, E. (2001). Emotion
knowledge as a predictor of social behavior and academic competence in children at risk.
Psychological Science, 12(1), 18-23. doi:10.1111/1467-9280.00304
Jenkins, J. R., Hudson, R. F., & Johnson, E. S. (2007). Screening for at-risk readers in a response
to intervention framework. School Psychology Review, 36(4), 582.
Jimerson, S. R., Burns, M. K., & VanDerHeyden, A. M. (2007). Handbook of response to
intervention: The science and practice of assessment and intervention. New York:
Springer. doi:10.1007/978-0-387-49053-3
Jolliffe, D., & Farrington, D. P. (2004). Empathy and offending: A systematic review and metaanalysis. Aggression and Violent Behavior, 9, 441-476.
Jones, D.E., Greenberg, M., & Crowley, M. (2015). Early social-emotional functioning and
public health: The relationship between kindergarten social competence and future
wellness. American Journal of Public Health, 105(11), 2283-2290.
doi:10.2105/AJPH.2015.302630
Juvonen, J., Nishina, A., & Graham, S. (2000). Peer harassment, psychological adjustment, and
school functioning in early adolescence. Journal of Educational Psychology, 92(2), 349.
doi:10.1037/0022-0663.92.2.349
Kamphaus, R. W., & Reynolds, C. R. (2007). Behavior Assessment System for Children—Second
edition (BASC–2): Behavioral and Emotional Screening System (BESS). Bloomington,
MN: Pearson.
Katsurada, E., & Sugawara, A. I. (1998). The relationship between hostile attributional bias and
aggressive behavior in preschoolers. Early Childhood Research Quarterly, 13(4), 623636.
Kaukiainen, A., Bjorkqvist, K., Lagerspetz, K, Osterman, K., Salmivalli, C., Rothberg, S., et al.
(1999). The relationships between social intelligence, empathy, and three types of
aggression. Aggressive Behavior, 25, 81-89.

109

Kroesbergen, E. H., Van Luit, J. H., Van Lieshout, E. M., Van Loosbroek, E., & Van de Rijit, B.
M. (2009). Individual differences in early numeracy: The role of executive functions in
subittzing, Journal of Psychoeducational Assessment, 27, 226-236
Lane, K. L., & Menzies, H. M. (2003). A school-wide intervention with primary and secondary
levels of support for elementary students: Outcomes and considerations. Education and
Treatment of Children, 431-451.
LeBuffe, P. A., Naglieri, J. A., & Shapiro, V. B. (2011). The Devereux Student Strengths
Assessment–Second Step Edition (DESSA-SSE). Lewisville, NC: Kaplan.
LeBuffe, P. A., Ross, K. M., Fleming, J. L., & Naglieri, J. A. (2013). The Devereux Suite:
Assessing and promoting resilience in children ages 1 month to 14 years. In Resilience in
children, adolescents, and adults (pp. 45-59). Springer: New York. doi:10.1007/978-14614-4939-3_4
LeBuffe, P. A., Shapiro, V. B., & Naglieri, J. A. (2009). Devereux Student Strengths
Assessment. Lewisville, NC: Kaplan.
Levitt, J. M., Saka, N., Romanelli, L. H., & Hoagwood, K. (2007). Early identification of mental
health problems in schools: The status of instrumentation. Journal of School
Psychology, 45(2), 163-191. doi:10.1016/j.jsp.2006.11.005
Low, S., Cook, C. R., Smolkowski, K., & Buntain-Ricklefs, J. (2015). Promoting social–
emotional competence: An evaluation of the elementary version of Second
Step®. Journal of School Psychology, 53(6), 463-477. doi:10.1016/j.jsp.2015.09.002
Malecki, C. K., & Elliot, S. N. (2002). Children's social behaviors as predictors of academic
achievement: A longitudinal analysis. School Psychology Quarterly, 17(1), 1.
doi:10.1521/scpq.17.1.1.19902
Masten, A. S. (1994). Resilience in individual development: Successful adaptation despite risk
and adversity. In M. C. Wang, E. W. Gordon (Eds.). Educational resilience in inner-city
America: Challenges and prospects (3-25). Hillsdale, NJ: Erlbaum
Masten, A. S., & Coatsworth, J. D. (1998). The development of competence in favorable and
unfavorable environments: Lessons from research on successful children. American
Psychologist, 53(2), 205. doi:10.1037/0003-066X.53.2.205
Mayer, J. D., & Salovey, P. (1997). What is emotional intelligence? In P. Salovey & D. J.
Sluyter (Eds.), Emotional development and emotional intelligence: Educational
Implications (pp. 3-31). New York: Basic Books.
McIntosh, K., Chard, D. J., Boland, J. B., & Horner, R. H. (2006). Demonstration of combined
efforts in school-wide academic and behavioral systems and incidence of reading and

110

behavior challenges in early elementary grades. Journal of Positive Behavior
Interventions, 8(3), 146-154. doi:10.1177/10983007060080030301
McClelland, M., Acock, A. C., & Morrison, F. J. (2006). The impact of kindergarten learningrelated social skills on academic achievement at the end of elementary school. Early
Childhood Research Quarterly, 21, 471-490.
Merrell, K. W. (1993). School social behavior scales: Test manual. Brandon, VT: Clinical
Psychology Pub.
Merrell, K. W., Juskelis, M. P., Tran, O. K., & Buchanan, R. (2008). Social and emotional
learning in the classroom: Evaluation of strong kids and strong teens on students' socialemotional knowledge and symptoms. Journal of Applied School Psychology, 24(2), 209224. doi:10.1080/15377900802089981
Merrell, K. W., Cohn, B. P., & Tom, K. M. (2011). Development and validation of a teacher
report measure for assessing social-emotional strengths of children and
adolescents. School Psychology Review, 40(2), 226.
Merrell, K. W., Ervin, R. A., & Peacock, G. G. (2012). School psychology for the 21st century:
Foundations and practices. New York: Guilford Press.
Merrell, K. W., & Gueldner, B. A. (2010). Social and emotional learning in the classroom:
Promoting mental health and academic success. New York: Guilford Press.
Miller, S., Malone, P. S., Dodge, K. A., & Conduct Problems Prevention Research Group.
(2010). Developmental trajectories of boys’ and girls’ delinquency: Sex differences and
links to later adolescent outcomes. Journal of Abnormal Child Psychology, 38(7), 10211032. doi:10.1007/s10802-010-9430-1
Muenning, P., Schweinhart, L., Montie, J., & Neidell, M. (2008). Effects of a prekindergarten
educational intervention on adult health: 37-year follow-up results of a randomized
controlled trial. American Journal of Public Health, 99(8), 1431-1437.
Naglieri, J. A., LeBuffe, P., & Shapiro, V. B. (2011). Universal screening for social–emotional
competencies: A study of the reliability and validity of the DESSA‐mini. Psychology in
the Schools, 48(7), 660-671. doi:10.1002/pits.20586
Nastasi, B. K., Moore, R. B., & Varjas, K. M. (2004). School-based mental health services:
Creating comprehensive and culturally specific programs. Washington, DC: American
Psychological Association. doi:10.1037/10724-000
National Registry of Evidence-based Programs and Practices (2009). Retrieved from
http://nrepp.samhsa.gov/

111

Nese, R. N., Doerner, E., Romer, N., Kaye, N. C., Merrell, K. W., & Tom, K. M. (2012). Social
emotional assets and resilience scales: Development of a strength-based short-form
behavior rating scale system. Journal for Educational Research Online, 4(1), 124.
Noble, K. G., Norman, M. F., & Farah, M. J. (2005). Neurocognitive correlates of
socioeconomic status in kindergarten children. Developmental Science, 8(1), 74-87. doi:
10.111/j.1467-7687.2005.00394.x
Northwest Evaluation Association. (2009). Technical manual for measures of academic progress
and measures of academic progress for primary grades. Lake Oswego, OR: Author.
Model Programs Guide, Office of Juvenile Justice and Delinquency Prevention (n.d.). Retrieved
from https://www.ncjrs.gov/pdffiles1/ojjdp/187079.pdf
Osher, D., Sprague, J., Weissberg, R. P., Axelrod, J., Keenan, S., Kendziora, K., & Zins, J. E.
(2008). A comprehensive approach to promoting social, emotional, and academic growth
in contemporary schools. In A. Thomas & J. Grimes (Eds.), Best practices in school
psychology, (pp. 1263-1278). Bethesda, MD: National Association of School
Psychologists.
Payton, J., Weissberg, R. P., Durlak, J. A., Dymnicki, A. B., Taylor, R. D., Schellinger, K. B., &
Pachan, M. (2008). The positive impact of social and emotional learning for kindergarten
to eighth-grade students: Findings from three scientific reviews. Retrieved from
http://www.casel.org/library/2013/11/1/the-positive-impact-of-social-and-emotionallearning-for-kindergarten-to-eighth-grade-students
Pianta, R. C., Hamre, B. K., & Allen, J. P. (2012). Teacher-student relationships and
engagement: Conceptualizing, measuring, and improving the capacity of classroom
interactions. In S. Christenson, A. L. Reschly, & C. Wylie (Eds.), Handbook of research
on student engagement (pp. 365-386). New York: Springer. doi:10.1007/978-1-46142018-7_17
Ponitz, C. C., McClelland, M. M., Matthews, J. S., & Morrison, F. J. (2009). A structured
observation of behavioral self-regulation and its contribution kindergarten outcomes.
Developmental Psychology, 45, 605-619.
Prince-Embury, S. (2007). Resiliency scales for adolescents: A profile of personal strengths.
Texas: Harcourt Assessment.
Public Schools Review, LLC, n.d. Public School Review. http://www.publicschoolreview.com/
Raudenbush, S. W., & Bryk A. S., (2002). Hierarchical linear models: Applications and data
analysis methods. Thousand Oaks, CA: Sage Publications, Inc.

112

Raver, C. C., & Knitzer, J. (2002). Ready to enter: What research tells policymakers about
strategies to promote social and emotional school readiness among three-and four-yearolds. Retrieved from http://academiccommons.columbia.edu/catalog/ac:127551
Rennie, D., Bull, R., & Diamond, A. (2004). Youth violence perpetration: What protects? What
predicts? Findings from the national longitudinal study of adolescent health. Journal of
Adolescent Health, 35, 424e1-e10.
Rimm-Kaufman, S. E., & Hulleman, C. S. (2015). Social and emotional learning in elementary
school settings: Identifying mechanisms that matter. In J. A. Durlak (Ed.), The handbook
of social and emotional learning: Research and practice, (pp. 151-166). New York:
Guilford Press.
Rimm-Kaufman, S. E., Lasen, R. A. A., Baroody, A. E., Curby, T. W., Ko, M., Thomas, J. B.,
Merritt, E. G., Abry, T., & DeCoster, J. (2014). Efficacy of the Responsive Classroom
Approach: Results from a 3-year longitudinal randomized controlled trial. American
Educational Research Journal, 20(10), 1-37.
Rimm-Kaufman, S. E., Pianta, R. C., & Cox, M. J. (2000). Teachers’ judgments of problems in
the transition to kindergarten. Early Childhood Research Quarterly, 15(2), 147-166.
doi:10.1016/S0885-2006(00)00049-1
Roeser, R. W., Eccles, J. S., & Freedman-Doan, C. (1999). Academic functioning and mental
health in adolescence: Patterns, progressions, and routes from childhood. Journal of
Adolescent Research, 14(2), 135-174. doi: 10.1177/0743558499142002
Romer, D., & Mcintosh, M. (2005). The roles and perspectives of school mental health
professionals in promoting adolescent mental health. What we know and what we don't
know. A research agenda for improving the mental health of our youth treating and
preventing adolescent mental health disorders, 597-615.
doi:10.1093/9780195173642.003.0032
Rones, M., & Hoagwood, K. (2000). School-based mental health services: A research
review. Clinical Child and Family Psychology Review, 3(4), 223-241.
doi:10.1023/A:1026425104386
Salvia, J., Bolt, S. E., & Ysseldyke, J. E. (2010). Assessment in special and inclusive education.
Belmont, CA: Wadsworth/Cencage Learning.
Saarni, C. (1997). Emotional competence and self-regulation in childhood. In P. Salovey and D.
J. Sluyter (Eds.), Emotional development and emotional intelligence: Educational
implications (pp. 35-66). New York: Basic Books.
Seifer, R., LaGrasse, L. L., Lester, B., Bauer, C. R., Shankaran, S., Bada, H. S., et al. (2004).
Attachment status in children prenatally exposed to cocaine and other substances. Child
Development, 75, 850-868.

113

Severson, H. H., Walker, H. M., Hope-Doolittle, J., Kratochwill, T. R., & Gresham, F. M.
(2007). Proactive, early screening to detect behaviorally at-risk students: Issues,
approaches, emerging innovations, and professional practices. Journal of School
Psychology, 45(2), 193-223. doi:10.1016/j.jsp.2006.11.003
Shure, M. B., & Spivack, G. (1980). Interpersonal problem solving as a mediator of behavioral
adjustment in preschool and kindergarten children. Journal of Applied Developmental
Psychology, 1, 29-44.
Shure, M. B., & Spivack, G. (1982). Interpersonal problem-solving in young children: A
cognitive approach to prevention. American Journal of Community Psychology, 10, 341356.
Simons, J. S., Carey, K. B., & Gaher, R. M. (2004). Lability and impulsivity synergistically
increase risk for alcohol-related problems. The American Journal of Drug and Alcohol
Abuse, 30, 685-694.
Sugai, G., & Horner, R. H. (2009). Responsiveness-to-intervention and school-wide positive
behavior supports: Integration of multi-tiered system approaches. Exceptionality, 17(4),
223-237. doi:10.1080/09362830903235375
Suldo, S. M., & Shaffer, E. J. (2008). Looking beyond psychopathology: The dual-factor model
of mental health in youth. School Psychology Review, 37(1), 52.
Tabachnick, B. G., & Fidell, L. S. (2013). Using multivariate statistics, 6th Edition. Boston:
Pearson Education.
Taub, J. (2001). Evaluation of the Second Step Violence Prevention Program at a rural
elementary school. School Psychology Review, 31, 186-200.
Tilly, W. D. (2008). The evolution of school psychology to science-based practice: Problem
solving and the three-tiered model. In A. Thomas & J. Grimes (Eds.), Best practices in
school psychology, (pp. 17-36). Bethesda, MD: National Association of School
Psychologists.
Tilly, W. D., Reschly, D. J., & Grimes, J. (1999). Disability determination in problem solving
systems: Conceptual foundations and critical components. In D. J. Reschly, W. D. Tilly,
& J. Grimes (Eds), Special education in transition: Functional assessment and
noncategorical programming, (pp. 221-251). Longmont, CO: Sopris West.
Tolan, P. H., & Guerra, N. G. (1994). Prevention of delinquency: Current status and issues.
Applied and Preventive Psychology, 3, 251-273.
Underwood, M. K., Coie, J. D., & Herbsman, C. R. (1992). Display rules for anger and
aggression in school-aged children. Child Development, 63, 366-380.

114

Valiente, C., Lemery-Chalfant, K., Swanson, J., & Reiser, M. (2008). Prediction of children's
academic competence from their effortful control, relationships, and classroom
participation. Journal of Educational Psychology, 100(1), 67. doi:10.1037/00220663.100.1.67
van Buuren, S. (2012). Flexible imputation of missing data. Boca Raton, FL: CRC Press.
VanDerHeyden, A. M., Witt, J. C., & Gilbertson, D. (2007). A multi-year evaluation of the
effects of a response to intervention (RTI) model on identification of children for special
education. Journal of School Psychology,45(2), 225-256. doi:10.1016/j.jsp.2006.11.004
Vitaro, F., Brendgen, M., Larose, S., & Tremblay, R. E. (2005). Kindergarten disruptive
behaviors, protective factors, and educational achievement by early adulthood. Journal of
Educational Psychology, 97(4), 617-629.
Vitaro, F., Ferland, F., Jacques, C., & Ladouceur, R. (1998). Gambling, substance use, and
impulsivity during adolescence. Psychology of Addictive Behaviors, 12(3), 185-194.
Wang, M. C., Haertel, G. D., & Walberg, H. J. (1997). Learning influences. In H. J. Walberg &
G. D. Haertel (Eds.), Psychology and educational practice (pp. 199-211). Berkeley, CA:
McCutchan.
Weist, M. D., Rubin, M., Moore, E., Adelsheim, S., & Wrobel, G. (2007). Mental health
screening in schools. Journal of School Health, 77(2), 53-58. doi:10.1111/j.17461561.2007.00167.x
Weisz, J. R., Sandler, I. N., Durlak, J. A., & Anton, B. S. (2005). Promoting and protecting youth
mental health through evidence-based prevention and treatment. American
Psychologist, 60(6), 628. doi:10.1037/0003-066X.60.6.628
Welsh, M., Parke, R. D., Widaman, K., & O'Neil, R. (2001). Linkages between children's social
and academic competence: A longitudinal analysis. Journal of School Psychology, 39(6),
463-482. doi:10.1016/S0022-4405(01)00084-X
Wentzel, K. R. (1991). Relations between social competence and academic achievement in early
adolescence. Child Development, 62, 1066-1078.
Wentzel, K. R. (1993). Does being good make the grade? Social behavior and academic
competence in middle school. Journal of Educational Psychology, 85(2), 357.
doi:10.1037/0022-0663.85.2.357
Ysseldyke, J., et al. (2006). School psychology: A blueprint for training and practice III.
Bethesda, MD: National Association of School Psychologists.
Zins, J.E., Bloodworth, M.R., Weissberg, R.P., & Walberg, H.J. (2004). The scientific base
linking social and emotional learning to school success. In J.E. Zins, R.P. Weissberg,

115

M.C. Wang & H.J. Walber (Eds.), Building academic success on social and emotional
learning: What does the research say (pp. 3-22). New York, NY: Teachers College Press.
Zins, J.E., & Elias, M.J. (2006). Social and emotional learning. In G.G. Bear & K.M. Minke
(Eds.) Children's needs III: Development, prevention, and intervention (pp. 113). Bethesda, MD: National Association of School Psychologists.
Zins, J. E., & Elias, M. J. (2007). Social and emotional learning: Promoting the development of
all students. Journal of Educational and Psychological Consultation, 17(2-3), 233-255.
doi:10.1080/10474410701413152
Zigler, E. F., & Bishop-Josef, S. J. (2006). The cognitive child versus the whole child: Lessons
from 40 years of Head Start. In D. G. Singer, R. M. Golinkoff, & K. Hirsh-Pasek
(Eds.), Play= learning: How play motivates and enhances children’s cognitive and
social-emotional growth, (pp. 15-35). doi:10.1093/acprof:oso/9780195304381.003.0002

116