Examining the Impacts of Middle School Disciplinary Policies on Ninth-Grade Retention

Abstract

We estimate the relationship between exclusionary discipline given in eighth-grade and the probability of ninth-grade retention. We use a rich 7-year, student-level, panel data set from Arkansas. We use a novel approach by limiting our sample to students who switch schools between eighth and ninth grades. This movement gives each student a fresh start and removes the potential confound of a student’s reputation as a “problem student” that could influence teachers to be harsher on students who already have a disciplinary record. Further, we control for student infractions in eighth-grade, to focus on the relationship between the exclusionary consequence (rather than the precipitating behavior) and future academic success. We find that students who receive exclusionary discipline in eighth grade are more likely to be retained in ninth-grade compared with similar students whose infractions did not result in exclusionary discipline. Moreover, we find that the likelihood of ninth-grade retention increases with the number of days of exclusionary discipline. We, however, did not find any statistically significant subgroup differences.

Keywords

school discipline exclusionary discipline grade retention impacts educational policy

Introduction and Research Questions

Exclusionary discipline has become a highly debated issue within K-12 education across the United States for numerous reasons. Both in-school suspension (ISS) and out-of-school suspension (OSS) have been shown to be associated with lower levels of student academic achievement and increased rates of dropping out before completing high school (Noltemeyer & Ward, 2015). However, the causal connection between school discipline and student outcomes is unclear—Does exclusionary discipline cause students to miss class time, feel alienated, perform worse, become demoralized, and drop out? Or do low-achieving students act out, prompting a disciplinary response, followed by continued low academic performance?

These mechanisms are difficult to disentangle, particularly when attempting to look at medium- and long-term outcomes for students who spend multiple years on the same school campus. It is certainly possible that a student’s disciplinary record may act as a signal to other teachers that a student deserves extra scrutiny and causes them to discipline the student for behaviors that would be tolerated in other children. In this article, we examine students at a critical juncture in their academic career to estimate the relationship between exclusionary discipline and future school performance while dealing with possible issues of reverse causality. Specifically, we consider junior high school students in Grade 8 with similar behavioral records and academic records; despite the similarities, some of these students received exclusionary discipline, defined as OSS, expulsion, or placement in an alternative learning environment (ALE), whereas some did not. Given this variation, we will investigate whether the exclusionary discipline affected academic performance in Grade 9. Finally, rather than measuring academic performance based on standardized test scores, we will use on-time promotion to Grade 10 as our outcome measure.

We use a rich 7-year student-level panel data set from the state of Arkansas from 2008-2009 to 2014-2015. We hypothesize that students who receive exclusionary discipline in eighth grade are more likely to be retained in ninth grade, controlling for other student characteristics and predictors of grade retention. We focus on the relationship between exclusionary discipline given in eighth grade and retention in ninth grade because many students switch schools between these grades. Limiting our sample to students who switch schools removes any potential bias created by a student’s past disciplinary record on the decision to retain a student in the following year. This analytic strategy allows for a focus on the independent longer-term effects of exclusionary discipline, not associated with the reputational effects of a student’s past disciplinary record.

Literature Review

In this section, we briefly summarize the literature on the impacts of exclusionary discipline on student outcomes, focusing specifically on the relationship between exclusionary discipline and student academic achievement as we believe this may illuminate our outcome of interest, in-grade retention. Furthermore, because grade retention is a relatively unused outcome measure, we will describe the literature examining the potential harmful impacts of grade retention in the first year of high school.

Exclusionary Discipline

Exclusionary discipline has become a highly debated issue in education. Exclusionary discipline has increased as zero-tolerance policies have spread throughout many schools in the past 20 years. Zero-tolerance policies were designed to dissuade students from acting out in fear of the consequences. Many students receive exclusionary discipline under zero tolerance. These policies have resulted in an increased number of students who are out of school for disciplinary actions as well as increased students’ contact with law enforcement (Skiba, 2014). It is important to note that exclusionary discipline is not exclusively used for violent or other major student infractions. It is commonly used for relatively minor nonviolent infractions such as insubordination (Mendez, Knoff, & Ferron, 2002). In Arkansas, the majority of infractions are minor nonviolent infractions.

Exclusionary discipline can affect student academic achievement. In a meta-analysis summarizing 34 different studies, Noltemeyer and Ward (2015) found an inverse relationship between suspensions and academic achievement and a positive relationship between suspensions and a student dropping out of school. They found that ISS was associated with a 0.10 standard deviation decrease in achievement, and OSS was associated with a 0.24 standard deviation decrease in achievement. OSS was also associated with a 0.25 standard deviation increase in the likelihood of a student dropping out. Similarly, using data from North Carolina, Beck and Muschkin (2012) found that the number of infractions a student received was associated with a decrease in achievement. They found that each additional infraction was associated with a 0.02 standard deviation decrease in achievement for White students and a 0.06 standard deviation decrease in achievement for Black students. While these results are suggestive of a negative relationship between exclusionary discipline and academic achievement, the majority of studies are simply correlational and do not prove the direction the relationship. It is not clear whether exclusionary discipline leads to worse academic outcomes, or whether low academic achievement leads students to misbehave at school.

Worryingly, racial and gender disparities exist between students who receive and who do not receive exclusionary discipline. Using national data, Losen, Hodson, Keith, Morrison, and Belway (2015) found that OSS rates have been increasing broadly, but they have been dramatically increasing for Black students compared with their White and Hispanic counterparts. Specifically, Losen et al. (2015) found that Black students are 3 times more likely to be suspended. Losen and Skiba (2010) found similar results using data from 18 large urban districts. They found that Black male students were suspended at twice the rate of their White male counterparts. Using district-level data from Florida, Mendez (2003) and Mendez et al. (2002) found that male and female Black students were disproportionally suspended, and Black students who also qualified for free or reduced-price lunch (FRL) were suspended more often than Black students who did not qualify for FRL. Similarly, in Arkansas, Anderson and Ritter (2018) found that students of color and students eligible for FRL are more likely to face exclusionary discipline than their White and higher income peers, respectively, even after accounting for infraction type.

The existing literature suggests that receiving exclusionary discipline is related to diminished student academic achievement, and that disadvantaged students (particularly Black students and students who receive FRL) are disproportionately likely to face exclusionary discipline. We hypothesize that exclusionary discipline also hurts students’ noncognitive skills and engagement with school, which puts them at risk for in-grade retention. Furthermore, we anticipate these policies will be most harmful to disadvantaged students, as they are less likely than more advantaged students to have a support system that could help them rebound from the experience of being sent out of school, or that could advocate for on-time promotion with a teacher or administrator.

Grade Retention

We briefly summarize the literature on grade retention because of its close relation with academic achievement. We use grade retention as our outcome of interest as an alternative measure of academic achievement. Grade retention is a highly debated policy aimed at helping students who are performing below grade level. The goal of retention is to keep students in the same grade for an additional year so that they can gain the necessary knowledge and skills to succeed in subsequent years. Arguably, to provide a meaningful education, students should be retained to help them succeed in later grades as well after they leave school. A body of literature, primarily looking at retention in early grades, shows that grade retention does not harm students in the long term and can help increase achievement (Greene & Winters, 2007; Jacob & Lefgren, 2004; Schwerdt & West, 2012).

In contrast, there is also a body of research showing that grade retention can harm student achievement and long-term outcomes (Holmes, 1989; Jimerson, 1999, 2001). Retained students may face additional social pressures and stigmatization by peers and teachers. As a result, retained students disengage from school and continue to struggle academically. Jimerson (2001) summarized three meta-analyses examining the effects of grade retention on academic and socioemotional outcomes. All three meta-analyses find a negative relationship between grade retention and academic and socioemotional outcomes. The analyses, however, do not distinguish between grade retention in high and lower grades. In another systematic review, Jimerson, Anderson, and Whipple (2002) analyzed 17 studies that look at the possible long-term effects of grade retention. They find that retention in early and later grades is one of the strongest predictors of a student dropping out.

In addition to analyzing the impact of grade retention policies, researchers have examined the relationship between school discipline and grade retention. There are competing theories of how discipline and retention affect each other. First, it is possible that students who are retained misbehave due to social pressures and an associated stigma of being held back in a grade. Students could feel alienated because they are older than other students in their grade and misbehave as a result. Another theory is that students who receive exclusionary discipline are more likely to face retention because they miss class time, which harms their achievement. Students could also feel that they were punished unfairly and disengage from school, which in turn harms their achievement and leads to a higher probability of retention.

It is difficult to disentangle the relationship between discipline and academic achievement generally (and grade retention in particular) because there is a possible reciprocal relationship. Using regression discontinuity, Ozek (2015) found that grade retention in early grades has a significant positive effect on disciplinary actions. He concluded that students who were just below the cutoff for grade promotion were more likely to be suspended in the 2 years following retention. These results were more prevalent among Black, male, and economically disadvantaged students, but faded out within 3 years. Muschkin, Glennie, and Beck (2014) uncovered a similar relationship using data of seventh graders in North Carolina. The authors found that retained students had negative peer effects; a 1% increase in the number of retained students in a school increased the odds of an individual student misbehaving by 3%.

By focusing on student retention, this article expands the literature on the relationship between exclusionary discipline and academic achievement. We hypothesize that students who receive exclusionary discipline will have a higher probability of retention. We are particularly interested in the relationship between exclusionary discipline given in junior high school (specifically eighth grade) and student outcomes in ninth grade, which marks the start of a student’s high school career. Grade retention is a relatively holistic academic measure, in that it represents overall academic progress (rather than simply test scores), and as such is an important outcome for students. Consequently, it is very important that policy makers and school leaders understand the extent of the relationship between exclusionary discipline and grade retention.

We take a novel approach by looking at the probability of retention in ninth grade as a function of exclusionary discipline in eighth grade for students who switch schools between eighth and ninth grades. Limiting our sample to students who switch schools helps eliminate bias created by a stigma attached to students with disciplinary records. We also test whether there is a dosage effect: Do students who receive more days of exclusionary discipline have a higher probability of retention? With this general picture of the impacts of exclusionary discipline and grade retention in mind, we move now to a discussion of how these policies are implemented in Arkansas. As our analysis is restricted to the state of Arkansas, a relatively rural, midsouthern state, the extent to which our results can be generalized to other contexts may be limited.

Discipline and Retention in Arkansas

Arkansas has 18 different infraction categories in which teachers can record student misbehavior, and eight different potential consequences. We group the 18 infractions into five categories: use or possession of drugs and alcohol or weapons and violent, major nonviolent, or minor nonviolent actions. The state provides general disciplinary reporting guidelines, but school districts and individual schools determine specific policies. For example, the state of Arkansas allows corporal punishment; however, not every district uses it. School administrators and teachers have discretion in what behaviors they report as infractions and what consequences they assign. For example, one teacher could cite a student for refusing to complete an assignment, a minor nonviolent infraction, whereas another teacher ignores the behavior. The principal who receives the infraction report on the student may choose not to give a consequence or could assign the student a detention, an ISS, or an OSS. The subjectivity of the process makes it difficult to separate the impact of the consequence from the impact of the contextual factors surrounding the initial behavior, the determination of whether the behavior was an infraction, and the assigned consequence.

Exclusionary discipline is typically regarded as the most severe type of punishment students can receive for an infraction at school, as it removes students from the school’s learning environment and potentially bars them from returning (in the case of expulsion). Even when students are suspended from school for a day, they are often prohibited from participating in certain school events, such as after-school activities or field trips. There is a concern that exclusionary discipline can have long-lasting negative impacts on students. The loss of class time due to exclusionary discipline could set students back academically as students lose access to teachers and other resources when they are barred from the school. In addition, exclusionary discipline quite literally sends the message that students are not welcome at school; this policy could leave students feeling alienated and lead to academic disengagement. Exclusionary discipline could therefore harm students both academically and socioemotionally.

In this article, we examine the relationship between exclusionary discipline, including OSS, expulsion, and ALEs, and ninth-grade retention, an important measure of whether or not a student is on track to complete high school. In our sample of Arkansas eighth and ninth graders who switch schools between eighth and ninth grades, 13% of students received exclusionary discipline for an average of 7.33 days, with a mode of 3 days, and a maximum of 131 days.

Districts also have discretion in determining grade retention. Students who do not pass all end-of-year courses or who are missing credits can be considered for retention. Before making a retention decision, the school principal meets with teachers and parents/guardians to discuss the needs of the student to create an Individualized Academic Improvement Plan. If there is any disagreement regarding whether the student should be retained, the final decision rests with the principal. In our sample of students who switch schools between eighth and ninth grades, 14% of students were retained in ninth grade.

Data

We use a 7-year panel data set which includes all student records in Arkansas from 2008-2009 to 2014-2015, including type of disciplinary infraction and corresponding consequence, demographic characteristics, and academic outcomes. The Arkansas Department of Education collects these data, so our data set includes the entire population of students in Arkansas during this time. We limit our analysis to a subsample of students who switch schools between eighth and ninth grades. Moving between schools gives students who have been labeled as “problem students” due to previous infractions a fresh start. Limiting our analysis to those who switch schools removes the potential bias of school administrators and teachers retaining students based on their prior discipline history.

We still have a large sample after restricting our observations. There are 304,109 eighth and ninth graders in all 7 years of data, with 24,035 eighth and ninth graders attending schools that terminate in eighth grade. To better illustrate the proportion of Arkansas students included in our subsample, in the 2014-2015 school year there were 34,797 ninth graders who also attended eighth grade in Arkansas in 2013-2014. Of the 34,797 ninth graders, 3,963 students attended schools where ninth grade was the lowest grade. Table 1 shows demographic information for various groups of students.

Table 1.

Student Demographics From 2008-2009 to 2014-2015.

	(1)	(2)	(3)	(4)	(5)	(6)	(7)	(8)	(9)
	All students	Eighth- and ninth-grade students	Eighth- and ninth-grade student with exclusionary discipline	Eighth-grade students with exclusionary discipline	Students who switch schools between eighth and ninth	Students who switch school with eighth exclusionary discipline	Ninth-grade students retained	Retained ninth-grade students who switched schools	Students who switch schools with eighth exclusionary discipline and retained in ninth
N	806,762	304,109	39,139	22,877	24,035	3,195	8,685	3,286	722
White	63.9%	65.4%	42.6%	40.4%	49.9%	32.1%	44.2%	36.7%	25.1%
Total minority	35.2%	34.4%	57.4%	59.6%	50.1%	67.9%	55.9%	63.3%	75.1%
Black	21.0%	21.5%	47.3%	50.3%	38.7%	60.9%	43.1%	48.7%	68.1%
Hispanic	10.6%	9.3%	7.6%	6.9%	9.2%	5.2%	10.3%	12.3%	5.5%
Two or more	1.6%	0.9%	1.2%	1.2%	0.9%	1.0%	0.8%	0.9%	0.8%
Other	2.0%	2.6%	1.3%	1.2%	1.3%	0.8%	1.6%	1.4%	0.6%
Pre-ninth retention		1.6%	5.0%	7.0%	2.0%	5.9%	4.3%	3.6%	8.7%
Female	51.2%	48.9%	36.2%	35.4%	40.5%	33.9%	35.0%	34.4%	31.3%
FRL	69.0%	74.4%	81.3%	83.1%	73.2%	88.2%	86.7%	84.7%	92.2%
LEP	7.7%	6.6%	5.0%	4.5%	6.0%	3.4%	7.5%	8.7%	4.5%
Fraction in region
Northwest		34.8%	22.4%	20.0%	26.5%	12.1%	24.6%	23.4%	9.4%
Northeast		20.7%	20.4%	20.6%	16.6%	16.1%	18.5%	14.0%	11.6%
Central		29.0%	39.9%	42.1%	37.9%	53.1%	40.1%	47.7%	62.3%
Southwest		9.7%	9.2%	9.3%	11.7%	13.5%	8.9%	9.5%	12.9%
Southeast		5.8%	8.2%	8.0%	7.3%	5.2%	7.9%	5.4%	3.7%

Note. FRL = free or reduced-price lunch; LEP = limited English proficiency.

The population of students who switch schools (column 5) is generally demographically representative of all eighth and ninth graders in Arkansas (column 2). Among the subsample of students who switch schools, there are roughly equal proportions of students who were previously retained, and who qualify for FRL or limited English proficiency (LEP) services. There is a slightly greater share of Black and female students and a lower fraction of White students. The majority of schools that begin in ninth grade are located in the Central region, which includes Little Rock and other large districts, as opposed to other areas of the state that are more rural and tend to have a single school for Grades 7-12 and that tend to serve higher fractions of White students. We compare all students who were retained in ninth grade with those who were retained and switched schools, in columns 7 and 8. These columns show there are similar proportions of students who have been previously retained and students who qualify for FRL; however, a higher percentage of students of color who switched schools were retained than were students of color who did not switch schools before starting ninth grade. Students who receive exclusionary discipline (column 3) and those who are retained in ninth grade (column 7 and 8) are disproportionately Black, male, and qualify for FRL compared with all eighth and ninth graders (column 2). Students who received exclusionary discipline in eighth grade and students who were retained in ninth grade have also been retained at higher rates in previous grades than their counterparts.

Method

We use a pooled probit model to estimate the probability of a student being retained in ninth grade as a function of exclusionary discipline in eighth grade.¹ We use a probit model² because our outcome variable, ninth-grade retention, is dichotomous. Our model is as follows:

+ β_{1} 8 t h e x c l_d i s c_{i} + τ 8 t h_i n f r a c + β_{3} p r i o r_r e t e n_{i} + \dots + ε_{i}

where $8 t h e x c l_d i s c_{i}$ is a dummy variable if student i received any exclusionary discipline in eighth grade. In a separate model, we substitute any exclusionary discipline for the number of days a student was excluded from school to capture any dosage effect. 8th_infractij is the number of j type infractions student i was written up for in eighth grade.³ $p r i o r_i n f r a c t$ is the total number of infractions student i had in eighth grade and prior. $p r i o r_r e t e n$ is a dummy variable if student i was retained in any grades prior to ninth. $X_{i}$ is a vector of student characteristics, including race, FRL status, LEP status, special education status, and average z score on eighth-grade achievement tests. τs is a vector of school fixed effects. Our outcome variable, $Y_{i}$ , indicates whether student i was retained in ninth grade.

There are two key parts of our analytical strategy. First, we control for the type of behavior. This strategy allows us to capture the effect of exclusionary discipline independent of the student’s behavior. Second, we examine only students who switch schools between eighth and ninth grades. Our theory is that students who do not switch schools face a stigma in their school, with teachers and administrators having categorized them as “problem kids,” which means they are more likely to be written up for the same behavior that is tolerated in other students. By examining only students who switch schools, we remove this source of bias and get closer to a clean estimate of the impact of exclusionary discipline. We also analyze the relationship for subgroups by race, gender, and FRL status by including interactions between each student characteristic and whether or not the student received exclusionary discipline. Finally, as a robustness check, we include time-varying school-level characteristics (share of students receiving free or FRL, share of students of color, and school-level average achievement in math and literacy) in addition to the school fixed effects.⁴

Intuitively, we are comparing two students, each with similar test scores, FRL status, race, and gender. They have the same number and type of infractions in eighth grade, but one student receives exclusionary discipline in eighth grade, whereas the other does not. Thus, it is reasonable to argue that the primary difference between these two otherwise similar students is the imposition of exclusionary discipline in Grade 8 for one of the students. We then ask whether the academic performance of that student in Grade 9, as measured by his probability of retention in Grade 9, is worse for the student who received exclusionary discipline than for the student who did not.

Results

In this section, we present the results of our analysis, using the methods described above. We begin by asking whether students who receive exclusionary discipline are more likely to be retained than similar students who did not receive exclusionary discipline, given a specific infraction history and observable characteristics, and then explore whether certain subgroups of students (students of color and students receiving free or FRL) have differential impacts of such a consequence.

Any Exclusionary Discipline

We begin by looking at the impact of receiving any exclusionary discipline on the likelihood that a student is retained in ninth grade. We present results from our probit model, which is our preferred estimator.⁵ Table 2 shows the impact of receiving at least one day of OSS, ALE, or expulsion in eighth grade on the probability that the student is retained in ninth grade.

Table 2.

Impact of Any Eighth Grade Exclusionary Discipline on Probability of Ninth-Grade Retention (Probit, Marginal Effects).

	(1)	(2)	(3)	(4)
Any exclusionary discipline, Grade 8	0.065***	0.054***	0.047***	0.025***
	(0.020)	(0.017)	(0.015)	(0.009)
No. of infractions in eighth grade, by type
Disorderly conduct	0.015**	0.015**	0.014**	0.014***
	(0.006)	(0.006)	(0.006)	(0.004)
Insubordination	0.017***	0.015***	0.016***	0.011***
	(0.004)	(0.003)	(0.003)	(0.003)
Truancy	0.031***	0.029***	0.030***	0.022***
	(0.010)	(0.010)	(0.009)	(0.008)
Other infractions	−0.003	−0.001	−0.001	−0.002
	(0.005)	(0.004)	(0.004)	(0.004)
Alcohol and other drugs	0.014	0.022	0.028*	0.043***
	(0.015)	(0.015)	(0.015)	(0.015)
Weapons	−0.041	−0.015	−0.017	0.015
	(0.042)	(0.043)	(0.045)	(0.047)
Violent infractions	−0.010	−0.009	−0.005	0.001
	(0.008)	(0.008)	(0.007)	(0.005)
Major nonviolent infractions	0.019	0.019	0.024	0.022
	(0.028)	(0.028)	(0.031)	(0.027)
Total number of infractions, Grade 8 and below	−0.001	−0.002	−0.002	−0.001
	(0.001)	(0.001)	(0.001)	(0.001)
Any prior retention	0.052**	0.041**	0.035**	0.029**
	(0.020)	(0.019)	(0.018)	(0.014)
Eighth-grade standardized math/ELA score		−0.103***	−0.114***	−0.121***
		(0.011)	(0.011)	(0.004)
Female			−0.023***	−0.032***
			(0.007)	(0.007)
Black			0.023	−0.020**
			(0.015)	(0.009)
Other student of color			0.054***	0.004
			(0.015)	(0.009)
FRL			0.027**	0.040***
			(0.011)	(0.011)
Special education			−0.111***	−0.123***
			(0.013)	(0.010)
School effects	No	No	No	Yes
Observations	24,089	20,848	20,843	18,660

Note. Standard errors clustered at district level. ELA = English language arts; FRL = free or reduced-price lunch.

p < .1. **p < .05. ***p < .01.

Column 1 of Table 2 shows the estimated impact of receiving exclusionary discipline in eighth grade on the likelihood of ninth-grade retention after only controlling for the student’s past actions, but without controlling for student or school characteristics. This simple specification indicates that students who face exclusionary discipline in eighth grade are 6.5 percentage points more likely to be retained in ninth grade than students who were not excluded. We see that each additional infraction for disorderly conduct, insubordination, or truancy is associated with an increased likelihood of retention (1.5, 1.7, and 3.1 percentage points, respectively). In contrast, each recorded incidence of violent infractions is associated with a decreased likelihood of retention in ninth grade (1 percentage point). Finally, the category for other infractions has no significant relationship with whether a student is retained in ninth grade. The coefficients on disorderly conduct, insubordination, and truancy indicate that ninth-grade retention is related to students’ observable noncognitive skills, such as self-management, that may also be related to classroom behavior. The negative coefficient on staff assault indicates that teachers may have some subjectivity in deciding whether or not to retain a student, and may be less likely to retain a student whom they do not want in their class again next year. Finally, we see in column 1 that students who have previously been retained (in eighth grade or earlier) are significantly more likely (5.2 percentage points) to be retained in ninth grade than students who have progressed normally.

In column 2 of Table 2, we add in each student’s lagged average standardized math and English language arts (ELA) scores on statewide assessments to control for past achievement and unobservable characteristics that contribute to achievement. As expected, there is a strong relationship between a student’s academic achievement and a student’s likelihood of retention. A 1 standard deviation increase in achievement in eighth grade is associated with a 10.3 percentage point decrease in the probability of retention in ninth grade. Once we control for achievement, our variable of interest, whether the student faced exclusionary discipline in eighth grade, decreases in magnitude, but remains highly significant and substantial in magnitude: Receiving at least one day of exclusionary discipline in eighth grade is associated with a 5.4 percentage point increase in the likelihood of being retained in ninth grade.

In column 3 of Table 2, we add controls for observable student characteristics: gender, race, FRL status, and special education status. These characteristics on their own have predictive power when looking at the likelihood that a student will be held back in ninth grade. However, the likelihood that Black students are held back in ninth grade is indistinguishable from the likelihood that White students are held back in ninth grade after controlling for infraction, consequence, and retention history, as well as past achievement, FRL status, and special education status. Past achievement continues to have the largest predictive power when looking at the likelihood of ninth-grade retention; a 1 standard deviation increase in achievement in eighth grade is associated with an 11.4 percentage point decrease in the likelihood that a student will be retained in ninth grade. Having been retained in a prior year increases the likelihood that a student will be retained in ninth grade by 3.5 percentage points. Our variable of interest, facing exclusionary discipline in eighth grade, remains highly significant and substantial. Receiving at least 1 day of OSS, ALE, or expulsion is associated with a 4.7 percentage point increase in the likelihood that a student is retained in ninth grade, after taking into account infraction, retention, and achievement history, as well as observable student characteristics.

In columns 1 through 3, we exploit variation between students who did and did not face exclusionary discipline in eighth grade, but we do not control for school-level characteristics that may affect the likelihood that a student is held back in ninth grade. Principals, teachers, and even parents have some discretion in determining whether or not a student is promoted to the next grade, and school culture is thus important to take into account when predicting the likelihood that a student will be retained. In column 4, we include school fixed effects to account for these between-school differences. In this specification, we only look at the impact of exclusionary discipline on the likelihood that a student is retained as compared with a student who did not receive exclusionary discipline in eighth grade and attended the same school in ninth grade. We prefer this specification, as it controls for student observed characteristics, unobserved characteristics captured in measures of past achievement, and school characteristics that affect the likelihood of retention independent of student characteristics. The decrease in observations in column 4 reflects the necessity of controlling for school. Enrollment in certain schools in the years examined perfectly predicted ninth-grade retention, and students who attended those schools (2,183) were not included in the estimation.

In our preferred specification, receiving at least 1 day of exclusionary discipline in eighth grade is associated with a 2.5 percentage point increase in the likelihood of ninth-grade retention. Prior retention is also significant and is associated with a 2.9 percentage point increase in the likelihood that a student will be retained in ninth grade. Past academic achievement is the most substantive protective characteristic against retention: a 1 standard deviation in achievement in eighth grade is associated with a 12.1 percentage point decrease in the likelihood that a student will be retained in ninth grade. Females are 3.2 percentage points less likely than males to be retained in ninth grade. There is no statistically significant difference in the likelihood of retention for Black and White students when we look across schools (shown in column 3). However, when we include school fixed effects to compare students within the same school, Black students are two percentage points less likely than White students to be retained (column 4). There is no difference in the likelihood of retention between non-Black students of color and White students within the same school. Students receiving free or reduced price lunch for at least one full year during 2008-2009 to 2014-2015 were 4.0 percentage points more likely to face retention in ninth grade than students who never received free or reduced price lunch. Finally, students with disabilities were 12.3 percentage points less likely to face retention than students without disabilities who attended the same school for their ninth-grade year. As a robustness check, we added time-varying school characteristics to our fully specified model. When we add these variables to the model, we find that receiving exclusionary discipline in eighth grade is associated with a 13.4 percentage point increase in the likelihood of ninth-grade retention, an effect that is statistically significant at the 99% confidence level. This estimate is larger than the coefficient we estimate from our main model, but leads to the same conclusion: Facing exclusionary discipline in eighth grade increases a student’s likelihood of retention in ninth grade. However, these variables are correlated with student-level characteristics and have little variation over time. Both of these issues introduce noise into our estimates, leading us to prefer the specification in which we include a school fixed effect, but not school-level variables. Our full results from models including time-varying school characteristics are available in the appendix.

Days of Exclusionary Discipline

Above, we focused on the impact of ever facing exclusionary discipline in eighth grade on the probability of retention in ninth grade. In Table 3, we expand our variable of interest into four categories for the number of days (1-5 days, 6-10 days, 11-15 days, and 16 or more days) a student was suspended, expelled, or in an ALE during eighth grade. Our outcome is still the probability of ninth-grade retention.

Table 3.

Impact of a Day of Exclusionary Discipline on Probability of Ninth-Grade Retention (Probit, Marginal Effects).

	(1)
1-5 days excluded, Grade 8	0.017*
	(0.009)
6-10 days excluded, Grade 8	0.041***
	(0.014)
11-15 days excluded, Grade 8	0.063***
	(0.018)
16+ days excluded, Grade 8	0.040
	(0.026)
No. of infractions in eighth grade, by type
Disorderly conduct	0.012***
	(0.005)
Insubordination	0.010***
	(0.002)
Truancy	0.022***
	(0.008)
Other infraction	−0.002
	(0.003)
Alcohol and other drugs	0.038**
	(0.016)
Weapons	0.008
	(0.049)
Violent infractions	−0.002
	(0.006)
Major nonviolent infractions	0.017
	(0.029)
Total infractions, Grades 8 and below	−0.001
	(0.001)
Any prior retention	0.029**
	(0.015)
Standardized eighth-grade math/ELA score	−0.121***
	(0.004)
Student characteristics	Yes
School effects	Yes
Observations	18,660

Note. Standard errors clustered at district level. ELA = English language arts.

p < .1. **p < .05. ***p < .01.

As the number of days excluded increases, so does the likelihood of retention. One to 5 days of exclusionary discipline in eighth grade is associated with a 1.7 percentage point increase in the likelihood of ninth-grade retention. Six to 10 days of exclusionary discipline is associated with a 4.1 percentage point increase, and 11 to 15 days is associated with a 6.3 percentage point increase likelihood of being retained in ninth grade. Interestingly, students who received 16 or more days of exclusionary discipline are not statistically more likely to be retained. Our results suggest that there is significant variation in school policies or implementation that explains the relationship between exclusionary discipline and retention within schools. Estimates of other observable student characteristics remain consistent with those in Table 2, when we treated exclusionary discipline as a dichotomous variable.

Subgroup Analyses

Our final analysis looks for differential impacts of exclusionary discipline on particular groups of students. We hypothesized that students of color and students from low-income backgrounds would be more hurt by exclusionary discipline, as they may be less likely to make up for missed class time or are already at risk of feeling marginalized and unwelcome in their school environment, both of which could put students at risk of disengagement, retention, and eventually dropping out. In addition, we test whether there are differential effects of exclusionary discipline by student gender, as prior work suggests that male students are more adversely affected by parental emotional distance than girls (Bertrand & Pan, 2013). This finding suggests the effect of exclusionary discipline is larger for male students than for female students if exclusionary discipline is interpreted as a form of emotional rejection. Table 4 tests these hypotheses. Importantly, we are again controlling for the number and type of infractions in Grade 8 and unobservable student characteristics that might make students more or less prone to retention.

Table 4.

Differential Impact of Any Eighth-Grade Exclusionary Discipline on Probability of Ninth-Grade Retention (Probit, Marginal Effects).

	(1)	(2)	(3)	(4)	(5)	(6)
	Gender	Gender	Race	Race	Poverty	Poverty
Male × Exclusionary Discipline	−0.012	0.020**
	(0.014)	(0.010)
Female × Exclusionary Discipline		0.033**
		(0.013)
Black × Exclusionary Discipline			−0.014	0.022**
			(0.011)	(0.009)
Latino/a × Exclusionary Discipline			−0.029	0.007
			(0.029)	(0.032)
Other Student of Color × Exclusionary			−0.025	0.011
			(0.049)	(0.049)
White × Exclusionary Discipline				0.036***
				(0.012)
FRL × Exclusionary Discipline					−0.033*	0.018*
					(0.018)	(0.009)
Non-FRL × Exclusionary Discipline						0.050***
						(0.020)
Any exclusionary discipline, eighth grade	0.033**		0.036***		0.050***
	(0.013)		(0.012)		(0.019)
Black	−0.020**	−0.020**	−0.018*	−0.018*	−0.025***	−0.025***
	(0.009)	(0.009)	(0.010)	(0.010)	(0.008)	(0.008)
Hispanic	−0.001	−0.001	0.003	0.003	−0.007	−0.007
	(0.010)	(0.010)	(0.010)	(0.010)	(0.009)	(0.009)
Other student of color	0.018	0.018	0.022	0.022	0.014	0.014
	(0.014)	(0.014)	(0.014)	(0.014)	(0.014)	(0.014)
Any prior retention, Grades 8 or below	0.030**	0.030**	0.030**	0.030**	0.028*	0.028*
	(0.014)	(0.014)	(0.014)	(0.014)	(0.014)	(0.014)
Female	−0.035***	−0.035***	−0.032***	−0.032***	−0.033***	−0.033***
	(0.007)	(0.007)	(0.007)	(0.007)	(0.007)	(0.007)
Standardized eighth-grade math/ELA score	−0.121***	−0.121***	−0.121***	−0.121***	−0.121***	−0.121***
	(0.004)	(0.004)	(0.004)	(0.004)	(0.004)	(0.004)
FRL	0.040***	0.040***	0.040***	0.040***	0.069***	0.069***
	(0.011)	(0.011)	(0.011)	(0.011)	(0.008)	(0.008)
SPED	−0.123***	−0.123***	−0.123***	−0.123***	−0.124***	−0.124***
	(0.010)	(0.010)	(0.010)	(0.010)	(0.010)	(0.010)
No. of infractions in eighth grade, by type	Yes	Yes	Yes	Yes	Yes	Yes
Total number of infractions, Grades 8 and below	Yes	Yes	Yes	Yes	Yes	Yes
School effects	Yes	Yes	Yes	Yes	Yes	Yes
Observations	18,643	18,643	18,643	18,643	18,643	18,643

Note. Standard errors clustered at district level. ELA = English language arts; FRL = free or reduced-price lunch; SPED = special education.

p < .1. **p < .05. ***p < .01.

Columns 1 and 2 test the hypothesis that exclusionary discipline differentially affects male and female students; however, we find no evidence that there is a differential effect of exclusionary discipline by gender. In column 2, we see that facing exclusionary discipline in eighth grade is associated with a 2 percentage point increase in the likelihood of ninth-grade retention for male students and a 3.3 percentage point increase in the likelihood of retention for female students; these estimates are not distinguishable from each other (column 1). Consistent with what we have seen in prior models, retention in eighth grade or earlier is a major risk factor for ninth-grade retention, whereas increased achievement in eighth grade is a substantial protective factor.

Columns 3 and 4 explore the differential impact of exclusionary discipline on students of different racial backgrounds. The coefficients can be interpreted as the relationship between exclusionary discipline and the likelihood of retention given a student’s race. For Black students, receiving any exclusionary discipline in eighth grade is associated with a 2.2 percentage point increase in the likelihood of ninth-grade retention. However, for Latino/a students and other students of color, receiving any exclusionary discipline in eighth grade is not significantly associated with the likelihood of ninth-grade retention. Interestingly, this effect is larger for White students; any exclusionary discipline is associated with a 3.6 percentage point increase in the likelihood of ninth-grade retention. However, we see in column 3 that these differences are not statistically significant. The attenuated impact of exclusionary discipline for students of color is not significantly different from the baseline impact of exclusionary discipline for White students.

Columns 5 and 6 test the hypothesis that FRL students face disparate impacts of exclusionary discipline; however, our results do not provide definitive support for this conclusion. Column 5 indicates a significant relationship between exclusionary discipline and retention, but that receiving FRL slightly attenuates the effect of exclusionary discipline relative to the impact on non-FRL students. We discuss this potentially counterintuitive finding at greater length in the next section.

Discussion and Policy Implications

At the beginning of this article, we theorized that receiving exclusionary discipline in eighth grade would make students more likely to face ninth-grade retention. Our proposed underlying mechanism was that students who faced OSS, expulsion, or placement in an ALE would act as a signal to students, indicating that they were unfit or unwelcomed in the regular school environment. This perceived marginalization would lead students to become disengaged in classes, eroding both their academic achievement and required noncognitive skills, such as showing up to class on time, completing work, and engaging in classroom tasks. This behavior would result in retention. In this article, we have examined this relationship between exclusionary discipline and retention.

Our first challenge to identification was disentangling the impact of a teacher’s perception of a student as a “troublemaker” or a “hopeless case” because of past infractions. It could be that if a teacher perceives a student as a problem, that teacher will be less likely to reach out to help that student and more likely to punish or hold back the student. To address this issue, we limited our sample just to students who switch schools between eighth and ninth grades. Thus, the students in our sample have a clean slate with their ninth-grade teachers, and our estimated impacts of exclusionary discipline should not be related to teachers’ attitudes toward students. We also included school fixed effects, to account for time-invariant unobservable school characteristics that affect the likelihood that a student is held back in ninth grade. It could be that some high school principals are more reluctant to retain students than others; if so, failing to include school fixed effects would bias our results. Finally, we controlled for observable student characteristics, including race, gender, FRL status, special education services, prior achievement, and prior retention. By controlling for these characteristics, we are separating out the impact of the intervention, exclusionary discipline, from other factors that affect the likelihood that a student is retained in ninth grade.

Main Results

When we included our full set of controls, we found that students who faced exclusionary discipline in eighth grade were 2.5 percentage points more likely to be retained in ninth grade than students who were not excluded. That is, after we control for the number and type of infractions each student received in eighth grade (so we are comparing students who were cited but did not face exclusionary discipline with students with the same number of citations but were excluded from the school environment), we still find a significant relationship between exclusionary discipline and ninth-grade retention. We also found a strong relationship between the number of days a student is excluded and their likelihood of being retained in ninth grade. The relationship varied from a 1.7 percentage point increase in the likelihood of retention for students who received 1 to 5 days of exclusionary discipline to a 6.3 percentage point increase in the likelihood of retention for students excluded for 11 to 15 days. There was no statistically significant increased probability of retention for students who received 16 or more days when including school fixed effects. All other results were significant at the 95% confidence level and were robust across model specifications. Our results suggest that the use of exclusionary discipline in middle school is associated with worse longer run outcomes for students.

Subgroup Analyses

We initially hypothesized that historically marginalized students (low-income students and students of color) would be more strongly affected by exclusionary discipline than more advantaged students. We theorized that more advantaged students would have greater support systems than disadvantaged students, which would make them more resilient to adverse events, such as exclusionary discipline, thus ameliorating the impact of facing OSS, expulsion, or ALE. However, our results did not support this hypothesis. Instead, we found that the magnitude of the relationship between exclusionary discipline and retention was significantly smaller for historically marginalized students. We found that while exclusionary discipline resulted in a 3.6 percentage point increase in the likelihood of retention for White students, exclusionary discipline resulted in a 2.2 percentage point increase in the likelihood of retention for Black students; however, we found no evidence that these impacts were significantly different from each other. We also found that Black students and Latino/a students who were not excluded were less likely to be retained than White students who were not excluded. While these results initially seemed counterintuitive, we believe they may indicate the presence of different expectations of behavior and achievement for students of different backgrounds.

If teachers hold the highest expectations for advantaged students and believe (consciously or unconsciously) that historically marginalized students can only meet lower standards of conduct and achievement, then we can reasonably expect teachers’ responses to the same behaviors by different students to vary. Let us imagine a White student and a Black student who both face exclusionary discipline in eighth grade. Both students feel alienated from and unwelcomed at school as a result, and in ninth grade both refuse to complete their classwork and instead sit in the back of the room and try to avoid the teacher. If the teacher on some level believes that the White student is capable of more, but the Black student is not, the teacher has a reason to retain the White student but not the Black student. From the teacher’s perspective, she is helping both students; the White student will learn more and catch up to her peers if she is retained, whereas the Black student will not get more out of ninth grade regardless of how much time she spends in the grade. Thus, the teacher believes retaining the Black student is academically pointless and punitive. The same behavior, prompted by the same initial cause of exclusionary discipline in eighth grade, leads to two different outcomes for the Black and White students and different probabilities of retention, on average, across the two groups. Analogous logic holds for our results concerning FRL students. We also examined whether there was a differential relationship between exclusionary discipline and the likelihood of retention by student gender, as prior literature suggested male students may be more affected by signals of emotional distance than female students (Bertrand & Pan, 2013). However, we found no support for this hypothesis in our sample.

Limitations

There are a number of limitations of the current study. First, our data are restricted to a single, largely rural, midsouthern state. Second, by restricting our sample to students who switch schools between eighth and ninth grades, we narrow our focus to relatively larger districts within the state. Our results therefore may not generalize to students in smaller, more rural districts. In addition, our results may not generalize to other regions in the United States. However, this study is one of the first to examine the impact of middle school discipline on an important outcome, ninth-grade retention, which combines achievement and noncognitive skills. Researchers should attempt to replicate our findings in other contexts to determine whether or not our findings generalize to other contexts.

As we include additional controls in our analysis, we restrict our analytic sample. Our preferred model, which includes student achievement in eighth grade, student demographics, and school effects, includes 77.5% of our original sample. Students who did not face exclusionary discipline in eighth grade are more likely to be lost from this sample than are students who did face exclusionary discipline in eighth grade. Specifically, we include 75.3% of students who did not face exclusionary discipline in our fully specified model and 91.8% of students who were excluded. These differential rates of missing data could introduce bias into our results, limiting our ability to estimate a causal impact of exclusionary discipline on ninth-grade retention.

The subjectivity inherent in the school disciplinary process further restricts our ability to make causal inferences. Our data are drawn from administrative records; behind those records are a series of actions and interpretations: a student behaves in a certain manner, an adult interprets that behavior to be an infraction or not, and then another authority figure determines what consequence to attach to that infraction. The subjectivity required on the part of adults to interpret students’ behavior and attach consequences makes it difficult to disentangle the impact of the consequence from the impact of the context that influenced the behavior, the interpretation of that behavior, and the eventual consequence. We address this in part by only comparing students within the same school who were cited for the same infraction, but who were given different consequences. By controlling for the number and type of past infractions, prior achievement, and other student characteristics, we hope to control for contextual factors that influence students’ behavior and how school personnel interpret students’ behavior. There is also subjectivity in our outcome variable, ninth-grade retention, as schools have considerable discretion when determining which students are retained. By including school fixed effects, we only compare students who are subject to the same schoolwide attitudes toward retention, which should account for most of this subjectivity. However, there could also be time-varying school-level unobserved factors that affect the likelihood of whether or not a student is retained that we do not account for in this analysis. Therefore, we cannot make causal claims about the relationship between eighth-grade exclusionary discipline and ninth-grade retention that we examine in this article.

Policy Implications and Future Research

Our results suggest two areas of concern: first, that exclusionary discipline has persistent negative impacts on student outcomes, and, second, that the impacts of exclusionary discipline could be mediated by teachers’ attitudes toward students based on student backgrounds. The first result is consistent with current efforts by states and districts to limit the use of exclusionary discipline to improve student outcomes. A line of future research would be to explore our proposed mechanism through which exclusionary discipline leads to a higher likelihood of retention by looking at intermediate outcomes such as tardiness, absenteeism, or incomplete coursework following exclusionary discipline. Such research would require detailed school-level administrative records, but would provide further information to schools on how to intervene with students who have faced exclusionary discipline to prevent the adverse impacts of exclusionary discipline. It may be that schools do not want to completely rule out the possibility of exclusionary discipline, but do want to create interventions to facilitate student reentry to school after OSS, ALE, or expulsion in a way that will not lead to disengagement from school.

The second policy implication, arising from the results in our subgroup analyses, indicates that is important for teachers, during their preparation and ongoing professional development, to examine their mindsets about student ability as well as to reflect on how their practices may be differentially affecting students. Revising the use of exclusionary discipline is not a straightforward policy lever for school leaders or policy makers, but it is nevertheless important for ensuring that all students can thrive in a supportive school environment.

Conclusion

In this article, we have shown that exclusionary discipline has a persistent and meaningful negative relationship with student outcomes. Specifically, we have shown that facing any exclusionary discipline in eighth grade increases the likelihood of ninth-grade retention by 2.5 percentage points independent of the student’s schools, his or her recorded behavior, and other observable characteristics including academic achievement. Furthermore, we find that students’ probability of retention increases when they receive more days of exclusionary discipline. We recommend that schools continue to cautiously use exclusionary discipline and explore interventions that help students reintegrate into the school community after facing such a consequence to ameliorate the adverse effects of OSS, ALE, and expulsion. We further recommend ongoing development and reflection on the part of teachers to ensure that high expectations are held for all students.

Footnotes

Appendix

Table A1.

Relationship between eighth Grade Exclusionary Discipline and Ninth-Grade Retention, Controlling for School-Level Characteristics.

	(1)
Any exclusionary discipline, Grade 8	0.134***
	(0.049)
No. of infractions in eighth grade, by type
Disorderly conduct	0.069***
	(0.021)
Insubordination	0.051***
	(0.015)
Truancy	0.119***
	(0.045)
Other infractions	−0.016
	(0.019)
Alcohol and other drugs	0.203**
	(0.086)
Weapons	0.076
	(0.257)
Violent infractions	−0.045
	(0.032)
Major nonviolent infractions	0.096
	(0.122)
Total number of infractions, Grade 8 and below	−0.006
	(0.006)
Any prior retention	0.173***
	(0.066)
Eighth-grade standardized math/ELA score	−0.653***
	(0.023)
School % FRL	−0.239
	(0.308)
School average standardized math/ELA score	0.072
	(0.061)
School % students of color	−0.176
	(0.615)
Student characteristics	Yes
School effects	Yes
Observations	16,289

Note. Standard errors clustered at district level. ELA = English language arts; FRL = free or reduced-price lunch.

p < .1. **p < .05. ***p < .01.

Authors’ Note

Elise Swanson is now affiliated with University of Southern California, LA, USA.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Notes

Author Biographies

Elise Swanson is a Postdoctoral Fellow at the Rossier School of Education. Her research examines multiple issues of educational equity, including evaluating interventions that aim to increase equitable access to postsecondary education and examining policies with disparate impacts on marginalized populations.

Heidi H. Erickson is a Doctoral Fellow in the Department of Education Refrom at the University of Arkansas. Her research focuses on using rigorous methods to study the impacts of school choice, school finance, and arts education.

Gary W. Ritter, PhD, is Dean of the School of Education and a Professor of Education Policy at Saint Louis University. His current areas of research interest are student discipline policy, teacher quality, postsecondary access for low-income students, and the implementation and evaluation of programs aimed as improving educational outcomes for low-income students. His work also focuses on conveying education data and evidence to non-specialists, including parents, practitioners, and policymakers.

References

Anderson

K. P.

Ritter

G. W.

(2018). Examining disparities in student discipline: Mapping inequities from infractions to consequences. Peabody Journal of Education, 93, 161-173.

Beck

A. N.

Muschkin

C. G.

(2012). The enduring impact of race: Understanding disparities in student disciplinary infractions and achievement. Sociological Perspectives, 55, 637-662.

Bertrand

Pan

(2013). The trouble with boys: Social influences and the gender gap in disruptive behavior. American Economic Journal: Applied Economics, 5, 32-64.

Greene

J. P.

Winters

M. A.

(2007). Revisiting grade retention: An evaluation of Florida’s test-based promotion policy. Education Finance and Policy, 2, 319-340.

Holmes

C. T.

(1989). Grade level retention effects: A meta-analysis of research studies. Flunking grades: Research and policies on retention, 16, 33.

Jacob

B. A.

Lefgren

(2004). Remedial education and student achievement: A regression-discontinuity analysis (NBER Working Paper no. 8918). Retrieved from https://www.nber.org/papers/w8918

Jimerson

S. R.

(1999). On the failure of failure: Examining the association between early grade retention and education and employment outcomes during late adolescence. Journal of School Psychology, 37, 243-272.

Jimerson

S. R.

(2001). A synthesis of grade retention research: Looking backward and moving forward. The California School Psychologist, 6, 47-59.

Jimerson

S. R.

Anderson

G. E.

Whipple

A. D.

(2002). Winning the battle and losing the war: Examining the relation between grade retention and dropping out of high school. Psychology in Schools, 39, 441-457.

10.

Losen

Hodson

Keith

M. A.

II Morrison

Belway

(2015). Are we closing the school discipline gap? The center for civil rights remedies. Retrieved from https://www.civilrightsproject.ucla.edu/resources/projects/center-for-civil-rights-remedies/school-to-prison-folder/federal-reports/are-we-closing-the-school-discipline-gap/AreWeClosingTheSchoolDisciplineGap_FINAL221.pdf

11.

Losen

D. J.

Skiba

R. J.

(2010). Suspended education: Urban middle schools in crisis. Southern Poverty Law Center. Retrieved from https://civilrightsproject.ucla.edu/research/k-12-education/school-discipline/suspended-education-urban-middle-schools-in-crisis

12.

Mendez

L. M. R.

(2003). Predictors of suspension and negative school outcomes: A longitudinal investigation. New Directions for Youth Development, 99, 17-33.

13.

Mendez

L. M. R.

Knoff

H. M.

Ferron

J. M.

(2002). School demographic variables and out-of-school suspension rates: A quantitative and qualitative analysis of a large, ethnically diverse school district. Psychology in the Schools, 39, 259-277.

14.

Muschkin

C. G.

Glennie

Beck

A. N.

(2014). Peer context: Do old for grade and retained peers influence student behavior in middle school? Teachers College Record, 116, 1-37.

15.

Noltemeyer

A. L.

Ward

R. M.

(2015). Relationship between school suspension and student outcomes: A meta-analysis. School Psychology Review, 44, 224-240.

16.

Ozek

(2015). Hold back to move forward? Early grade retention and student misbehavior. Education Finance and Policy, 10, 350-377.

17.

Schwerdt

West

M. R.

(2012). The effects of early grade retention on student outcomes over time: Regression discontinuity evidence from Florida (Program on Education Policy and Governance Working Paper Series, PEPG 12-09). Retrieved from https://dash.harvard.edu/handle/1/9544244

18.

Skiba

R. J.

(2014). The failure of zero tolerance. Reclaiming Children and Youth Journal, 22, 27-33.