School-Based Interventions for Reducing Disciplinary School Exclusion. An Updated Systematic Review

What Studies Are Included?

The review included 275 effect sizes from 67 studies, combining data from Valdebenito et al. (2018) and recent findings, covering 1980 to 2022. This evidence includes 394,242 students involved in school-based interventions to reduce school exclusions through RCT or quasi-experimental evaluations.

Types of Studies

The present review included experimental, quasi-experimental and process evaluations of school-based interventions. Experimental and quasi-experimental designs (QEDs) were used to address research questions (RQs) 1–6 and process evaluations (i.e., those that use qualitative methodologies) were used to answer RQs 7 and 8.

Types of Participants

Types of Interventions

For the purpose of both our quantitative assessment of impact evaluations and the qualitative synthesis of process evaluations, evaluations of school-based interventions were included. We include interventions defined as school-based: that is, delivered on school premises, or supported by schools with at least one component implemented in the school setting. In the present review, we include interventions explicitly aimed at preventing/reducing school exclusion or those measuring exclusion as an outcome. This definition encompasses a wide array of intervention programmes. For example, included interventions may have involved teacher training (e.g., Kendziora et al. 2019), mentoring (e.g., MacIver et al. 2017), mindfulness (e.g., Meadows 2018), or teen courts (e.g., Smokowski et al. 2020).

In addition to interventions that take place solely in schools, we also included evaluations of multi-system interventions with the provision that at least one aspect of the intervention involved schools. For example, interventions that worked with families and/or parents to support children considered at-risk of exclusion (e.g., Mason et al. 2016) were eligible for inclusion. Furthermore, interventions that involved multi-disciplinary teams were eligible for inclusion if one (or more) of the components involved the child's school (e.g., Thompson et al. 2021; Weist et al. 2022).

Types of Outcomes

Duration of Follow-up

Types of Settings

Updated Review of Quantitative Impact Evaluations

As the present review is an update of a previous review of school-based interventions to reduce school exclusion (Valdebenito et al. 2018), studies included in this earlier report are also included in this update. Furthermore, Valdebenito et al. (2018) searched for studies up to 2015, and so searches undertaken for this update were restricted to evaluations published between 2015 and 2022. Studies were not excluded based on the language of publication or their publication status and availability.

We followed recommendations put forward by Kugley et al. (2017) in relation to updating reviews and so minor changes were made to the searches from the previous review. For example, Open Grey was not searched and removed from the proposed list of electronic databases in our protocol. This database is no longer available. As per these guidelines, we also included numerous searches to supplement searches of online databases and included both published and unpublished studies.

Search Terms and Keywords

All searches were conducted using a selected set of keywords, which were present and outlined in our protocol (Valdebenito et al. 2023). Keywords covered terms related to the type of study, type of interventions, population, and outcomes. Table 1 outlines the keywords. These keywords are also consistent with those used in the previous review (i.e., Valdebenito et al. 2018).

Keywords were combined using Boolean operators (e.g., AND, OR, NOT), wildcards and truncation symbols, and combinations of key terms and search operators were created according to the specific database. Following recommendations for best practice, where possible, we adapted searches using subject headings or descriptors appropriately for each database (Kugley et al. 2017). For example, for education databases such as ERIC or Canadian Business & Current Affairs (CBCA), we use the thesaurus or subject index to assess the fit of our keywords for these specific databases. For each electronic database, pilot searches were conducted, and the results of these pilot searches were used to adjust the terms, synonyms, and wildcards. The pilot searches also assisted in the creation of combinations of keywords that captured relevant evaluations.

Appendix S1A outlines the search syntax for searches conducted on each electronic database for quantitative impact evaluations and qualitative process evaluations, along with the dates searches were undertaken, the total number of studies located, and the total number of studies retrieved.

Electronic Searches

As per our protocol, a comprehensive number of electronic databases were searched to identify as many evaluations of school-based interventions to reduce school exclusion as possible. We included databases for published literature, grey literature (e.g., ProQuest Dissertation and Theses Global), and literature in languages other than English (e.g., SciELO).

All databases that were searched are listed in Table 2. Some databases (e.g., Australian Education Index and CBCA) were not searched for qualitative process evaluations, given our geographical restrictions in the qualitative evidence synthesis. ⁴

The selection of databases was informed by searches preformed for the previous review. However, some databases were no longer available and hence could not be searched for the present update. These databases were: Index to Theses database (ProQuest), The Netherlands National Trial Register (NTR), and Open Grey.

Searching Other Resources

In addition to electronic databases, we undertook several additional searches to supplement the systematic search strategy. For example, we searched several institutional websites, hand searched relevant journals, screened and harvested references from relevant reviews, and contacted experts. We also rescreened studies previously excluded by Valdebenito et al. (2018) due to changes in the inclusion criteria for the current updated review (i.e., inclusion of QEDs).

These additional searches were conducted concurrently for quantitative impact evaluations and qualitative process evaluations. Multiple members of the research team were involved in the supplemental search process. The following sections outline these additional searches in more detail.

Selection of Studies

Screening of studies was conducted in Covidence. Initially, studies were screened based on the title and abstracts. M.J. A.-B. screened the studies identified in searches for quantitative impact evaluations and S.H. screened studies identified in searches for qualitative process evaluations.

Studies retained after screening of titles and abstracts were then screened based on the full text. Full text screening was undertaken in duplicate at this stage, and all studies were screened by at least two members of the review team. The eligibility checklists in Appendix S1C were used to guide screening.

Screening of studies eligible for inclusion in our meta-analysis of impact evaluations was undertaken by M.J. A.-B., S.V., D.J., and H.G., who acted as independent second screeners for impact evaluations. The inter-rater reliability ranged from (0.69) to (0.93), and the overall agreement ranged from (92.5%) to (99.3%). Disagreement between reviewers on whether to include or exclude a study was discussed and resolved.

S.H. and H.G. screened all full texts retained for our qualitative evidence synthesis of process evaluations independently. The inter-rater reliability was (0.75), and the overall agreement was good (93.6%). The majority of conflicts occurred due to different reasons for excluding studies. Conflicts were resolved until a consensus was reached.

Data Extraction and Management

All screening was undertaken in Covidence, but data extraction and management were performed in MS Excel. Using formulae previously described raw data to compute effect sizes was extracted from primary studies and recorded. Where possible, we computed difference in difference standardised mean differences (SMDs) as the chosen effect size for studies that reported outcomes as continuous variables. If no baseline data were reported, we computed endline only SMDs. If outcomes were reported dichotomously, we computed odds ratios and then transformed these to d (Appendix S1E).

In cases where results of primary evaluations were reported using statistical models (e.g., regression or ANOVAs) and no raw data was available, effect sizes were computed using test statistics reported. The formulae to compute effect sizes in this way are those that are used in the Campbell Collaboration effect size calculator.

Effect sizes for all outcomes, timepoints, and subgroups were extracted from all studies included in this review. We re-extracted effect sizes from the RCTs included in the previous review. In total, 819 effect sizes were extracted. H.G. and S.V. extracted effect sizes independently of one another. Effect size extraction was not done in duplicate. However, H.G. extracted all effect sizes from the previous review, where S.V. had previously extracted effect sizes. As such, these effects were extracted in duplicate.

The data extraction tool outlined in Appendix S1D was used to code information relating to moderator variables. Data extraction of moderator variables was undertaken by S.V. and H.G. It was not undertaken in duplicate.

Assessment of Risk of Bias in Included Studies

As per our protocol, assessment of risk of bias in included studies was conducted using the EPOC tool for RCTs (Cochrane Effective Practice and Organisation of Care [EPOC] 2007) and the ROBINS-I tool (Sterne et al. 2016) for QEDs. RCTs were assessed using the EPOC tool to ensure consistency with the risk of bias assessment conducted in the earlier review (i.e., Valdebenito et al. 2018). The risk of bias assessment for quantitative impact evaluations was led by A.S. and carried out in duplicate with S.V. or D.J.

Measures of Treatment Effect

The effectiveness of school-based interventions was assessed using effect sizes for treatment-control comparisons reported by included studies. Studies reported the effectiveness of outcomes, measured either dichotomously (e.g., percentage of students excluded vs. percentage of students not excluded) or as a continuous measure (e.g., the mean number of exclusions in schools). Formulae used here can be found in Appendix S1E.

Unit of Analysis Issues

For the purposes of this systematic review, we include primary studies involving pupils and clusters of pupils as units of analysis. One key issue emerges when meta-analysis includes studies randomising clusters or units. Participants nested in the same cluster tend to share similarities (intra-class correlation [ICC]). When this correlation is not accounted for, standard errors (SEs), confidence intervals and p-values tend to be too small. These conditions affect the meta-analysis in two different ways. First, the primary trial gets a mistakenly high weight. Second, the pooled result produces a meta-analysis with an overly small SE. To avoid the combination of individual and clustered data, we plan to follow the strategy proposed by the Cochrane Handbook for Systematic Reviews of Interventions (Higgins and Green 2011). The handbook suggests that the effective sample size in a cluster-randomised trial can be obtained by dividing the original sample size by the design effect, which equals 1 + (M − 1) × ICC. In this equation, M is the average cluster size and ICC is the intra-cluster correlation coefficient.

Dealing With Missing Data

Where statistical information required to compute an effect size was missing, we contacted primary and corresponding authors of papers to retrieve missing information. Where information about moderators was missing, we followed the same procedure.

Assessment of Heterogeneity

We report weighted mean effect sizes with robust adjusted SEs. Heterogeneity between effect sizes is assessed by the Q-value, degrees of freedom and the value of I ². We include the I ² as the value of Q, which can appear distorted if the number of studies included in a meta-analysis is small (Higgins et al. 2003), and it can also be transformed easily to a percentage value.

We report the variance components for each level of a three-level meta-analytical model and conduct a sensitivity analysis to assess the suitability of the multilevel model.

Assessment of Reporting Biases

In the present meta-analysis, we assessed the risk of bias for RCTs and QEDs using two distinct evaluation tools tailored to each study type.

For the RCTs, we employed the EPOC tool, a recognised instrument for evaluating the methodological quality of healthcare interventions in systematic reviews. This tool allowed us to systematically assess various aspects of RCT quality, including randomisation, allocation concealment, blinding, and handling of incomplete outcome data.

To assess risk of bias in studies employing a QED design, we utilised the Risk of Bias in Non-randomised Studies of Interventions (ROBINS-I) tool. ROBINS-I is specifically designed to evaluate the risk of bias in non-randomised studies, including QEDs. This tool enabled us to thoroughly examine the quality and potential sources of bias in QEDs, considering factors such as confounding, selection bias, and measurement of outcomes.

By employing these tailored assessment tools for each study type, we ensured a comprehensive and context-specific evaluation of the risk of bias, enabling us to draw more precise conclusions regarding the quality and reliability of the evidence within our study.

Data Synthesis

To evaluate the impact of interventions aimed at reducing school exclusions, we conducted a meta-analysis of effect sizes. This included both primary outcomes (e.g., school exclusion) and secondary outcomes (e.g., antisocial behaviour and substance use). As anticipated, the data presented both hierarchical (between-study) and correlated (within-study) structures of dependency across the extracted effect sizes. To appropriately model this complexity, we employed robust variance estimation (RVE), which allows for the inclusion of dependent effect sizes without violating assumptions of independence.

To guide the selection of the most suitable meta-analytic model, we applied the decision-making framework developed by Pustejovsky and Tipton (2022). This decision tree provided a structured and empirically supported approach to choosing among RVE models based on key characteristics of the data, including the number of studies (clusters), the presence and nature of within-study dependence, the nesting structure of effect sizes, and the inclusion of moderator variables. Depending on these characteristics, the decision tree recommends whether to use a correlated effects model, a hierarchical model, or a model with small-sample corrections. For example, in our case, the presence of multiple effect sizes per study, variation in outcome type, and a moderate number of studies led us to implement a three-level random-effects model with robust SEs using the CR2 correction.

All analyses were conducted using the metafor package in r-studio (Viechtbauer 2010), with the coding script included in the technical appendices of the final report. This analytic strategy ensured that our modelling approach was both theoretically grounded and statistically appropriate for the data structure, increasing the robustness and replicability of the findings.

Investigation of Heterogeneity: Meta Regression

The exploration of heterogeneity encompasses several key factors, including gender, ethnicity, grade at school, and the type of intervention. To explore sources of statistical heterogeneity, meta-regression models were employed. Moderator analyses examined whether differences in intervention type, implementation context, participant characteristics, or study design contributed to variability in effect sizes. These analyses were critical in identifying conditions under which interventions were more or less effective.

Sensitivity Analysis

In the context of our multilevel meta-analysis, we conducted sensitivity analyses specifically tailored to address outliers and estimate correlations within the covariance matrix. These analyses were instrumental in assessing the stability and reliability of our results.

We systematically examined the influence of outliers on our pooled effect sizes, ensuring that our findings were not unduly affected by extreme data points. Additionally, we scrutinised the impact of estimated correlations within the covariance matrix, a critical aspect of multilevel meta-analysis. By doing so, we enhanced the robustness and precision of our meta-analytic outcomes, providing more confidence in the validity of our findings.

Treatment of Qualitative Research

We assessed all included process evaluations for methodological quality. This was performed using the CASP checklist (Critical Appraisal Skills Programme 2024) for appraisal of qualitative research (see Table 7). This tool is designed to enable reviewers to consider three issues related to the quality of a qualitative process evaluation. Items refer to the validity of the results, the content of the results, and also the application of the results (CASP 2024), and responses include ‘yes’, ‘no’, or ‘can't tell’. Assessment of qualitative process evaluations was undertaken by two members of the research team independently (H.G. and D.J.), and all differences or disagreements were reconciled.

Data Extraction and Management

Data extraction from process evaluations was undertaken in MS Excel using an inductive thematic analytical approach. Two members of the research team (H.G. and S.H.) independently extracted data from included process evaluations in relation to the factors that facilitated or hindered the implementation of the intervention in schools. The themes and subthemes categorised by researchers were recorded, alongside supporting evidence and information about the source of the evidence (e.g., a quote from a teacher/student/school leadership, or an interpretation by the researcher, and the page number).

Data Synthesis and Analysis

We adopted a thematic synthesis framework to synthesise findings from included process evaluations. Themes and subthemes that were extracted from individual process evaluations were recorded and then grouped into overarching streams to explore the factors that facilitated the implementation of interventions in a school-based setting and those factors that were perceived as a barrier. Our results are discussed in relation to these facilitators and barriers.

Results of the Searches

Systematic searches were undertaken to identify and retrieve a population of both published and unpublished studies that aligned with our inclusion criteria to assess the quantitative effectiveness of school-based interventions to reduce school exclusion. The results of these searches are illustrated in the PRISMA flow diagram (i.e., Figure 1).

OPEN IN VIEWER

Figure 1

PRISMA flow diagram for quantitative studies.

The implementation of the planned search strategy resulted in the identification of 11,346 articles sourced from 22 electronic databases. Consistent with our established methodology, we augmented the electronic searches using three supplementary strategies: (i) an examination of reference lists from manuscripts identified in systematic searches; (ii) contacting experts in the field; and (iii) a comprehensive review of backward citations. The all-encompassing list of studies (N = 11,346), accompanied by their corresponding references, was imported into Zotero and Covidence. Covidence facilitated the management of pertinent bibliographic references, citations, screening processes, full-text reviews, assessments of risk of bias, extraction of study characteristics and outcomes, as well as the subsequent exportation of data and references (Babineau 2014).

Following the elimination of duplicate entries, a total of 8181 manuscripts were retained for the screening phase. This screening process, overseen by three members of the research team (i.e., M.J. A.-B., S.V., and H.G.), facilitated the evaluation of each manuscript based on the information presented in its title and accompanying abstract. Upon completion of the screening, a total of 7404 studies were excluded due to their lack of relevance to the objectives of the present review. Consequently, 777 manuscripts were retained for full-text screening.

Eligibility was determined at this stage using the full text of each manuscript, where one was available. Our screening was guided by our inclusion criteria, with a primary focus on aspects related to study design, targeted population, and outcome measures and results. The eligibility stage was carried out independently by four members of the research team (i.e., M.J. A.-B., D.J., H.G., and S.V.), utilising a screening tool (Appendix S1C) aligned with the inclusion/exclusion criteria detailed in Section 4. The assessment of eligibility concluded with 737 studies being excluded (refer to Section 5.2 for reasons for exclusion), while 8 studies were categorised as ‘studies awaiting classification’. These latter manuscripts met our inclusion criteria but did not report sufficient data for effect size calculations. Details of the studies awaiting classification are documented in Table 10.

In total, 32 evaluations provided a sufficient amount of statistical data, qualifying them for inclusion in the meta-analysis. These studies were included alongside 35 studies previously included by Valdebenito et al. (2018) resulting in 67 studies included in our updated meta-analysis.

Studies Awaiting Classification

Eight studies remained unclassified due to insufficient statistical data for meta-analysis calculations. In all the cases, the authors were contacted via email or social networks at least twice. A compilation of studies that could potentially be incorporated into a future iteration of this review is provided in Table 9.

Characteristics of Included Impact Evaluations

This meta-analysis included 32 new studies. The subsequent sections of this report summarise the characteristics of these studies, including their methodological features, participant characteristics, and the intervention specifics.

Characteristics of the Included Studies: As per our protocol, we included studies reporting interventions that were carried out and reported between 2015 and 2022. Exceptionally, this review involves one manuscript published in 2023 (Huang et al. 2023). The study was identified via the electronic searches, and for this reason, we decided to include its results in the present study. Also, we included five studies that were a product of the re-screening exercise (Ballard et al. 2014; Brady et al. 2007; Fisher and Robinson 1994; Neace and Muñoz 2012; Rentschler and Wright 1996). Therefore, the date of publication of the included papers ranged from 1994 to 2023 (M _date = 2016; SD = 6.4). As indicated by the data presented in Table 10, we have incorporated both published and unpublished reports in a balanced distribution. Each of these reports was composed in English and predominantly represents studies conducted in the United States (94%). Only one study (3%) included a sample originating from the United Kingdom. Despite our comprehensive search through global databases and international organisation websites, with a specific focus on databases from non-European countries, our efforts did not yield studies conducted in alternative geographical locations.

The scope of the quantitative review focused on both experimental and quasi-experimental study designs. Overall, 53% of evaluations included in the meta-analysis were labelled experimental or RCTs and 47% of evaluations included were considered QEDs.

In the case of experimental studies or RCTs, participants were randomly assigned to one or more experimental groups (who received the intervention) and at least one control group. These control groups encompassed options such as a group with no intervention, a group receiving intervention as usual, a group receiving an alternative intervention, or a wait-list control group. While 31% of the included studies randomised individuals, 69% randomised clusters of students, that is, entire schools or classrooms within schools. Where necessary, we corrected data in clusters to combine them with individual-level data.

The included QEDs compared the effects of the independent variable by incorporating at least one treatment group and one control group. 25% of the studies tested statistical equivalence, employing techniques such as Propensity Score Matching.

In terms of analytical models, a majority of the studies compare means (38%) of the treatment and control groups. That was the case, specially for quasi-experimental studies. In the case of RCTs, 11 out of 17 employed robust SE calculations, such as hierarchical regressions or multi-level modelling.

Characteristics of Participants in Included Studies

In the current meta-analysis, most of the students were enroled in higher school grades, particularly middle and high schools (see Table 11). Additionally, the included data showed that these students were enroled in schools characterised by relatively high proportions of students identified as Black (34%), and a smaller proportion of those of White (16%). In fact, within the studies incorporated, four were conducted in schools where a minimum of 80% of the pupils were of Black (Barrett and Harris 2018; Glenn et al. 2021; MacIver et al. 2017; Meadows 2018). However, it is important to note that a considerable proportion of the studies did not provide metrics related to ethnicity (47%).

Within this report the percentage of students eligible for free school meals was used as an indicator of low socio-economic status or population vulnerability (Hobbs and Vignoles 2010). Across our included studies, pupils who received free school meals were prevalent (M _fsm = 67%; SD = 18.1%).

The Characteristics of Interventions

A wide range of school-based interventions were included in the quantitative review. These included, for example, counselling, mentoring, community services, classroom management, after-school academic support, and school-wide strategies. Notably, 28% of these interventions targeted changes at the school, teacher, or parent level, while 63% aimed to impact pupils' skills and behaviours to influence suspension rates (Table 12).

As shown in Table 13, school facilitators (34%) or school facilitators assisted by externals (28%) delivered most of the interventions. This means that schools interested in reducing exclusion typically put resources into the intervention process.

We coded data concerning the role of the evaluator. We identified four potential approaches undertaken by evaluators: (i) administering the programme, signifying that the researcher not only implemented the intervention but also served as the evaluator; (ii) designing the programme, indicating that the evaluator formulated or participated in formulating the theoretical foundation, objectives, and activities of the intervention (Frank et al. 2017; Meadows 2018); (iii) both designing and delivering the programme (Flynn et al. 2016); and (iv) independent evaluation, relating to those researchers or research teams where no research member was involved in any stage of the design or implementation (e.g., Humphrey et al. 2018; Siennick et al. 2020; Weist et al. 2022). As per our review, most of the interventions were evaluated by independent researchers (72%).

Table 13 shows the types of interventions that were used to address school exclusions either as a primary or secondary outcome. While this categorisation might seem constrictive in certain instances (due to the involvement of multiple components within some interventions), our aim was to formulate a comprehensive roster of categories that matched with the review published by Valdebenito et al. (2018).

Risk Reduction: We found five (16%) studies (i.e., Brady et al. 2007; Faria et al. 2017; Rentschler and Wright 1996; Sepanik et al. 2021; Siennick et al. 2020) that implemented interventions aimed at preventing risks such as antisocial behaviour/crime, drug use, and school dropout. Brady et al. (2007) and Siennick et al. (2020) examined the impact of school-based interventions involving the police and the criminal justice system, targeting children with high-risk profiles and low school attendance rates. Generally, these interventions, aimed at risk reduction, were influenced by the Risk-Needs-Responsivity model (Bonta and Andrews 2010). Faria et al. (2017) and Sepanik et al. (2021) were more focused on the prevention of students leaving school before graduation. Those interventions also identified risks and attempted to provide support and resources to the targeted students.

In general, these interventions utilised a multisystem team to assess youths and provide tailored services or referrals. The interventions emphasised customisation and coordination through team-driven planning, offering various services such as tutoring, mediation, and substance abuse evaluations.

After-School Programmes: One study provided data on interventions that offered students after-school activities. The Oregon Mathematics, Engineering, Science Achievement (Greenberg Motamedi and Singh 2016) directed its afterschool initiatives toward middle and high school students and centred on the creation of inventions tackling critical issues in developing nations, including sustainable lighting, water transportation, water filtration, and prosthetics. In pursuit of this objective, MESA has established four curricular modules that encourage students to collaborate in conceptualising, constructing, assessing, and showcasing a functional solution tailored to a specific client.

Enhancing Academic Skills: We found one study targeting the enhancement of academic skills as a strategy to improve academic performance, increase motivation and promote more adaptive behaviour. Martinez (2012) tested an intervention to boost the academic progress of students and improve they behaviour at school.

Mentoring and Monitoring: These types of initiatives were evaluated in two studies (MacIver et al. 2017; Zoblotsky et al. 2020). These programmes established structured and nurturing connections between a young individual encountering academic, emotional, or behavioural challenges and a non-parental adult mentor. The mentor, often a community volunteer, offers role modelling and prolonged support (MacIver et al. 2017). Alternatively, mentors served as student tutors, guiding academic performance, providing guidance, and aiding with scholastic tasks (Zoblotsky et al. 2020).

Skills Training for Students: We found six studies reporting the impact of skills training for school children. Some of these programmes were based on social learning and cognitive behavioural theories (Fisher and Robinson 1994; Gonzalez et al. 2020; Humphrey et al. 2018), and their goal was to enhance individuals' socio-cognitive, socio-emotional, and behavioural skills to regulate maladaptive behaviour. Social skills training programmes typically consist of a curriculum with focused training modules. Group-based sessions and occasionally one-to-one sessions offer the opportunity to implement specific techniques (e.g., instruction, modelling, role-playing, feedback and reinforcement) in a real-world environment. Some more specific programmes target communication skills or approaches to reducing stress (Frank et al. 2017; Meadows 2018). One additional study reported the results of a skill training programme called Self Affirmation (Borman et al. 2022). The intervention aimed to enhance academic outcomes for students who might feel threatened by racial stereotypes. Primarily, the intervention targeted the reduction of the racial disparities in the use of exclusionary discipline. This approach encouraged Black students to concentrate on and affirm positive facets of their identities, such as excelling as siblings, children, athletes, and artists, among others. These changes are expected to improve discipline at school among minority students affected by the discipline gap.

Skills Training for Teachers: Two evaluations targeted teachers' skills. Manzeske et al. (2017) examined an intervention targeting the enhancement of teachers' academic proficiency in mathematics and reading. The study evaluated its effect on the academic achievements and behavioural aspects (attendance and disciplinary referrals) of students in primary school. Flynn et al. (2016) reported the impact of a professional development model designed to empower teachers with skills and resources to create positive classroom environments benefiting a wide range of students. The intervention aimed to assist teachers in perceiving behaviour as a form of communication and creating supportive environments that mitigate misbehaviours through addressing needs and imparting skills to students. Workshops and coaching sessions centred on transitioning teachers from their punitive responses to fostering inclusivity. Taking a slightly different approach.

School-Wide Interventions: We identified six evaluations that evaluated the impact of school-wide interventions on students' behaviour. The programmes were characterised by holistic approaches encompassing systemic modifications throughout the entire school. These interventions encompass students, educators, parents, and occasionally the community within which the school operates. Three of the studies focused on testing PBIS in elementary and middle schools (Barrett and Harris 2018; Gage et al. 2019; Gage and Stevens 2018). These initiatives strive to establish a positive atmosphere through well-defined regulations that encourage positive conduct, learning, and safety. School-wide interventions possess the potential to address both institutional and individual needs of schoolchildren. An additional three studies assessed interventions with the goal of enhancing safety and reducing negative outcomes such as aggression or violence. Others aimed to ease the difficult transition between middle and high school. These programmes employed data-driven approaches to determine appropriate student support. For instance, they provided alternative approaches to managing problematic behaviour, address mental and behavioural health challenges, and promote prosocial behaviour by targeting influential malleable risk and protective factors (Bos et al. 2022; Corrin et al. 2016; Kendziora et al. 2019).

Restorative Justice: Restorative justice was used by three studies (Augustine et al. 2018; Glenn et al. 2021; Huang et al. 2023) included in the present review. Restorative practices (RP) are an alternative to exclusionary disciplinary approaches. These emphasise the resolution of misconduct through addressing relational harm and promoting the reparation of relationships.

Mental Health Interventions: We included three studies which primarily focused on the provision of mental health services. Thompson et al. (2021) reported findings on the effects of a programme known as the Family Access Centre of Excellence (FACE). FACE utilises the Family Check-Up method to assess family strengths and challenges within a child-focused, family systems framework. A treatment strategy is then devised, which establishes connections to appropriate services and employs intensive case management to address obstacles directly. Another approach, the Interconnected Systems Framework (ISF), addresses the common limitations of many school-wide programmes by providing specific guidance on systematically connecting mental health clinicians and schools (Weist et al. 2022). ISF involves expanding school team membership to include administrators and active PBIS and SMH experts. This approach broadens data systems to include universal mental health screening, which is then used to implement evidence-based programming to all tiers. ISF fosters cross-system partnerships, benefiting education by enhanced programming depth and mental health by improving connections to students and families.

Skills Training for Parent: One evaluation examined the impact of skills training for parents (i.e., Mason et al. 2016). The standard version of the Comprehensive Parenting Programme (CSP) entailed six 2-h sessions, encompassing instruction, skill practice, video examples, and reviews. CSP Plus augments this structure by adding two sessions for adolescents and their parents—before and after the standard sessions. These additional CSP Plus sessions mirror the original CSP format but introduce content addressing high school transition goals, family communication, and decision-making. The focus centred on adolescents' involvement in their learning journey.

Violence Reduction: We included two studies measuring the impact of violence reduction programmes (Dymnicki et al. 2021; Neace and Muñoz 2012). Although these interventions could be classified as skills training, we have isolated them because they are specifically targeted at increasing self-control and safety and reducing violence.

RoB in Experimental Studies Using EPOC

The EPOC instrument proposes the following eight criteria for the assessment of quality bias: 1.

Sequence generation.

Each of these domains was judged on a 3-point scale (i.e., low risk, high risk, unclear risk). The EPOC tool provides guidance and examples for each domain that facilitate the decision of assigning low, high or unclear risk. Three members of the team performed the assessment of risk of bias (S.V., D.J., and A.S.). Assessment of bias was performed independently.

The methodological quality of each publication included in the review was evaluated using the EPOC risk of bias tool. The instrument helps reviewers to evaluate the internal validity of reported results across several dimensions. Three coders (S.V., D.J., and A.S.) independently applied the EPOC tool to each study at different locations and did not confer on ratings until after all papers were assessed. Below, we report the results for each of the eight criteria involved in EPOC. Figure 2 shows a summary of the overall result.

OPEN IN VIEWER

Figure 2

EPOC risk of bias randomised controlled trials.

Focusing on Figure 2, we can see the variation between RoB domains for the experimental studies, with criteria ‘7. Contamination’ carrying the greatest proportion of ‘high risk’ overall (58.5% of assessed studies). That said, in six of eight domains, more than 50% of studies were assessed as either ‘high’ or ‘unclear’ risk of bias. Recalling that assessments are driven by what is written in outputs, this does not mean there is bias, but without clear reporting, we cannot assume low risk of bias. For domain seven, contamination, this is driven by the use of individually randomised trials that randomise pupils or units such as classrooms within the same schools. The nature of some interventions may mean that the risk of contamination—where controls receive treatment—is lower. But where units are randomised within schools, it means that treatment and control pupils are in the same school, typically in the same year, and may even be in the same class.

Adequate Sequence Generation: Adequate sequence generation relates to whether a study team used an appropriate procedure to randomise groups in an experimental design, that is, was allocation actually random? Appropriate methods used to generate the allocation are, for instance, the use of random number tables or a computer random number generator (Higgins et al. 2024). Regardless of the approach taken, it should be described in enough detail to allow an assessment of whether the sequence generation was adequate.

Five studies (9%) presented a high risk of bias with respect to allocation sequence (Figure 2). Mason et al. (2016) used a sequential allocation sequence after stratifying by school and pupil gender. Sequential allocation itself is perfectly predictable, which is one source of concern as predictability opens itself to subversion, but Mason et al. also report that allocation factored in the order with which families returned permission slips (p. 4). It is not clear whether, at any point, putative participants were randomly sorted before the sequential allocation sequence was used. As such, this approach represents a high risk of bias. Another paper assessed as having a high risk of bias from the previous review was Allen et al. (1997). In that study, the authors describe using three different methods of randomisation (i.e., picking names from a hat, coin toss and using an alphabetical list of names) as well as running randomisation at different levels in the same study (i.e., individuals and classrooms). This mixture of methods could lead to a high risk of bias in the results: in particular, the use of a list of names is known not to be appropriate for allocating cases.

Under the categorisation of ‘unclear risk’, more than half of the studies (56.6%) were assessed as unclear for sequence generation. This is because many reports presented descriptions of randomisation or having randomised without detailing the methods of sequence generation, or, for example, if software was used (e.g., Arter 2005; Brett 1993; Reese et al. 1981; Meadows 2018).

Finally, nearly one-third (32.7%) of manuscripts were defined as presenting a low risk of bias (e.g., Hirsch et al. 2019; Augustine et al. 2018). For example, one study explained that randomisation was stratified by pupil sex and year group, and that stratified randomisation was conducted using Stata following the specified procedure.

Allocation Concealment: Correct allocation concealment safeguards a rigorous implementation of the randomisation process by not allowing researchers or participants prior knowledge of the results of assignment (i.e., by using sealed envelopes or another procedure that prevents knowledge about the group that the participant is going to be allocated to). This is to prevent subversion of allocation on the basis of preference or choice based on knowledge of what the next patient/unit will ‘get’. To achieve this, Schulz and Grimes (2002) suggest that allocation should be centralised and executed at the beginning of the study, but can also be achieved with ‘trickle trials’ where a centralised allocation system is used to randomise patients as they enter the trial (Ariel et al. 2021).

In this meta-analysis, 7% of studies were evaluated as having a high risk of bias for allocation concealment. Some examples of high-risk studies are those where reports suggested that schoolteachers instead of researchers performed the random allocation (e.g., Crowder 2001), or where randomisation appears to have failed (e.g., Kendziora et al. 2019).

We also found a high percentage of studies (44%) reporting minimal details of allocation concealment. Those studies were classified as ‘unclear risk’ of bias. More positively, nearly half of the remaining studies were classified as having a low risk of bias, typified by randomisation being done at the start of the study, with all units randomised at once. For example, Humphrey et al. (2018, 7) describe the allocation process as having been conducted:

…independently of the authors by the Clinical Trials Unit at the Manchester Academic Health Science Centre and was balanced by proportions of children eligible for free school meals (FSMs) and speaking English as an additional language (EAL) via adaptive stratification (minimisation).

So, a team independent of the study authors carried out the randomisation, and we know from the study that the units randomised were schools. Schools were also stratified according to pupil deprivation and EAL, and randomisation was via minimisation.

Baseline Equivalence in the Outcome Measured: Pre-existing baseline differences between groups—particularly of outcomes—could suggest problems with randomisation, hence, it is a key focus when assessing risk of bias (Shadish et al. 2002). Having assessed this for the RCTs, the majority (nearly 60%) were classified as ‘low risk of bias’—meaning that before randomisation treatment and control were equivalent in terms of exclusion and/or suspension. A further one-quarter of studies (24.5%) were classified as ‘unclear risk’—for example, Dymnicki et al. (2021) discuss balance but present no evidence of this in their paper, meaning it was not possible to assess this.

Finally, we found that nearly 20% of studies had a high risk of bias, meaning that before randomisation, treatment and control groups were not equivalent on pre-intervention outcomes (e.g., suspension or exclusion). This matters because an imbalance at baseline that is not appropriately adjusted for in analysis might then drive a later ‘result’ (i.e., introduce bias). One example of a study that did not report such an adjustment was Panayiotopoulos and Kerfoot (2004). Another example relates to the imbalance of a specific type of exclusion in Berlanga (2004) and Kendziora et al. (2019), where in-school exclusion was not balanced before randomisation.

Note that for assessment of balance in outcome or baseline characteristics we did not focus on whether there were statistically significant differences between groups, but focused on means and distributions (Altman 1985).

Baseline Equivalence in Other Participants' Characteristics: We found a higher proportion of imbalance on participant characteristics, leading to around 30% of studies being assessed as high risk of bias. Again, imbalance at baseline does not invalidate a study or prevent valid inference (Senn 2013), but not adjusting for a pre-existing difference may reduce power. That said, systematic differences between group characteristics—as with Kendziora et al. (2019)—that is not consistent with chance variation suggests a failure of randomisation (Higgins et al. 2024). A further 22% did not report enough data for judgement (i.e., unclear risk), for example, Kendziora et al. (2019), and 50% of studies were assessed as presenting a low risk of bias (e.g., Faria et al. 2017).

Addressing Incomplete Outcome Data: Incomplete outcome data, whether through attrition or being unable to collect outcome data, is an obvious issue for any intervention study. The fact that many studies pursued more than one measure across time makes it likely that attrition or other forms of missingness affected sample sizes.

As per the previous review, we assess a high risk of bias when (i) substantial attrition was present in the study and the researchers did not mention a strategy to deal with that issue (e.g., Peck 2006); (ii) when they used list-wise deletion (e.g., Russell 2007), and (iii) in those cases where the attrition affected the treatment or control group in an unequal proportion of missing cases across arms (e.g., Hostetler and Fisher 1997). Using these criteria, we found that 20% of the studies presented a high risk of bias. A further 30% did not report enough data to be judged. The remaining 50% were evaluated as presenting a low risk of bias. Low risk cases involved (i) those studies reporting zero attrition; (ii) studies where attrition was represented by a low percentage of cases; (iii) when missingness was equivalent in the treatment and the control group, and (iv) when the researcher reported attrition, analysed it and used appropriate methods to deal with attrition (e.g., multiple imputation, full information maximum likelihood).

Blinding of Outcome Assessment: This criterion covers bias arising in situations where those collecting outcome data are aware of the condition assigned to each participant (e.g., individuals, classrooms, schools). In the majority of studies, administrative data from school official records were used to collect outcome data, which carries a lower risk of bias than self-reports of exclusion. But the use of administrative data means that blinding to allocation is likely infeasible for treatment pupils. This is because pupils being excluded or suspended will go through a process before being sent away from school. (One can imagine being on a new scheme might be seen as mitigation in that process before exclusion. Similarly, we can speculate that blinding to allocation might be possible for control pupils if those disciplining a pupil were not informed/did not know they were part of a study.) Overall, knowledge of allocation would not, in our view, be a source of bias, precisely because of the process around exclusion and suspension. In short, manipulating administrative records to bias them for or against a particular intervention—which would raise serious questions about the probity of all involved—seems implausible.

Having said that, 30% of the studies coded were assessed as a high risk of bias. We assigned a high risk of bias when those in charge of delivering the intervention were also collecting data or when teachers rated the students' behaviour while being aware of the allocated condition (e.g., Tilghman 1988). In any case, it must be said that in school-based experiments, blinding is likely infeasible. Most of the studies require at least a minimal participation of school staff, and the allocated condition is mostly evident for participants (Hutchison and Styles 2010). (But see the discussion of this point below relating to ROBINS-I—in brief, it is unlikely that suspension and exclusion would be biased even with knowledge of treatment allocation.).

As in previous cases, the number of reports lacking the data to evaluate the blindness of the outcome assessment (i.e., unclear risk) was high (∼40%).

Protection Against Contamination: Contamination means that control groups receive treatment, either directly through accessing resources, or via interaction with treated units. Schools present a challenging environment when it comes to contamination, by virtue of the way schools are organised or want to implement (e.g., by class or whole school). As a result, it is very difficult to keep separate treatment and control pupils, even if schools are willing to try (and speaking from experience, they often are not). Study design is a key means to protect against contamination. In the case of educational research, cluster randomised designs are viewed as the best way to prevent the mixing of treatment and control pupils, as well as aligning with school preferences for implementation (Connolly et al. 2018). As such, any school study implemented via a cluster-randomised design inherently lowers the risk of bias via contamination and was rated as such in our assessment. However, as Figure 2 illustrates, contamination risk was very high in assessed studies, with nearly 60% being ‘high risk’, and a further ∼6% being unclear risk. This is almost entirely due to the use of individual randomisation, with pupils as the unit of randomisation, meaning that schools in the trials hosted both treatment and control pupils. That said, it is mainly studies from the previous review that this problem relates to (e.g., Cook et al. 2014; Smith 2004); of the new trials assessed in the updated review, nearly two-thirds were cluster-randomised trials (e.g., Obsuth et al. 2017; Humphrey et al. 2018).

Three studies did not report enough data for evaluation and were categorised as unclear risk of bias (e.g., Mason et al. 2016). Finally, 36% of the studies were classified as low risk of bias. Normally low risk studies were cluster-randomised experiments where control and treatment participants were in different schools (e.g., Corrin et al. 2016; Snyder et al. 2010).

Selective Outcome Reporting: Selective outcome reporting occurs when there is a difference between the proposed outcomes for evaluation and those finally reported. In our evaluation, five studies (10%) presented a high risk of selective outcome reporting (e.g., Panayiotopoulos and Kerfoot 2004; Russell 2007). In the case of thesis or trials without published protocols, the assessment was only based on discrepancies between outcomes proposed and reported in those documents, but we acknowledge that not having a protocol should likely default to studies being classed as ‘high risk’. Approximately 80% of studies were assessed as low risk of bias, and 10% did not report enough data for judgement so were ‘unclear’.

Effects Moderated by Participants' Demographic Characteristics

For studies reporting data on gender, we ran a moderator analysis aimed at understanding variations between male and female students. As observed in Table 17, only four studies reported data adjusted by sex. Based on the meta regression analysis, the impact of the intervention was slightly higher for male students, however the difference for both groups was non statistically significant (F = 0.243; df ₁ = 1; df ₂ = 8; p > 0.05). This analysis only included the impact on female and male students. Overall effect sizes were excluded to the aim of capturing the differential impact by sex.

In terms of age, the best proxy variable was school grade (Table 18). This information was reported in 48 studies reporting 173 effects. To test the hypothesis that effect sizes vary by age, we ran a moderator analysis for studies involving primary (SMD = 0.174; 95% CI: 0.016 to 0.332; p < 0.05), middle school students (SMD = 0.133; 95% CI: −0.249 to 0.166; p > 0.05), and high school students (SMD = 0.060; 95% CI: −0.370 to 0.143; p > 0.05). The impact of the intervention was small, positive and statistically significant for students in primary school. However, the between-effect difference was not statistically significant (F = 0.3799; df ₁ = 2; df ₂ = 170; p > 0.05).

We also coded data identifying the country of the selected sample in each primary studies included in the present multilevel meta-analysis. As observed in Table 19, a large proportion of the studies were implemented in the United States, with only 3 out of 67 focusing in the United Kingdom. The effect in both countries was small and the effect was not statistically significant (F = 0.007; df ₁ = 1; df ₂ = 265; p > 0.05).

Effects Moderated by Characteristics of the Studies

We extracted data on characteristics of the studies, such as the methodological design used to calculate the impact of school-based interventions on school exclusion. In this updated version of the review, we included RCTs but also quasi-experimental studies. Table 20 suggests that the highest impact was detected using experimental designs (i.e., RCTs). In the case of quasi-experiments, the impact was close to zero. None of the effects were statistically significant and differences by moderator analysis showed non-significant differences (F = 0.631; df ₁ = 2; df ₂ = 271; p > 0.05).

We coded data identifying the role of the evaluator (Table 21). Independent evaluators were those not taking part in the design or implementation of the evaluated programme. Dependent evaluators were those also contributing to the design and/or the implementation of the programme. For this analysis, we included 52 trials and 206 effect sizes. Evaluations carried out by independent evaluators produced the lowest impact SMD = 0.055 (95% CI: −0.16 to 0.176; p > 0.05). Unsurprisingly, based on what is known about developer-led trials, the studies conducted by dependent evaluators (i.e., those who also developed and/or designed the intervention) produced better results. In fact, trials where the evaluator designed and delivered the intervention yielded results where SMD = 0.436 (95% CI: 0.096 to 0.680; p < 0.001). The meta-regression analysis demonstrated that effect sizes are lower for studies conducted by independent evaluators. The test of moderators showed differences were statistically significant (F = 4.34; df ₁ = 2; df ₂ = 203; p < 0.0001).

Effect of Different School-Based Programmes on School Suspension

As a result of the systematic searches, we found a range of school-based interventions, each with distinct objectives and methods. Some interventions focused on implementing changes at the school-wide level. For instance, programmes like School-Wide Positive Behavioural Interventions and Supports (SWPBIS) adopted a tiered approach, which included implementing strategies for the entire school community and also for those with more specific risk and needs. Another subset of studies reported the impact of interventions targeting changes at the level of students' behaviour, for instance, skills training for students or cognitive behavioural treatments for reducing violence. Conversely, another category of interventions aimed to enhance teachers' skills and competencies. To explore potential differences in their effectiveness, we conducted a moderator analysis comparing school-wide, student-centred, and teacher-centred interventions. Table 22 suggests that interventions centred on changes at the school level or teacher level were slightly more effective than interventions centred on students' behaviour. Even if school centred intervention reported significant results, the test of meta-regression showed differences were not statistically significant (F = 0.652; df ₁ = 2; df ₂ = 263; p < 0.05).

One of the aims of the present meta-analysis was to compare the effect of different interventions on the reduction of school suspension. Table 23 presents the SMDs, confidence intervals as well as measures of heterogeneity for each of the nine types of programmes included in the review.

First of all, as observed in Table 23, many of the interventions are represented by a restricted number of studies/effect sizes, and therefore these results should be interpreted with caution.

Second, the SMDs of four types of programmes present positive (small) and not statistically significant results in favour of the reduction of school exclusion. Those programmes are categorised as: (i) mental health; (ii) violence reduction; (iii) teacher skills; (iv) school-wide interventions.

Third, to test the hypothesis that differences were significant among the compared sub-types, we ran the moderator analysis. The comparison demonstrates that differences are not statistically significant (F = 0.652; df ₁ = 9; df ₂ = 1.512; p < 0.05), meaning that variation in effect sizes cannot be explained by the type of intervention.

Finally, we run a multiple-predictor meta-regression using the metafor package in r-studio. The regression analysis included the role of the evaluator, grade at school, percentage of male students and a variable representing the score of RoB for each included study. The results suggested one statistically significant finding. The intervention effect was larger for interventions centred on teachers' skills (b = 0.77, p > 0.01). However, it is important to keep in mind that the number of studies centred on teachers' skills improvement was limited in this study (i.e., 4 studies and 10 effect sizes).

Results of the Search

Overall, our electronic database searches for qualitative process evaluations of school-based interventions to reduce school exclusions returned 13,733 search results. Search results were first uploaded to Endnote for title and abstract screening. The software identified 3517 duplicates and a further 51 were removed manually.

Therefore, 10,165 studies were screened by title and abstract. These studies were screened by one screener, and 50% were screened by H.G. and 50% were screened by S.H. using Endnote. The majority of studies were excluded based on their title and abstract (n = 10,070), and a total of 95 studies were retained for full-text screening. One additional study was identified in our supplementary searches (i.e., Ashworth 2018 was identified in hand searches of the EthOS database), bringing the total to 96 studies screened at full text.

All studies were retrieved and screened based on the full text by two members of the research team independently, and conflicts were resolved in discussion (H.G. and S.H.). Full text screening was conducted in Covidence, and the results are represented in Figure 9. The majority of conflicts arose due to selecting different reasons for exclusion, as even at full-text screening, there were often multiple reasons to exclude a study. The inter-rater reliability was (0.75) and the proportion agreement was (93.6%).

OPEN IN VIEWER

Figure 9

PRISMA flowchart for qualitative searches.

Most studies were subsequently excluded at this stage (n = 82), the majority of which were studies that discussed school exclusions in the United Kingdom but did not evaluate an intervention (n = 26) or studies that were conducted outside of the United Kingdom (n = 25). Other reasons for exclusion included, wrong outcomes (i.e., did not measure/discuss school exclusions or student disruptive behaviour; n = 12), quantitative study (n = 7), review (n = 3), wrong population (i.e., not mainstream school students or teachers or school staff; n = 6), protocol study (n = 2), or the study was published before 2015 (n = 1).

As such, 14 studies were retained for inclusion in the qualitative evidence synthesis (i.e., Allen et al. 2023; Ashworth 2018; Blandford-Elliott 2021; Calcutt 2020; Humphrey et al. 2018; Knowler et al. 2019; Marchant et al. 2019; Middleton 2022; Reynolds 2021; Rivers 2016; Stanbridge and Campbell 2016; Sparling et al. 2022; Warren et al. 2020; Wright 2020).

However, five studies were subsequently excluded at data extraction phase (Calcutt 2020; Knowler et al. 2019; Marchant et al. 2019; Reynolds 2021; Rivers 2016). These studies are described in further detail in Section 7.1.5, alongside an explanation for their exclusion following data extraction.

In total, nine process evaluations were therefore included in our qualitative evidence synthesis (i.e., Allen et al. 2023; Ashworth 2018; Blandford-Elliott 2021; Humphrey et al. 2018; Middleton 2022; Stanbridge and Campbell 2016; Sparling et al. 2022; Warren et al. 2020; Wright 2020). Ashworth (2018) and Humphrey et al. (2018) drew on findings from participants involved in the same trial. The characteristics and features of these nine process evaluations are summarised in Table 25. In addition, a brief narrative summary of each evaluation, with a specific focus on the intervention implemented, is outlined in Appendix S1F.

Characteristics of Included Studies

A variety of interventions were evaluated in the qualitative process evaluations included in the present review. Two studies examined participants' perceptions of the Good Behaviour Game (GBG), an intervention developed in the United States but implemented globally primarily in primary schools (Ashworth 2018; Humphrey et al. 2018). The findings reported in these separate studies are from the same evaluation of the GBG in English primary schools. Other interventions included the Incredible Years teacher classroom management programme (Allen et al. 2020), the Risk Avert screening tool (Wright 2020), the nurtureUK Violence Reduction Unit (VRU) programmes (Middleton 2022), an intervention planning and screening tool (multi-element plans [MEPS] and target-monitoring and evaluation [TME]; Stanbridge and Campbell 2016), School-Wide Positive Behavioural Supports (SWPBIS; Blandford-ElliotT 2021); a health-based intervention ‘Learning Together’ (Warren et al. 2020) and a trauma-informed whole-school intervention (Sparling et al. 2022). Narrative summaries of each evaluation with a specific focus on the intervention evaluated are provided in Appendix S1F.

The majority of qualitative evaluations were undertaken in English schools (n = 7), with one evaluation in Welsh schools (i.e., Blandford-Elliott 2021), and one evaluation did not provide the geographical location of schools, beyond that they were in the United Kingdom (i.e., Stanbridge and Campbell 2016). Most included process evaluations were of interventions implemented in primary schools (n = 5). Two evaluations (i.e., Warren et al. 2020; Wright 2020) assessed interventions implemented in secondary schools and one evaluation included participants from both primary and secondary schools (i.e., Middleton 2022). One evaluation included participants from an alternative provision setting (Middleton 2022).

Generally, included process evaluations included perspectives from teachers, school leadership (e.g., headteachers or deputy headteachers), or other members of school staff (e.g., teaching assistants). Only one evaluation included perspectives from teachers only (i.e., Allen et al. 2020), but as this programme was considered a teacher training intervention, this is appropriate. Furthermore, only three process evaluations included perspectives of students from the students themselves (i.e., Ashworth 2018; Humphrey et al. 2018; Warren et al. 2020).

All included process evaluations used thematic analysis, or a variation of this approach, to code their findings and the majority used a combination of focus groups and interviews to collect data (n = 4). Three process evaluations used interviews only (i.e., Blandford-Elliott 2021; Warren et al. 2020; Wright 2020). Process evaluations were published between 2016 and 2022 and mostly in peer-reviewed journal articles (n = 4). Three theses were included and one research report (i.e., Humphrey et al. 2018).

Assessment of Quality of Process Evaluations

The CASP tool for quality appraisal in qualitative evidence synthesis was used in the present review to examine the quality of included process evaluations. The items included in this tool were previously outlined in Table 8 (see Section 4.4.12).

This tool is recommended for novice researchers in qualitative evidence synthesis, and experts have noted that some modifications are necessary (Long et al. 2020). Thus, in the context of the present review, we often felt that the ‘can't tell’ response option was unclear and inappropriately added ambiguity to the assessment. As such, we modified this to a ‘somewhat’ criteria whereby when the response was not a clear ‘yes’ or a clear ‘no’, an appropriate option was available. The addition of a fourth response option in this vein is endorsed by other users (e.g., Long et al. 2020).

As previously outlined, the CASP checklist includes 3 sections and 10 questions related to various aspects of qualitative studies. Appendix S1H provides a detailed breakdown of our assessments for the included process evaluations. Two members of the research team conducted an assessment of quality for the included process evaluations, and the mean percentage agreement across all nine studies was 66%. Overall, the result suggests that the quality of the included qualitative process evaluations was good. The following sections narratively describe the findings.

Section A: Are the results valid? 1.

Was there a clear statement of the aims of the research?

This item requires the assessor to consider the goals, importance, and relevance of the study. The majority of evaluations (n = 7) did include a clear statement of the aims of the process evaluation, and only two evaluations (n = 2; Blanford-Elliott 2021; Wright 2020) did not. In the case of Blandford-Elliott (2021), this study scored a ‘no’ because while the aims were outlined, the relevance and goals of the qualitative strand of this evaluation were not. Wright (2020) did not clearly outline the aims, over and above referring to the need for more qualitative studies.

Perceptions of Impact on School Exclusions

As previously discussed, there was a parity of process evaluations that met our inclusion criteria. The most relevant study we identified in our systematic searches was the mid-programme progress report by Middleton (2022). This study used a focus group methodology to explore the effectiveness of nurtureUK VRU programmes implemented in London, Kent, and Medway. The findings on the facilitators and barriers to implementation of these programmes are discussed in full in Section 7.2, but a brief description of the findings in relation to impact on school exclusions is provided here due to the relevance to the current mixed methods review and the UK policy context.

This process evaluation looked at programmes informed by ‘nurture practice’, a form of trauma-informed intervention. A full review of this approach is beyond the scope of the present review, but a brief overview is included in Appendix S1F. Middleton (2022) conducted focus groups with project leaders who led school-based restorative and nurturing programmes between 2020 and 2022. Nine participants took part, including senior leadership from mainstream secondary schools and alternative provision institutions. Using thematic analysis, Middleton (2022) found that overall, the participants perceived the intervention as effective in reducing school exclusions. One participant noted:

We have used [nurturing practice] in a small area to begin with and have seen a reduction in verbal abuse towards staff, whilst another outlined: Two boys who were looking at permanent exclusion now come in … and are doing so well.

Excluded Studies: Process Evaluations

Fourteen studies were originally retained for inclusion in our qualitative evidence synthesis, but during data extraction five studies were excluded (i.e., Knowler et al. 2019; Marchant et al. 2019; Reynolds 2021; Rivers 2016). These studies were excluded because, upon further inspection during data extraction, it was concluded that they did not meet our inclusion criteria. A brief description of these programmes is provided in Appendix S1G.

The primary reason for excluding these process evaluations was that the findings did not sufficiently address student behaviour, which could lead to a child being excluded from school. Three evaluations (i.e., Marchant et al. 2019; Rivers 2016) were excluded at the data extraction phase for this reason, as during coding of findings, it was apparent that the findings related to general classroom behaviour rather than disruptive or misbehaviour that we would consider indicative of increased likelihood for exclusion. The evaluations by Knowler et al. (2019) and Reynolds (2021) were excluded because the intervention setting was not mainstream schools, and as such did not meet our inclusion criteria.

Overview of Thematic Framework

In our synthesis of process evaluations for this review, we were mostly interested in exploring findings related to what participants perceived as facilitators to implementation and/or barriers to implementation. We applied a thematic framework, grouping findings from individual process evaluations into themes and subthemes and categorising them as either a facilitator or a barrier.

The main themes were: (1) Intervention format; (2) (in)Consistency; (3) Buy-in; and (4) Seeing results. Multiple subthemes were identified for each of these themes, and many subthemes related to both facilitators and barriers. These themes and the corresponding subthemes are outlined in the following sections alongside accompanying extracts and quotations from process evaluations and participants themselves as evidentiary support.

Main Theme: Intervention Format

Many process evaluations reported that the intervention format was a significant facilitator, but aspects of the intervention format were also noted as a barrier to implementation. By intervention format in this context, we refer to any aspect of the composition of the intervention programme implemented, for example, the inclusion of group activities, implementation of teacher training, or the materials used to implement the intervention.

In addition to the subthemes relating to intervention format which are outlined in the following sections, there were some general overarching themes that emerged in our qualitative evidence synthesis. Notably, a significant facilitator to implementation was that teachers, school staff, and students viewed the intervention itself quite positively and reported enjoying the activities.

Students were found to have enjoyed intervention programmes in evaluations by Blandford-Elliott (2021) and Humphrey et al. (2018). However, few evaluations interviewed students directly about their perception of the intervention. In an evaluation of the GBG, Humphrey et al. (2018) report that students enjoyed the intervention activities and said:

…it was the best game, learning game, in the world.

In an evaluation of the Learning Together programme, students enjoyed their involvement in the intervention activities and involving the student voice was perceived as a facilitator to implementation. For example, as one member of staff reported:

I think that the students will certainly enjoy the fact that we're doing something like this so they can be involved in it and that they can actually have their voice heard, that they can feel safe at school, that they can feel they're listened to.

Participants in a process evaluation of SWPBIS appreciated the involvement of researchers and felt they benefited from the ability to ‘ask questions and gain a deeper understanding of the rationale for the systems’ (Blandford-Elliott 2021, 68). Other participants commented positively on the evidence-based nature of the intervention programme. For example, one headteacher noted:

So when I met [researcher's supervisor] and you, and when you opened up the world of research to say, “Look, this works and this is why it works,” and the evidence that backed it up, it just gave that strength of conviction that, A, what I believe is true and B, here it is. Here it is, that's worked. And C, here it is in action, it's fascinating. Absolutely brilliant.

Similarly, the evidence base supporting the impact of SWPBIS was received positively by teachers interviewed by Blandford-Elliott (2021). For example, one teacher commented that ‘…knowing it had been used in other classrooms or whatever … and that it wasn't just a theoretical thing based on somebody who's written a book somewhere about their opinions’ (Blandford-Elliott 2021, 68) was a positive aspect of the intervention.

In the same vein, teachers interviewed by Allen et al. (2020) in an evaluation of the Incredible Years Teacher classroom management programme appreciated the aspects of the programme that explained the theoretical framework of the intervention. Allen et al. (2020) identified that this appreciation motivated teachers to implement the intervention. For example, one teacher commented:

…really understanding where those strategies come from as opposed to just going “Oh yeah we'll give you stickers because that's nice” … It's really going through why you're saying what you're saying and what effect that that's having so it's much more thought through.

These theoretical frameworks and evidence-based were generally presented to teachers in designated training sessions. Middleton (2022, 79) report that participants perceived training sessions positively, and a significant facilitator was that the content ‘complemented existing knowledge, provided additional knowledge and increased the confidence of staff to meet the needs of pupils’. Moreover, having experts with both intervention knowledge and previous teaching experience conduct the training was also perceived positively (Middleton 2022).

However, teachers interviewed by Humphrey et al. (2018, 58) did not appreciate the theoretical focus of training sessions, and would have preferred practical, ‘streamlined’ and shorter training sessions. Likewise, training content in the ACE-Informed Schools intervention that was seen as

…quite heavily scientific was a barrier for some participants, and head teachers commented that staff would prefer ‘more practical things’.

Teachers interviewed by Stanbridge and Campbell (2016) appreciated the early intervention approach and the fact that the purpose of the intervention was to understand and address the reasons for misbehaviour, instead of simply reacting to the behaviour. As one teacher commented:

I think it's just really, really thinking about what that behaviour means and why it's happening. Rather than just addressing it like we tend to do … [] rather than just addressing the behaviour on the spot, you're actually looking into why it's happening. And I think that's the key thing.

This approach was a fundamental element of the intervention evaluated, and it was well received by participants. Others noted that the targeted children were those who were not quite eligible for Child and Adolescent Mental Health services (CAMHs) at the time of the intervention, but that without it, they may end up needing these services. Indeed, staff identified that by intervening early, instead of letting problems become ‘embedded’ (Stanbridge and Campbell 2016, 276), the outcomes and trajectory of these students can be improved. This may include the risk of being excluded from school.

…be school failures, you know, they're gonna be the people that don't leave um … education with the qualifications they need to do what they want in life. So it's about getting in there early for children that, that just, just don't reach that high threshold.

Similarly, in the ACE-Informed Schools model, training facilitated a ‘mindset change’ among teachers and staff to understand the causes and explanations for misbehaviour better (Sparling et al. 2022). This led to teachers adopting a more nurturing and empathetic approach to students who have experienced adversity, and as Sparling et al. (2022) outline, this led to improved teaching and also more inclusive schools.

Overall, although the format and type of interventions varied between included process evaluations, participants generally viewed the intervention format positively; however, there were some notable barriers and limitations. The following sections describe the facilitators and barriers with respect to the sub-themes related to intervention format in more detail.

(In)Consistency

Consistency was a recurring theme identified across the included process evaluations. Many participants noted that consistency of implementation and intervention policies was a notable facilitator. However, in contrast, participants also found that inconsistency of leadership (e.g., staff turnover), implementation across the whole school, and between new and old policies, was a significant barrier. The following sections will explain the theme of (in)consistency in further detail.

Buy-in

Buy-in was an interesting theme that emerged from evidence across the included process evaluations. When referring to ‘buy-in’, we are discussing the concept of participants engaging in, accepting, and actively supporting an intervention programme. Generally, a good degree of buy-in from members of the school community was a facilitator to implementation, and a lack of buy-in was a barrier. The following sections discuss these subthemes related to good ‘buy-in’ and a lack of ‘buy-in’ in further detail.

Seeing Results

Many process evaluations noted that seeing the impact of the intervention was a facilitator to implementation and was perceived as improving buy-in, increasing engagement and overall, contributing to participants' positive perception of the intervention programmes. In the context of our qualitative evidence synthesis, this impact refers to changes in student behaviour and/or discipline. However, some participants also highlighted how not seeing a desirable impact was a barrier to implementation. The following sections outline these subthemes in further detail.

Overall Reduction in School Suspension

The primary finding of this meta-analysis indicates that school-based interventions had a small, positive, and statistically significant impact on reducing school exclusions when comparing the intervention group to the control group (SMD = 0.106; [95% CI: 0.039 to 0.173]; p < 0.001). This suggests that participants in these interventions were less likely to be excluded from school. However, it is important to note that the effect size, while statistically significant, was relatively small. This suggests that while school-based interventions are associated with a reduction in suspension rates, the practical significance of this reduction may vary across different contexts and populations.

The results of this updated meta-analysis differ from the first version of this review (Valdebenito et al. 2018), where the overall impact reported was higher (SMD = 0.30; [95% CI: 0.20 to 0.41]; p < 0.001). There are several factors that may contribute to this difference. For example, the updated version includes a larger number of studies and also expanded the scope of the previous review by including both QED and RCTs. What is more, the updated version uses refined analytical methods to account for correlated outcomes, that is, the use of multilevel meta-analysis techniques with robust errors. On the other hand, Mielke and Farrington (2021) conducted a meta-analysis sharing a similar objective with the present review by evaluating the effectiveness of school-based interventions in reducing suspensions and related outcomes. However, there are key differences in scope, methodology, and findings. Mielke and Farrington's review included only 14 RCTs conducted between 2008 and 2019 with a minimum sample size of 100 students, focusing exclusively on arrests and suspensions. Their results showed very small, non-significant reductions in suspensions and arrests overall, but highlighted that well-implemented interventions and those delivered in high schools produced more favourable effects. In contrast, our review employed a broader inclusion strategy encompassing both RCTs and high-quality QEDs, resulting in a larger evidence base (67 studies and over 276 effect sizes), spanning a wider time frame (1980–2022) and addressing both primary (suspensions) and secondary outcomes (e.g., delinquency, conduct problems). Our results display a null or negative effect on behavioural outcomes.

The present review reports the findings from additional separate analyses for different types of school exclusion to examine whether there were any possible differences. While the results for ‘general suspension’, showed a small, positive, and statistically significant reduction, indicating that school-based interventions were most effective in reducing this form of disciplinary action this was not the case for all types of school exclusion. For more severe disciplinary measures, such as out-of-school suspension and expulsion, the interventions had a smaller and often non-statistically significant impact. This suggests that while these interventions can be effective in addressing less severe behavioural issues, they may be less effective in preventing more serious disciplinary issues, as far as we know at present.

Secondary outcomes: The exploration of secondary outcomes related to the impact of interventions designed to reduce school exclusion on externalising behaviours, including conduct problems, delinquency, violence, and substance use, revealed mixed results. In some cases, the interventions had no significant impact, and in others, they even demonstrated possibly harmful effects. For example, the evaluation by Obsuth et al. (2017) reported that the intervention resulted in an increase in problematic behaviours. As such, school-based interventions may have unintended consequences on these secondary outcomes if not carefully designed and implemented. The findings of the present review also found no impact of school-based interventions on behavioural problems such as conduct problems, violence, delinquency, or substance use. In summary, the interventions designed to mitigate school exclusion showed only a modest effect, with outcomes indicating either no discernible impact or, in some cases, a potentially adverse effect on behaviour and substance use. From a practical perspective, it was evident that students who underwent disciplinary measures for school exclusion did not manifest significant changes in their serious behavioural issues. This observation underscores the complexity of addressing the underlying causes of exclusionary behaviour within school settings. It suggests that a one-size-fits-all approach may not adequately address the needs of students, particularly those grappling with more severe behavioural challenges. As suggested by previous research (e.g., Kovalenko et al. 2022), it is increasingly apparent that students exhibiting violent or delinquent behaviours may require more tailored and specialised interventions to effectively address their individual circumstances and foster positive outcomes. Therefore, future research may benefit from a nuanced understanding of the diverse needs of students and the implementation of targeted interventions that cater to their specific requirements.

Moderator Analysis

Heterogeneity in Results: The significant heterogeneity observed in the meta-analysis results is not surprising given the wide range of school-based interventions included in the studies. Moreover, heterogeneity is somewhat expected in a meta-analysis, and is not necessarily a limitation (Higgins and Green 2011). Post hoc analysis of heterogeneity allowed for the examination of the possible sources and causes of differences, thus furthering our understanding of ‘what works, for whom, and in what context’. These interventions varied in terms of their content, duration, and target populations. Additionally, the studies encompassed different geographic locations (including different countries) and school grade levels, all of which contributed to the inherent variability in outcomes.

We included several possible moderator variables that may explain the heterogeneity between primary evaluations. Overall, the findings were mixed. For example, the gender of participants did not significantly influence the effectiveness of interventions, suggesting that these programmes may be equally beneficial for male and female students. The age of participants was also explored as a possible moderator. Age was represented by school grade, and the results suggest that school-based interventions to reduce school exclusions had an overall desirable impact in primary schools, although the mean effect size was relatively small. This effect did not significantly vary for middle and high school students, indicating that we should seek to understand more about which interventions work ‘best’ for which age groups.

Our findings also suggested that the role of the evaluator had a significant impact on the effectiveness of included interventions. As in the previous review (Valdebenito et al. 2018), we observed that independently conducted evaluations (i.e., those where evaluation teams were not programme developers or part of the delivery team) yielded smaller effect sizes when compared to studies carried out by researchers intimately involved in programme development and implementation. This finding is consistent with previous research on the influence of the involvement of programme developers in assessment of effectiveness (Eisner 2009; Eisner and Humphreys 2012; Lösel and Beelmann 2006; Petrosino and Soydan 2005). Indeed, previous meta-analyses of school-based interventions have found that when an intervention is evaluated by the developer, the overall impact is higher (Gaffney et al. 2019). Therefore, this result highlights the advantages of conducting independent trials to generate more precise results.

A moderator analysis of the different types of school-based interventions showed mixed results. Overall, there were no significant differences in the mean effect sizes for different types of interventions. While some interventions, such as academic skills enhancement and mental health programmes, were associated with small positive effects, others, such as mentoring, were associated with a null effect. The lack of significant differences suggests that the effectiveness of these interventions may depend on various contextual factors, and no single type of intervention emerges as superior.

In relation to restorative programmes, a recent non-probability survey conducted among teachers in the United Kingdom revealed some notable insights. It was found that approximately half of primary school teachers and nearly 6 out of every 10 secondary school teachers reported that restorative justice or restorative conversations were explicitly mentioned in their school's behaviour policy (Allen et al. 2023). However, despite the prevalence of these practices, their effectiveness in reducing exclusion or suspension rates remains somewhat limited based on these findings.

It is possible that the effectiveness of RP, as reported in the present review, is explained by methodological limitations of the included evaluations. For example, two included evaluations of RP, Augustine et al. (2018) and Huang et al. (2023), could be described as underpowered cluster-RCTs. Huang et al. (2023) had the larger sample size of these two evaluations, and yet it is still underpowered. This evaluation measured the impact of RP with 18 clusters (in this case, schools) and a total of 5878 students, which averages to approximately 327 students per school. When we assess the statistical power of this study, and assume a normally distributed outcome, without considering potential benefits from factors like stratification or baseline controls, the estimated statistical power of the study is only 40%. In practical terms, this means that even if RP had a real effect, studies of this size would only have a 40% chance of detecting it in 40 out of 100 similar studies.

We conducted a multiple-predictor meta-regression using the metafor package in r-studio. The regression analysis examined various factors, including the role of the evaluator, grade at school, percentage of male students, and a variable representing the score of RoB for each included study. From the results, one significant finding emerged: the intervention effect was notably higher for interventions focused on improving teachers' skills. It is worth noting, however, that the number of studies centred on teachers' skills improvement in this analysis was limited (i.e., 4 studies and 10 effect sizes).

Overall, school-based interventions have the potential to reduce school suspension rates, especially for general suspension, but their impact may be limited in preventing more severe forms of exclusion and addressing externalising behaviours. The significant heterogeneity across studies underscores the need for tailored approaches that consider the specific context, target population, and goals of interventions. Further research is essential to identify the most effective strategies for improving student behaviour, reducing disciplinary actions, and promoting a positive school environment. Additionally, careful programme design and evaluation by those intimately involved in the intervention's development and implementation may enhance their effectiveness.

Intervention Format

Overall, the format of the intervention (i.e., how and what was actually implemented) was perceived as a facilitator to implementation. As highlighted in the supporting quotes from teachers themselves, positive elements of the intervention format related to the inclusion of group discussion and collaboration, flexibility, and structure. Teachers appreciated the opportunity to work in groups, especially with supportive group leaders (e.g., Allen et al. 2020; Humphrey et al. 2018). Group discussion was particularly positive when it allowed for discussion and collaboration with participants from other schools; providing different perspectives and shared experiences (e.g., Warren et al. 2020). Flexible interventions that were adaptable to different contexts, classrooms, and individual students were also viewed positively. Intervention formats that involved a ‘planning’ element were also perceived well (e.g., Stanbridge and Campbell 2016).

However, elements of the intervention format were also perceived as barriers to implementation—for example, outdated, unsuitable, and expensive intervention materials (e.g., Wright 2020). Moreover, a common barrier to implementation was when teachers noted that the intervention did not fit the context of a UK school or classroom, or in other words, there was a barrier in the transferability of the programme (e.g., Ashworth 2018; Humphrey et al. 2018). This was particularly an issue with ‘Americanised’ interventions, or those which may have been developed in the United States and not adapted or adjusted appropriately before implementation in the United Kingdom.

(In)Consistency

Many teachers noted how their existing school policies on behaviour and discipline were inconsistent, and as such, a facilitator to implementation and continued implementation of interventions was when new policies were seen as being consistent (e.g., Blandford-Elliott 2021). Interventions that involved a whole-school approach encouraged consistent implementation across all members of the school community, and this was a notable facilitator to implementation (e.g., Blandford-Elliott 2021). In the same vein, interventions that did not involve a whole school approach often faced the barrier of inconsistent implementation (e.g., Allen et al. 2020). Teachers felt that if all staff members are not trained or educated about intervention strategies, this may ultimately undermine the impact and delivery of these approaches.

Buy-in

Buy-in, and a lack thereof, was another main theme that emerged from the data in our qualitative evidence synthesis. When there was a good degree of ‘buy-in’ from staff and participants, this was a notable facilitator to implementation, especially when school leaders bought into the intervention (e.g., Middleton 2022). Teachers who took part in intervention programmes also noted that a lack of buy-in was a significant barrier, and that they often struggled to encourage their colleagues to respond to new approaches or strategies (e.g., Blandford-Elliott 2021). Buy-in was particularly difficult with certain groups of staff, such as colleagues who may not teach or interact with children with disruptive behaviour or colleagues with many years of experience (e.g., Sparling et al. 2022).

Seeing Results

Finally, the effect of seeing the impact of an intervention on student behaviour was a considerable facilitator to the implementation of programmes (e.g., Middleton 2022; Warren et al. 2020). Although the purpose of our qualitative evidence synthesis was not to assess participants' perceptions of impact, many teachers and school leaders noted that when they could see desirable changes in student behaviour because of the intervention, this was a key facilitator. Seeing results was not only a key facilitator to implementation, but also to continued implementation of programmes following cessation of the evaluation (e.g., Middleton 2022). Often, desirable changes did not need to be observed across a whole school, or even multiple children in a classroom; teachers responded positively and favourably when change was observed with just one student (e.g., Warren et al. 2020). This has particular importance considering of the findings from our meta-analytical examination of impact evaluations. The overall finding suggests that included evaluations had a relatively small impact on school exclusions. However, it may be that even small changes are desirable to teachers and indicate a need for further and wider implementation of intervention programmes.

Quantitative Analysis

The present review faces several limitations that may impact the generalisability and reliability of the results. It is crucial to acknowledge that the included studies provide limited information for assessing quality bias. Notably, approximately 36% of the studies included in our analysis reported results based on samples comprising fewer than 200 participants. The small sample sizes in some primary research studies impose clear constraints on our ability to accurately estimate the effects of interventions accurately, even when aggregated across a meta-analysis. Another source of bias, one that is arguably beyond the control of a meta-analyst, is the large number of studies that did not conduct an independent evaluation. In 45% of included evaluations, the developer of the intervention was also involved in the evaluation of impact, and these studies were associated with higher overall effectiveness. As such, our findings may be biased due to this issue. However, future research is needed to determine whether this is truly a malicious form of bias. For example, it is plausible that evaluations that involve the programme developer benefit from greater implementation fidelity and this is the cause behind the larger effects (Valdebenito et al. 2018). Indeed, findings from our qualitative evidence synthesis suggest that teachers in UK schools appreciated involvement from researchers and those with expert knowledge. This was perceived as a facilitator to implementation, and as such, further investigation to better understand this relationship is needed.

Regarding the searches, although we try to be inclusive and comprehensive, we did not conduct dedicated searches of conference proceedings databases or major education conference archives (e.g., AERA, EERA, BERA). However, throughout the screening process, any conference abstracts or proceedings that appeared in the general database results were screened for eligibility on a case-by-case basis. We acknowledge that this may have resulted in the omission of some relevant grey literature, and we agree that this represents a limitation. In addition, we acknowledge that, while we did not apply any language restrictions in our eligibility criteria, the scope of our database searches was limited by resource constraints. As noted, SciELO was included specifically to capture relevant content from Latin American and Iberian sources. Although we were not able to search more databases that index non-English language studies, two members of the review team are fluent Spanish speakers, which allowed us to screen and assess relevant Spanish-language literature retrieved through our existing strategy.

Regrettably, a notable limitation of the present review is the disproportionate inclusion of research conducted in the United States. However, this is not certainly an issue unique to the present meta-analysis. While the United States has produced a substantial body of research on interventions aimed at reducing school suspensions, the overrepresentation of American studies raises concerns about the generalisability of the findings to other regions and educational contexts (e.g., Cartwright and Hardie 2012). This concentration of evidence from a single country limits our ability to draw generalisable conclusions and insights, as educational systems and practices vary significantly between countries. Studies conducted in the United States may offer valuable insights into interventions' efficacy; however, the applicability of these findings to educational landscapes worldwide may be limited. The cultural, socioeconomic, and systemic differences between different countries could significantly influence the effectiveness of interventions targeting school exclusion. Therefore, it is essential to acknowledge this geographical bias and exercise caution when extrapolating the results of American studies to educational settings in other parts of the world. To establish a more comprehensive and globally relevant understanding of interventions to reduce school exclusion, future research efforts should strive for greater geographical diversity in their study samples and contexts. The inclusion of languages other than English could also add more evidence to the analysis.

Qualitative Analysis

The main limitation to our qualitative evidence synthesis was the lack of available and includable research in the United Kingdom on interventions to reduce school exclusions. We initially intended to synthesise the qualitative literature on perceptions of barriers and facilitators to the implementation of school-based interventions to reduce or prevent children being excluded from school in the United Kingdom. There is quite a lot of literature in a UK context surrounding the issue of school exclusions, but existing studies tend to either interpret national statistics (e.g., Black 2022) or explore a range of perspectives on the use and experiences of exclusionary discipline (e.g., Done and Knowler 2021; Embeita 2019). We were only able to identify one UK process evaluation that explored participants' experiences of interventions to specifically reduce or address the issue of school exclusions (i.e., Middleton 2022). The remaining included evaluations focused on interventions to reduce disruptive behaviour in school.

Given the polarised debate in the United Kingdom, especially in England and Wales, surrounding the use of exclusionary discipline in schools, future research could aim to synthesise studies that examined teachers, school leadership, students, and parents' views and experiences of school exclusion. In particular, future qualitative evidence syntheses could explore the evidence on the reasons and justifications for excluding a child from school and the use of other alternatives that are possibly used (e.g., ‘off-rolling’ and ‘managed moves’). Similarly, future research could examine the experiences, barriers and facilitators to Alternative Provision services for children who are excluded from school. School exclusion, while the focal outcome here, is effectively an indicator of student conduct or a point at which modifications in the management and/or response to student behaviour can be introduced. We also have to be realistic that the elimination of exclusion as a disciplinary sanction, as is sometimes advocated, would likely be met with strong opposition, particularly from teachers. Again, based on data from UK teachers, nearly 90% who responded to a survey (with responses weighted) disagreed with the statement ‘schools should never exclude students, for any reason’ (Allen et al. 2023). Furthermore, removing the option of excluding a child from school may create situations where schools are unsafe for all students and teachers, particularly when interventions to address the root causes of exclusion (i.e., disruptive and often violence/aggressive behaviour) are not put in place. That would not rule out, for example, approaches to try and reduce the use of exclusion in high-excluding schools that relied more on ‘light touch’ approaches, such as behavioural nudges, as has been used effectively in health care settings (Hallsworth and Sanders 2016).

Another possible limitation to our qualitative evidence synthesis is our use of thematic analysis to interpret findings. Thematic analysis is often seen as a flexible approach to understanding qualitative data, and a number of steps and guidelines are available (Braun and Clarke 2006). However, we used an inductive data-driven approach to the interpretation of qualitative data from included process evaluations. Some experts in primary qualitative research suggest that a hybrid approach to thematic analysis, whereby both data-driven and theoretical frameworks are used to interpret findings (Proudfoot 2023). Similarly, the methods and development of qualitative evidence synthesis is a growing area of research, and so perhaps future updates of this review will avail of more advanced methods.

Our qualitative evidence synthesis could also be improved by incorporating global research to improve the generalisability of the findings to additional educational contexts. Despite a vast variety of educational systems, approaches, and standards worldwide, there are lessons to be learned from looking at evidence outside of a national scope, such as in our review. This is an interesting avenue for future qualitative evidence syntheses. Similarly, the generalisability and comparability of our findings may be impacted by the broad range of interventions included in the review. It is possible that teachers' perceptions of the barriers and facilitators to intervention implementation may vary across different types of interventions. A possible avenue for future research would be to examine the qualitative evidence specific to one type of intervention (e.g., whole-school approaches) or one frequently evaluated intervention (e.g., SWPBIS only). Moreover, perhaps future research could compare the perceived barriers and facilitators to implementation across different types of interventions.

One key takeaway from our qualitative evidence synthesis is that there is a need for more research in this area, as well as a need for continued development, implementation, and evaluation of school-based programmes to reduce the use of, and need for, exclusionary discipline. As one teacher noted, ‘no one strategy will work for everyone or forever’ (Allen et al. 2020). Therefore, we must continue to improve our knowledge of what works and how to best implement effective interventions in this area.