Positive Behavioral Interventions and Supports and Substance Misuse in Secondary Schools

Abstract

Amid a proliferation of highly potent opioids, school systems have sought to invest in programs for preventing substance misuse. School teams often prioritize early screening and intervention to prevent student substance misuse; however, given the limited resources of many schools, it is crucial to understand the extent to which commonly available data and existing frameworks are relevant for prevention efforts. In this study, we examined (a) how commonly available data on substance misuse (office discipline referrals [ODRs]) relate to self-reported substance misuse and (b) how positive behavioral interventions and supports (PBIS) implementation across tiers relates to these substance-related outcomes. We found that self-reported substance misuse is significantly related to substance-related ODRs, but that self-report rates are much higher than ODR rates. We also found that Tier 2 PBIS implementation was associated with significantly lower rates of both self-reported substance misuse and substance-related ODRs. These findings suggest that aspects of PBIS implementation could be beneficial in preventing substance misuse, but more research is needed to understand what specific school-based practices best address these student behavioral health needs.

Keywords

adolescent age group data analysis studies school-based mental health services

Following the 2017 declaration of the opioid epidemic as a national public health emergency in the United States, school systems have sought to invest in programs for preventing substance misuse with special attention to the risks of highly potent opioid medications, like fentanyl. Although risk-taking behaviors are normal during adolescence (Ciranka & van den Bos, 2021), concern for substance misuse (using alcohol, drugs, and prescribed medications in harmful ways; McLellan [2017]) has gained much public attention due to the increasing availability of highly potent drugs (Chatterjee, 2022). It is also a time of developmental sensitivity, when early-onset substance misuse is hazardous for long-term health and wellbeing, contributing to risk of myriad physical ailments including infections, overdose, stroke, and so on (van Amsterdam et al., 2013), as well as lower social competence and weaker social relationships (Poudel & Gautam, 2017). In the endeavor to promote favorable behavioral health for adolescents, pivotal prevention strategies include accessible screening procedures and effective interventions.

School-based prevention strategies have become central to supporting youth academic success and wellbeing. However, even with the growing application of school-based substance misuse–prevention programs (Ennett et al., 2003), the evidence of effectiveness has been limited due to issues with implementation and unreliable measures of substance misuse. System-level prevention strategies like positive behavioral interventions and supports (PBIS) have demonstrated meaningful improvements in student outcomes (Bradshaw et al., 2010), but its impact on substance misuse has not been adequately explored. Moreover, many schools lack resources for implementing effective screening or additional interventions, and existing disciplinary school data may prove useful as a low-cost screening, and PBIS implementation may impact social determinants of substance misuse. In this study, we examined (a) how common school data on substance misuse (office discipline referrals [ODRs]) relate to self-reported substance misuse and (b) how PBIS implementation relates to substance-related outcomes.

Risk and Protective Factors for Substance Misuse

Researchers have documented common paths to problematic use of controlled substances, including alcohol, tobacco, and drugs (Nelson et al., 2015; Weinberg, 2001). Among risk and protective factors for adolescent substance use (Cleveland et al., 2008), several individual and environmental influences have proven especially influential. Research has produced evidence linking numerous individual factors to substance use disorders, including genetics, early substance use, executive functions, and psychiatric problems (Weinberg, 2001). Substance misuse is also influenced by various environmental factors, ranging from social to system-level factors, that can interact with individual differences. Protective factors such as parental monitoring (Rusby et al., 2018), positive relationships with teachers, supportive classroom environments, and positive peer relationships are strongly associated with reduced undesired behaviors (Gubbels et al., 2019; Rimm-Kaufman et al., 2005). These protective factors have also been more salient for students at higher risk of emotional and behavioral concerns and students from minoritized backgrounds (Meehan et al., 2003; Skiba et al., 2011); however, substance possession on campus usually results in out-of-school suspension (Austin et al., 2024), which has been linked to outcomes like lower academic achievement and involvement with the justice system (American Academy of Pediatrics Council on School Health, 2013; Fabelo et al., 2011; Noltemeyer et al., 2015).

Schools as a Venue for Intervention

Onset of substance misuse in early adolescence puts individuals at greater risk of developing substance use disorders (Jordan & Andersen, 2017). Given the amount of time adolescents spend at school, school campuses can be valuable venues for effective substance use prevention and mitigation. Schools have the organizational capacity to develop and sustain supportive systems to set students up for success (Bryan et al., 2020) and can be leveraged to intervene on the factors that are highly relevant to substance misuse (Austin et al., 2020).

Current Prevention Approaches

Fortunately, there are several practices in schools, some of which are increasingly common, that serve to prevent substance misuse (Austin et al., 2020). Social-emotional learning programs often focus on developing key skills such as self-regulation, which have an impact on substance misuse (Kimber & Sandell, 2009), and one study found that improvements in these skills following a social-emotional instructional intervention fully mediated decreases in use (Lewis et al., 2012). Engagement with families through schools has also demonstrated success in substance use prevention through various methods, whether through brief, individualized supports for parents (Dishion et al., 2003) or through educational family nights focused on reducing substance availability (Collins et al., 2012). Although some programs have consistent documentation of preventing substance use (i.e., Life Skills Training; Botvin & Griffin, 2004), universal substance misuse–prevention programs have seen varying success (Tremblay et al., 2020). As schools are underutilized in substance misuse–prevention efforts (Compton et al., 2019), research is needed to test new preventive approaches, including more comprehensive school-based frameworks.

Although not explicitly focused on substance misuse, school-wide PBIS is commonly implemented to support student behavior in schools in the United States (Kittelman et al., 2019). PBIS utilizes a multi-tiered approach, wherein Tier 1 provides universal supports for all students, Tier 2 is composed of targeted supports for students at risk of further issues without intervention, and Tier 3 includes the most intensive supports for students demonstrating acute need. These tiers of support are mutually supportive and intended to meet student needs proactively, rather than after they reach an acute threshold and require extensive resources (Horner & Sugai, 2015).

PBIS Implementation and Possible Protective Effects for Substance Misuse

Tier 1 PBIS focuses on promoting a positive, safe, and predictable school-wide culture through teaching and reinforcing of school-wide expectations (Horner et al., 2009). PBIS has demonstrated efficacy in improving observed student behavior in schools (Bradshaw et al., 2010) and reducing the use of in-school and out-of-school suspensions (Gage et al., 2018). A systematic review indicated the use of school-wide interventions targeting social and behavioral domains is effective in increasing student sense of belonging (Allen et al., 2022) through promotion of positive social interactions among students and between students and staff. Furthermore, a meta-analysis by Faggiano et al. (2008) found that universal programs focused on developing student social skills are the most effective in preventing substance use. Supporting these findings, universal-level PBIS is associated with lower perceived student substance use as reported by teachers (Bastable et al., 2015), and implementing specific components of Tier 1 (teaching expectations, reward systems) was linked to lower rates of drug-related ODRs (Molloy et al., 2013).

In Tier 2, PBIS addresses common needs for students who struggle to follow expectations. Key characteristics of Tier 2 supports include increased instruction, acknowledgment, and data collection through standardized and efficient interventions (Horner & Sugai, 2015). Check In/Check Out (CICO; Crone et al., 2003) is a common Tier 2 interventions used to support development of social skills. CICO has been shown to improve student engagement and reduce perceived problem behavior (Maggin et al., 2015). Primarily developed as an intervention for elementary students, CICO has also been adapted in practice in secondary school settings (Kittelman et al., 2018) and demonstrated positive effects on student behavior there as well (Hawken, 2006; Kittelman et al., 2023; Simonsen et al., 2011). Students engaged in CICO routinely check in and out with a staff member at the beginning and end of the day who provides positive, non-contingent attention and encouragement, as well as behavioral instruction, feedback, contingent rewards, and collaboration with families. Increased non-contingent and contingent positive reinforcement and family-school collaboration through CICO may enhance students’ sense of school belonging and relationships with staff, which is a protective factor against maladaptive outcomes (Baker et al., 2008) and is particularly beneficial for students with more risk factors (Meehan et al., 2003; Skiba et al., 2011). Drevon et al. (2019) and Wolfe et al. (2016) conducted two of the most recent literature reviews on CICO, neither of which captured studies assessing substance-misuse outcomes of the intervention. Other interventions delivered though Tier 2, such as Coping Power (Lochman & Wells, 2003), are based on a cognitive-behavioral approach, which emphasizes the interconnectedness of emotions, thoughts, and actions (Clifford et al., 2020). Coping Power is an intervention program designed for students with aggressive behaviors and consists of both a child-focused component administered in small groups of five to eight students and a parent-focused training component (Lochman & Wells, 2003). The child component of Coping Power includes content on generating alternative solutions to problems with peers, emotional regulation, increasing social skills, and building positive peer networks. As an intervention tested primarily in Grades 4 through 8, a meta-analysis by McDaniel et al. (2023) found that Coping Power resulted in a small to moderate mean effect size on externalizing behavior, and in a quasi-experimental study by Lochman and Wells (2003), researchers found that the intervention led to significantly lower rates of substance misuse for fourth-grade students at a 1-year follow-up.

Tier 3 supports students with significant emotional and behavioral needs. In secondary grades, these students are often at high risk of dropout (Lipscomb et al., 2017). Furthermore, adolescent students who struggle with social skills and maintain positive peer relationships are at higher risk of developing substance use disorders (Bozzini et al., 2020; Weinberg, 2001). Tier-3 support typically involves coordination of multiple services, intensive instructional programs, and robust contingency management to support student behavioral success (Benner et al., 2010). Even if effective, given the lack of empirical research on the relation between Tier 3, substance misuse, and school-wide data, it is possible that intensive supports at this tier may not produce perceivable differences in substance misuse rates at a school level due to the few students engaged with them. Research has found implementation of Tiers 1, 2, and 3 with fidelity is associated with reduced rates of school exclusion (Gage et al., 2020; Grasley-Boy et al., 2022), which could impact substance use rates both immediately and in long term (Dong & Krohn, 2020; Fletcher et al., 2008). Lee and Gage (2020) conducted a broader review of PBIS outcomes across tiers, and none of the studies included measured student substance misuse as an outcome.

ODRs as a Substance Misuse Screener

Previous studies evaluating the relationship between PBIS and substance misuse relied on indirect measures of student substance misuse, but systematic screening is more sensitive and enables more proactive engagement with services. Splett and colleagues (2018) found that in schools implementing multi-tiered systems of support (MTSS) for mental health needs, a formal school-wide screening identified 180% more students with behavioral and emotional needs than were identified by the school using typical data and processes. For schools that do use a screening process, teachers are regularly used as the informants on student behavior and are useful in several contexts (Romer et al., 2020). Yet, a system dependent on teacher supervision alone (i.e., ODRs) to identify students who misuse substances may underestimate the prevalence of students with related needs. For example, many students who use substances may not demonstrate externalizing behaviors typically captured through ODRs. Many students with substance misuse issues instead demonstrate internalizing symptoms (e.g., anxiety, depression) which may never warrant an ODR (Gregory et al., 2021). In addition, a student misusing a substance exclusively at home or outside of school would not be issued an ODR at school for this behavior. Although it requires more coordination and resources, expanding from teacher nomination to self-report for students, particularly in adolescence, offers a more comprehensive assessment of student behavioral health (von der Embse et al., 2019) and can be used to inform school-wide and individual supports (Dowdy et al., 2015). Lastly, although self-report of youth substance misuse can be prone to under-reporting, the correlation between self-report and youth drug test results is higher for quicker metabolizing substances (e.g., methamphetamine and many opioids) and shorter recall periods (Folk et al., 2022), indicating that screening for more recent use may yield more accurate responses.

Purpose of This Study

The purpose of this study is to inform how school teams interpret ODR data and to support informed decisions for substance misuse prevention and screening approaches. To do this, we examined whether PBIS implementation is related to substance possession ODRs and self-reported rates of substance misuse. Given the prevalence of potent opioids, special attention was focused on misuse of prescription medications. Although previous research has established promise of PBIS in preventing adolescent substance misuse (Bastable et al., 2015; Molloy et al., 2013), this study is unique in analyzing student self-reported substance use to more directly assess the efficacy of PBIS in affecting student behavioral health, providing new information outcomes associated with implementation of advanced tiers.

Research Questions

To what extent are substance possession ODRs and student-reported substance use related?

To what extent are associations between substance possession ODRs and student-reported substance use moderated by school characteristics?

To what extent does PBIS implementation at Tiers 1, 2, and 3 predict lower rates of student-reported substance use?

To what extent does PBIS implementation at Tiers 1, 2, and 3 predict lower rates of substance use ODRs?

Method

Sample

The sample used in this study was generated by combining data from two sources: the Georgia Student Wellness Survey and the School-Wide Information System (SWIS; May et al., 2013). The SWIS dataset containing student ODRs and PBIS fidelity data was derived from a national dataset containing all substance-related ODRs recorded in the system from 2012–13 to 2018–19. The Georgia Student Wellness Survey is completed by students in all Georgia secondary schools in the United States each school year and contains questions related to student wellness, including recent substance misuse. We restricted our sample to only include Georgia schools for the purpose of merging with the Georgia Student Wellness Survey. We only excluded schools who did not have necessary data for analysis (see Measures below). In all, our dataset included 760 total observations of middle (506 observations, 172 schools) and high schools (254 observations, 95 schools) within years that contained SWIS data and Georgia Student Wellness Survey data. Observations across years were not distributed evenly; there were 57 observations in 2012–13, 47 in 2013–14, 58 in 2014–15, 103 in 2015–16, 139 in 2016–17, 168 in 2017–18, and 188 in 2018–19. Mean demographic statistics, ODR rates, and survey outcomes are displayed in Table 1.

Table 1.

Descriptive for All Schools and by PBIS Fidelity.

School characteristic	All (N = 760 observations, 267 schools)	Tier 1 (N = 241 observations, 146 schools)	Tiers 1 and 2 (N = 76 observations, 52 schools)	Tiers 1, 2, and 3 (N = 26 observations, 22 schools)
FRL %	68.6	67.9	71.5	70.8
White %	39.4	39.6	32.7	39.2
Enrollment	1,074	1,090	1,130	990
School level
Middle	66.6	65.1	76.2	88.5
High	33.4	34.9	23.7	11.5
Discipline outcomes (% with ODRs)
Any substance	1.43	1.72	1.41	1.75
Alcohol	0.09	0.09	0.08	0.11
Drug	0.50	0.52	0.49	0.56
Tobacco	0.88	1.16	0.88	1.14
Survey outcomes (% using in last 30 days)
Any substance	13.37	13.47	11.55	11.25
Alcohol	7.38	7.15	5.86	5.65
Marijuana	4.88	5.23	4.43	3.72
Methamphetamine	0.87	1.36	1.29	1.27
Prescription	3.90	4.54	4.36	4.38
Tobacco	4.06	4.03	3.22	3.08
Vape	6.23	7.30	5.83	5.86

Note. Values for columns by tier are aggregated for schools that implemented indicated tiers with fidelity (70% or higher). FRL = free and reduced-price lunch; ODRs = office discipline referrals.

Measures

Demographic, Building Type, and School Year Data

We used three demographic variables as control variables in the analytic models. We included the percent of students receiving free and reduced lunch (FRL), the percent of White students enrolled in the building, and the overall student enrollment in our models. School year (referred to as wave) and school building type (i.e., middle school vs. high school) were also used as a continuous control variable.

Percentage of Students With Substance-Related ODRs

The dataset contained all ODRs for alcohol, drug, or tobacco use/possession for schools in Georgia using SWIS. Other ODRs that did not include substance use/possession were not examined. Total counts of ODRs, as well as counts for alcohol, drugs, and tobacco separately, were divided by the school enrollment from that school year and multiplied by 100 to create four variables: the percent of students who received ODRs for any substance, for alcohol, for drugs, and for tobacco.

Percentage of Students With Self-Reported Substance Misuse

The Georgia Student Health Survey is designed to assist schools in identifying health issues that affect student achievement and designing prevention plans. It contains many items inquiring into students’ mental and behavioral wellness. The average rate of survey completion as a percentage of school enrollment was 75.84% across schools and across years. Students each year were asked about how frequently they used various substances during the previous 30 days, including alcohol, marijuana, methamphetamine, prescription medication without a prescription (includes painkillers, tranquilizers, sedatives, stimulants, and other prescription drugs), tobacco (smoking cigarettes and using other tobacco products), and vape (electronic vapor product). We also used these data to create an any-substance-misuse variable to indicate whether a student endorsed the use of any of the listed substances. Not all items were assessed across all years; a survey item on vaping was not added until the 2014–15 school year. Survey responses were transposed into a dichotomous “Used” or “Did Not Use” and then aggregated at the school level as the percent of students reporting the use of a specific (or any) substance.

PBIS Fidelity

The Tiered Fidelity Inventory (TFI) was used as the measure of PBIS-implementation fidelity in this study (Algozzine et al., 2014). The TFI is completed by school teams to assess PBIS fidelity separately at Tiers 1, 2, and 3 and has demonstrated strong construct validity, inter-rater reliability, and a strong relation with other measures of PBIS fidelity (McIntosh et al., 2017). Across all three tiers, items are scored on a three-point scale (i.e., 0 to 2, where 0 indicates not implemented, 1 indicates partially implemented, and 2 indicates fully implemented) with total raw scores ranging from 0 to 30 for Tier 1, 0 to 26 for Tier 2, and 0 to 34 for Tier 3. Total raw scores of Tiers 1, 2, and 3 were used as continuous variables. TFI scores were recorded for 347 observations (i.e., completion of the TFI by a school team) from 200 schools in 67 districts from 2017 to 2018 and from 2018 to 2019. The average TFI scores were as follows: Tier 1 total score was 22.67 (SD = 7.02), TFI Tier 2 total score was 8.31 (SD = 9.13), and TFI Tier 3 total score was 4.25 (SD = 9.26). Although TFI scores are often used as a dichotomous variable in regression analyses (i.e., at/above 70% of maximum total score vs. below 70%), Cohen (1983) outlined that dichotomization substantially reduces statistical power, meaning it is unlikely that dichotomizing would be more sensitive to differences over the continuous measure.

Data Analysis

Procedure

Georgia student surveys were obtained through a data request submitted to the Georgia Department of Education and were received as student-level datasets by year. These datasets were aggregated to the school level and then merged to assess school-level survey outcomes across years. The aggregated Georgia student survey data were then merged with the SWIS ODR dataset by matching school and year.

Variable Transformation

Descriptive analysis of key survey and ODR outcome variables showed all demonstrated positive skew. Similar to recent research assessing effects of PBIS (Grasley-Boy et al., 2022), two transformations, square root and cube root, were conducted to better fit assumptions of normality of residuals for multilevel linear modeling (Manikandan, 2010). Evaluation of skew and kurtosis found that cube root transformation was appropriate for the three outcome variables (percent of students with substance-related ODRs, percent reporting any substance misuse, and percent reporting misuse of prescription medication).

Correlation Matrix

To answer Research Question 1, a correlation table was constructed using 11 substance-related outcome variables. We used the Hmisc package for R (Harrell, 2023) to run a Pearson correlation and obtain p-values to evaluate statistical significance of relationships.

Multilevel Modeling

To answer all research questions, we used lme4 (Bates et al., 2015) for R statistical software to fit random intercept models with fixed slopes to account for nesting of schools within districts. Each model included all measures at Level 1 (schools) and variance among districts at Level 2 (districts). For each outcome, we used an iterative model-building process to compare the fit of each model based on Akaike Information Criteria (AIC), Bayesian Information Criteria (BIC), and chi-square significance tests of the model deviance (Burnham & Anderson, 2002). We evaluated unconditional models as a baseline for comparison with more complex models that included both control variables and independent variables. Omnibus testing was conducted using p-value, wherein a value at or below .05 was reported as statistically significant.

Results

Relationship Between Substance-Related ODRs and Self-Reported Substance Misuse

The association between substance-related ODR rates and self-reported substance use rates was first examined through a Pearson correlation matrix (see Table 2). Associations between all 11 substance-related outcome measures were statistically significant at p < .001. The percent of students with ODRs had a moderate association with the percent of students reporting any substance use (r = .47) and the percent of students reporting vaping (r = .58). Figure 1 depicts the mean self-reported substance use rates and ODR rates across years. The top panel displays the two aggregated variables for ODR rates and self-reported substance misuse rates, whereas the rates for substance-specific ODRs and self-report rates are displayed in the bottom panel. In the top panel, the overall trend between the two aggregated measures were similar, increasing from 2012–13 to 2018–19. In the bottom panel, the rates of self-reported vape and ODRs for tobacco decreased by about 50% from 2014–15 to 2016–17, followed by a sharp increase to over 100% and 200%, respectively, through to 2018–19. Comparing across panels, increased rates of self-reported misuse of any substance in 2017–18 coincided with increased self-reported misuse across various substances. The peak ODR rates in 2018–19 were due to higher tobacco ODRs, whereas the peak self-reported substance misuse in the top panel that year was primarily due to increased vape usage.

Table 2.

Correlation Matrix Across Substance-Related Outcome Measures.

Outcome	1	2	3	4	5	6	7	8	9	10	11
1. % Using any substance	—
2. % Using tobacco	.903*	—
3. % Using vape	.851*	.822*	—
4. % Using marijuana	.901*	.818*	.719*	—
5. % Using methamphetamine	.458*	.521*	.531*	.612*	—
6. % Using prescription	.755*	.653*	.691*	.762*	.692*	—
7. % Using alcohol	.961*	.906*	.804*	.878*	.482*	688*	—
8. Alcohol ODRs	.245*	.225*	.260*	.231*	.203*	.186*	.274*	—
9. Drug ODRs	.268*	.164*	.252*	.308*	.187*	.235*	.269*	.322*	—
10. Tobacco ODRs	.459*	.466*	.615*	.381*	.311*	.330*	.446*	.355*	.431*	—
11. % Students with ODRs	.466*	.434*	.581*	.419*	.322*	.352*	.458*	.471*	.693*	.944*	—

Note. ODRs = office discipline referrals.

p < .05.

Figure 1.

Substance-Related Office Discipline Referrals (ODRs) and Self-Reported Substance Use Across Years.

We used several multilevel models to investigate the associations between ODR rates and self-reported substance use rates. We began with an unconditional model and then added substance-related variables and then school demographics. Vape misuse data were reported only in later waves (school years 2014–15 to 2018–19), so models were compared without the vape variable (760 observations) and with vape survey variable (656 observations). Model fit, as measured by AIC and BIC, improved in each step. Due to linear dependency with other substance use variables, percent of students using any substance was modeled separately.

Parameters for the models with and without the vape variable are displayed in Table 3. For the model without vape as a predictor, tobacco use was percent of student with ODRs (t = 3.56, p < .001). In contrast, when the vape use variable was added to the model as a predictor, the association between ODRs and tobacco was not significant, whereas the association with vape rates was significant (t = 7.65, p < .001). In both models, a higher prescription medication rate was significantly associated with lower rates of substance-related ODRs (p < .01).

Table 3

Model Parameter Estimates for Cube Root-Transformed percentage of Students Referred for Substance Use/Possession.

Parameter	Model 1			Model 2			Model 3
Parameter	Estimate (SE)	df	p	Estimate (SE)	df	p	Estimate (SE)	df	p
Intercept	0.86 (0.04)	547.41	<.001	1.00 (0.05)	556.97	<.001	0.98 (0.04)	580.30	<.001
% Alcohol use	<0.01 (<0.01)	745.90	.323	<−0.01 (0.01)	643.99	.365	—	—	—
% Marijuana use	0.01 (0.01)	746.63	.078	0.01 (0.01)	642.31	.065	—	—	—
% Methamphetamine use	−0.01 (0.01)	748.15	.443	<−0.01 (0.01)	641.98	.870	—	—	—
% Prescription use	−0.03 (0.01)	745.53	.002	−0.04 (0.01)	643.51	<.001	—	—	—
% Tobacco use	0.02 (0.01)	732.62	<.001	0.01 (0.01)	628.75	.245	—	—	—
% Vape use	—	—	—	0.03 (<0.01)	639.80	<.001	—	—	—
% Any substance use	—	—	—	—	—	—	0.01 (<0.01)	748.96	.013
White %	<0.01 (<0.01)	266.68	<.001	<0.01 (<0.01)	242.18	<.001	0.01 (<0.01)	207.21	<.001
FRL %	<0.01 (<0.01)	367.84	<.001	<0.01 (<0.01)	337.67	<.001	<0.01 (<0.01)	350.77	<.001
Enrollment	<0.01 (<0.01)	534.76	<.001	<0.01 (<0.01)	502.27	.037	−0.08 (0.04)	719.01	.038
Middle school	−0.01 (0.04)	695.39	.825	−0.04 (0.05)	626.17	.411	<0.01 (<0.01)	525.21	.063
Wave	0.05 (0.01)	642.92	<.001	0.02 (0.01)	643.80	.005	0.03 (0.01)	571.95	<.001
% Any substance use: % White	—	—	—	—	—	—	<0.01 (<0.01)	746.85	.022
% Any substance use: % FRL	—	—	—	—	—	—	<−0.01 (<0.01)	745.14	.077
% Any substance use: middle school	—	—	—	—	—	—	0.01 (<0.01)	744.16	.105
% Any substance use: enrollment	—	—	—	—	—	—	<0.01 (<0.01)	748.34	.672

Note. All substance survey variables and school demographic variables are mean-centered, meaning the intercept should be interpreted as the cube root-transformed percentage of students with substance-related ODRs for a high school with average levels of substance use and demographics. Unadjusted intraclass correlation coefficient for Model 3 = .10. FRL = free and reduced-price lunch. Wave = school year.

Moderation by School Demographics

To evaluate possible moderation effects between substance misuse and school demographic variables, another model was fit using the any substance use variable and removing the individual substance use variables to make the model more parsimonious and interpretable (see Table 3). The model including the interaction terms between any substance use and each school demographic variable was a significantly better fit to the data based on AIC and chi-square test (p < .001) than the previous unconditional model for ODR rates, the model with only the any-substance-use variable, and the model with any substance use and all control variables.

Results indicate a significant relationship between substance misuse rate and percent of White students in the school (t = 2.293, p = .022), wherein a higher proportion of White students predicted a greater association between substance use and substance-related ODRs. For example, a high school in 2018–19 with 13% White students (one standard deviation below the mean; M = 39.4%, SD = 26.7%) had an estimated 0.97% of students receiving ODRs when substance use was reported by 3% of students, and 1.37% of students receiving ODRS when substance use rates were 43%. For a school with 66% White students, the lower use rate predicted 2.8% of students to receive ODRs, and a high self-reported use rate predicted 4.7% to receive ODRs. Middle schools were observed to experience significantly lower rates of ODRs (t = −2.074, p = .038). A middle school with average demographics and substance misuse in Year 7 was predicted to experience 1.33% of students with substance-related ODRs compared to 1.64% for a similarly situated high school (23% higher).

Effects of PBIS Fidelity on Substance-Related ODRs

The relationship between substance-related ODRs and PBIS fidelity was examined through multilevel modeling (see Table 4). The dependent variables for these models were percent of students with any substance-related ODRs. We started with an unconditional model, followed by a model including TFI scores, and then a model including those total raw scores and school demographic variables. The model using only TFI scores slightly improved model fit based on AIC, BIC, and chi-square test (p < .001), whereas adding demographic variables in the final models substantially improved fit on all three indicators (p < .001).

Table 4.

Model Parameter Estimates for the Effect of PBIS Implementation on Cube Root-Transformed percentage of Students with Any Substance Use, Referred for Substance Use/Possession, and Using Prescription Medications.

Parameters	Model 1: Any substance use				Model 2: Substance use/possession ODR				Model 3: Prescription medications use
Parameters	Estimate	SE	df	p	Estimate	SE	df	p	Estimate	SE	df	p
Intercept	2.76	0.03	177.86	<.001	1.14	0.03	143.02	<.001	1.81	0.12	328.56	<.001
TFI Tier 1 total score	−0.001	0.002	336.32	.743	0.006	0.002	331.63	.009	0.000	0.002	337.60	.851
TFI Tier 2 total score	−0.005	0.002	337.72	.013	−0.005	0.002	334.04	.018	−0.002	0.002	331.46	.292
TFI Tier 3 total score	0.002	0.002	333.33	.140	0.004	0.002	325.00	.030	0.002	0.002	337.97	.203
White %	0.003	0.001	148.29	<.001	0.01	0.001	139.96	<.001	0.000	0.001	100.68	.855
Free and reduced lunch-price %	0.004	0.001	189.30	<.001	0.003	0.001	183.95	<.001	0.002	0.001	133.09	.011
Enrollment	<0.001	<0.001	273.83	.403	<0.001	<0.001	286.15	.055	<0.001	<0.001	167.13	.367
Middle school	−0.71	0.03	329.63	<.001	−0.24	0.03	332.86	<.001	−0.32	0.03	301.07	<.001
Wave	0.02	0.02	315.76	.445	0.17	0.03	301.17	<.001	0.000	0.02	322.60	.998

Note. All TFI variables and school demographic variables were mean-centered, meaning the intercept should be interpreted as the cube root-transformed percentage of students with substance-related ODRs for a high school with average levels of PBIS implementation and demographics. Unadjusted intraclass correlation coefficient (ICC) for Model 1 = .05, Model 2 = .13, and Model 3 = .06. Wave = school year. TFI = Tiered Fidelity Inventory (Algozzine et al., 2014); ODRs = office discipline referrals.

The final model estimated that TFI Tier 2 total score predicted significantly lower rates of students with ODRs (t = −2.377, p < .05). If an average high school improved Tier 2 PBIS implementation from an 8-point to a 20-point Tier 2 raw score, 17% fewer students would be expected to receive a substance-related ODR that year. TFI Tier 1 (t = 2.618, p < .05) and Tier 3 scores (t = 2.184, p < .05) were also significant predictors of students receiving substance-related ODRs, where higher fidelity scores at these tiers predicted a higher percent of students with ODRs. If the average high school in 2018–19 increased Tier 1 fidelity scores from 23 to the 70% criterion of 24, 1.5% more students were predicted to receive substance-related ODRs. If the same school improved their Tier 3 implementation from the average of 4 points to the 70% criterion of 24, 22% more students were predicted to receive ODRs. Middle schools were observed to experience significantly lower rates of ODRs within this model (t = −7.18, p < .001). A middle school with average demographics and PBIS implementation in Year 7 was predicted to experience 0.72% of students with substance-related ODRs compared to 1.48% for a similarly situated high school (105% higher).

Effects of PBIS Fidelity on Self-Reported Substance Misuse

Models examining the impact of PBIS implementation on substance misuse were constructed identically to those examining the impact on ODRs, with an unconditional model being examined first. Model fit improved significantly after adding PBIS implementation variables (p < .001) and demographic variables (p < .001). The final models estimated that TFI Tier 2 total score predicted significantly lower rates of students reporting substance misuse (t = −2.508, p < .05; see Table 4). More specifically, a high school in 2018–19 with average demographics and PBIS fidelity could expect the percent of student misuse to decrease by 6.23% if Tier 2 was implemented with fidelity (improving from 8 points to 20). For an average school (1,074 students), this equates to 14 fewer students reporting any substance misuse. TFI Tiers 1 and 3 were not significant predictors of self-report use of any substance (p > .10; see Table 4). Middle schools were observed to have students report significantly lower rates of substance misuse within this model (t = −7.18, p < .001). A middle school with average demographics and PBIS implementation in Year 7 was predicted to have 8.66% of students reporting substance misuse compared to 21.05% of students for a similarly situated high school (143% higher).

Effects of PBIS Fidelity on Self-Reported Prescription Medication Misuse

As with previous dependent variables, modeling prescription medication misuse started with an unconditional model, adding the PBIS implementation variables in the subsequent model, and then adding demographic control variables for the final model. The model including only PBIS implementation variables did not improve fit over the unconditional model, but the final model including demographic variables demonstrated significantly better model fit over the other two (p < .001). Based on the final model, the percent of students self-reporting prescription medication misuse was not predicted by PBIS implementation fidelity at Tier 1 (t = .0188, p = .85), Tier 2 (t = −1.055, p = .29), or Tier 3 (t = 1.276, p = .20; see Table 4). Middle schools were observed to have students report significantly lower rates of prescription medication misuse within this model (t = −11.68, p < .001). A middle school with average demographics and PBIS implementation in Year 7 was predicted to have 3.33% of students reporting substance misuse compared to 5.95% of students for a similarly situated high school (77% higher).

Discussion

School teams and prevention-oriented supports require careful attention in light of the increasing prevalence of substance misuse among school-age youth and the growing concerns about potential hazards for even first-time drug use amid the opioid epidemic (Chatterjee, 2022). This study is the first to compare ODRs and underlying rates of substance misuse as reported by youth. The findings indicate that substance-related ODRs and self-reported substance misuse are related, but that ODRs are a substantial undercount of misuse rates, and the relationship is stronger for schools with more White students. This study also tested the relationship between PBIS fidelity and substance-related outcomes. Tier 2 fidelity was predictive of both lower rates of ODRs and self-reported substance use, suggesting that targeted interventions may be a focal point in contemporaneous substance misuse prevention. The positive relationship for Tier 1 and Tier 3 fidelity with substance-related ODRs was unexpected given previous findings (Bastable et al., 2015; Molloy et al., 2013), but factors such as the use of different measures, samples, and perhaps delayed effects (Tier 1), undetectable effects for intensive supports applied to only a few students (Tier 3), and more supervision in healthy systems may explain these findings (both Tiers 1 and 3).

What School Discipline Data Tell Us About Substance Use

Although ODRs are accessible data and could be used to screen for student substance misuse, we found they substantially underestimate substance misuse. In our models, substance-related ODRs accounted for a much smaller proportion of middle- and high-school students (averaging less than 2% of students) than the proportion that reported substance use or misuse in the same schools (over 13% of students). This could be because a substance-related ODR is an indicator of substance access in schools, rather than a measure of student substance misuse. That is, students who receive a substance-related ODR have been caught in possession of substances at school, which is a distinctly different and rarer occurrence than students using substances. As such, substance-related ODRs might be more useful as a surveillance indicator of potential issues with substances on school campuses but are too far removed from substance use behaviors to be an accurate metric for substance misuse. Previous research on screening for mental health concerns in schools found a similar disconnect between students identified as needing support through a traditional teacher nomination process compared with results from a school-wide screener (Splett et al., 2018).

Still, we found a positive association between self-reported substance use and substance-related ODRs, wherein the predictive effect of students misusing any substance on substance-related ODRs was greater for schools with a higher proportion of White students. The reason for this relationship is unclear, but it may depend partially on patterns of substance use among race and ethnic groups. Researchers analyzing a national sample of Youth Risk Behavior Survey found that White youth use tobacco more than other racial/ethnic groups (Seo & Chang, 2022). Tobacco ODRs contributed to the greatest proportion of substance-related ODRs in our sample, and in separate models, self-reported tobacco and vape misuse was predictive of higher substance-related ODRs as well. These results indicate that tobacco ODRs specifically (which likely includes vape) may explain this moderation effect.

Tier 2 and Reduced Substance Misuse

Tier 2 PBIS implementation significantly predicted lower reported substance misuse and lower substance-related ODRs. Even if not specifically intended to target substance misuse, our results indicate provision of structured, supplemental supports may be protective for students that have been identified as having some risk of further behavioral concerns. Interventions like CICO are carried out with attention paid to individual student behavioral skills and student-teacher relationships and may include more school-family partnership (see the study by Weist et al., 2018). Although research on CICO focuses on externalizing behavior, it has demonstrated positive effects for student engagement in high school (Kittelman et al., 2023) and may improve their sense of belonging in school. Coping Power engages parents and provides additional student instruction on emotional regulation and social skills (McDaniel et al., 2023). School belonging, parent involvement (Rusby et al., 2018), emotional regulation, and social skills are all protective factors for substance misuse (Gubbels et al., 2019; Rimm-Kaufman et al., 2005) that are addressed through these interventions. Given that supportive classrooms and positive relationships with teachers are particularly protective for students at higher risk of emotional and behavioral concerns (Meehan et al., 2003), our findings suggest that Tier 2 interventions may be a potent component of multi-tiered substance misuse prevention.

Inconsistency Across Tiers

Unexpectedly, we found that higher implementation of Tiers 1 and 3 were linked to significantly higher rates of substance-related ODRs (though not to significantly higher self-reported substance misuse). As mentioned earlier, this is a departure from previous research examining the relationship of Tier 1 PBIS with measures of substance misuse (Bastable et al., 2015; Molloy et al., 2013). A possible reason for this difference could be partially in the measures used. Bastable et al. (2015) analyzed how responses to one teacher survey item asking about perceived student substance misuse were different based on school TFI scores, whereas Molloy et al. (2013) tested subscales of a different measure of PBIS fidelity (School-Wide Evaluation Tool [SET]; Horner et al., 2004) on substance-related ODRs. Molloy et al. (2013) also found that other subscales of the SET (District Support and Management) were linked to significantly higher drug-related ODRs. Next, the samples differed substantially, wherein Molloy et al. (2013) relied on a sample or predominantly elementary schools across 3 years, and Bastable et al. (2015) used a sample of middle- and high-school buildings from just 1 year.

We find it implausible at this stage that our findings point to deleterious effects of Tier 1 or Tier 3 PBIS implementation on student substance misuse. In particular, it is important to note that although a significant relationship was detected between implementation of these tiers and substance-related ODRs, there was no such relationship with self-reported substance misuse (outcomes which were significantly correlated to each other). This indicates that this outcome may be attributable instead to better detection of substance misuse on campus. Specifically, schools implementing PBIS with higher fidelity may have better monitoring of common areas (e.g., hallway, bathroom) where students can more covertly misuse substances than other schools. Similarly, a school with a Tier 3 team is better situated to respond with more supervision for a student with more severe behavioral health concerns, and increased supervision of the highest risk students may yield more discoveries of substance possession. Still, PBIS was not developed explicitly to address substance misuse. Tier 1 PBIS elements in particular may be expanded to be more explicitly inclusive of substance misuse prevention, including screening, selection of evidence-based practices, and evaluation of exclusionary discipline policies for substance possession.

Limitations and Areas for Future Research

Evaluating preventive frameworks is not without its challenges, and as a result, there are limitations to these findings. First, this study evaluated data from a sample restricted to one state in the Southeastern Unites States. In other regions, the relationship between substance misuse rates, ODR rates, and PBIS fidelity may be different due to several factors including demographic, policy, and enforcement differences between states. Second, this study looked at measurements of PBIS fidelity and substance use within the same year, but it is possible that preventive effects of supports across tiers play out over an extended timeline. For example, our study found that implementation of Tier 1 supports was associated with higher ODR rates in the same year, but the positive effects for these supports may be more distal (e.g., through the reduction of ODRs and suspensions), with effects being evident years later. Longitudinal research on the effects of PBIS implementation on substance misuse over multiple years could be particularly informative in this regard. Third, this study did not identify what aspects of Tier 2 PBIS were preventive for substance misuse, and future research could test the effects of specific interventions, such as CICO. Fourth, our study intentionally analyzed middle and high schools together and contained more middle schools than high schools. Lower base rates of substance misuse in middle schools may have made it harder to detect possible outcome differences. With a larger sample, future studies could evaluate these settings separately to understand if relationships differ by building type. Fifth, the TFI is an assessment completed by school teams. Although team scoring is common in the evaluation of systems implementation in schools, and the TFI is a validated measure of PBIS fidelity, there is a possibility that bias could have affected scoring of PBIS fidelity.

Last, the TFI measures implementation of school-wide PBIS, which was not explicitly designed to reduce substance misuse. The Interconnected Systems Framework (ISF) integrates school-based mental health support and PBIS and may provide more robust supports for students struggling with substance misuse (Barrett et al., 2013). A study using the ISF-II (Splett et al., 2020), a fidelity measure for the ISF, could provide valuable insights about the extent to which more deliberate mental health–focused supports are helpful in this regard. Still, the need for a substance misuse–specific prevention measure may persist; the ISF-II does not explicitly assess the presence of substance misuse–specific expertise, intervention, or supports that could be pivotal in producing consequential reduction in misuse rates.

Implications for Practice

This study presents practitioners with new knowledge that can be helpful in guiding school team decision-making. First, although ODRs may serve as a flag of needed support for a small number of students, the results of this study do not support the use of ODRs as a standalone screener for substance misuse. Most of the students who report substance misuse avoid ODRs and would be likely to go without detection if ODRs were the only source of service referral. School teams may consider implementing substance misuse–specific screeners (e.g., Car, Relax, Alone, Forget, Friends, Trouble [CRAFFT]; Knight et al., 2002) to integrate them as a part of a Screening, Brief Intervention, and Referral to Treatment (SBIRT; Curtis et al., 2014) approach, but they are advised to advance these efforts carefully by actively engaging stakeholders and consulting local and state policies around collection of these data (Austin et al., 2023).

Although these results are correlational, Tier 2 implementation predicted significantly lower substance misuse and substance-related ODRs. Secondary school teams may benefit from exploring expanded implementation of behavioral supports in Tier 2 that support protective factors for substance misuse. Schools implementing PBIS with fidelity already intervene on social behavior skills through instruction and evidence-based interventions (e.g., TFI Items 1.4, 2.5, and 2.6), but more focus may be needed on specific social-emotional skills to achieve desired effects on substance misuse akin to those observed from universal social emotional skills curriculum at Tier 1 (Lewis et al., 2012). School teams could leverage PBIS by providing instruction on social emotional skills through their Tier 1 (Barrett et al., 2018), utilizing small groups in Tier 2 to provide additional instruction and practice opportunities, and aligning language to across interventions and tiers of implementation to teach about these skills.

Conclusion

Amid a public health crisis, screening of potentially problematic substance-related outcomes for youth is challenging and necessary for school systems to support healthy school environments. Based on our findings, substance-related ODRs and self-reported substance misuse are related outcomes, especially for schools with more White students; however, ODRs are not as informative for screening as self-report measures, as they substantially under-represent rates of substance misuse. When action planning around substance misuse prevention, school teams can use the PBIS framework to support the inclusion of self-report measures to inform decisions around supports across all tiers. Although Tier 2 implementation was the only tier of PBIS fidelity found associated with lower substance misuse in this study, supports across tiers are still advised to address the full range of severity of behavioral health needs, including substance misuse. With finite resources, schools need to leverage systems to effectively address dynamic problems like the substance misuse, and as this study demonstrates, PBIS both has promise and may need to adapt to better promote wholistic student wellness.

Footnotes

Authors’ Note

No official endorsement by the U.S. Department of Education of any product, commodity, or enterprise mentioned in this document is intended or should be inferred.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

This document was supported from funds provided by the Center on Positive Behavioral Interventions and Supports cooperative grant supported by the Office of Special Education Programs (OSEP) and Office of Elementary and Secondary Education (OESE) of the U.S. Department of Education (H326S180001). The views expressed herein do not necessarily represent the positions or policies of the U.S. Department of Education.

ORCID iD

Sean C. Austin

References

Algozzine

R. F.

Barrett

Eber

George

Horner

R. H.

Lewis

T. J.

Putnam

R. F.

Swain-Bradway

McIntosh

Sugai

(2014). SWPBIS Tiered Fidelity Inventory. OSEP Technical Assistance Center on Positive Behavioral Interventions and Supports. http://www.pbis.org

Allen

K.-A.

Jamshidi

Berger

Reupert

Wurf

May

. (2022). Impact of school-based interventions for building school belonging in adolescence: A systematic review. Educational Psychology Review, 34, 229–257.

American Academy of Pediatrics Council on School Health. (2013). Policy statement: Out-of-school suspension and expulsion. Pediatrics, 131, e1000–e1007. https://doi.org/10.1542/peds.2012-3932

Austin

S. C.

Anderson

Seeley

J. R.

McGovern

(2024). Substance possession and school discipline: How substance type and student race/ethnicity affect decisions to suspend [Under review].

Austin

S. C.

Cioffi

C. C.

Storie

Bromley

K. W.

Lissman

D. C.

Rochelle

J. L.

Meng

Seeley

J. R.

(2020). Using the PBIS framework to address the opioid crisis in schools. Center on PBIS, University of Oregon. www.pbis.org

Austin

S. C.

Small

J. W.

Seeley

J. R.

(2023, March). School-based screening for student substance misuse. Center on PBIS, University of Oregon. www.pbis.org

Baker

J. A.

Grant

Morlock

(2008). The teacher-student relationship as a developmental context for children with internalizing or externalizing behavior problems. School Psychology Quarterly, 23(1), 3–15.

Barrett

Eber

McIntosh

Perales

Romer

(Eds.). (2018). Teaching social-emotional competencies within a PBIS framework [PBIS evaluation brief]. OSEP TA Center on Positive Behavioral Interventions and Supports.

Barrett

Eber

Weist

(Eds.). (2013). Advancing education effectiveness: Interconnecting school mental health and school-wide positive behavior support. OSEP Center for Positive Behavioral Interventions and Supports.

10.

Bastable

Kittelman

McIntosh

Hoselton

(Eds.). (2015). Do high schools implementing SWPBIS have lower rates of illegal drug and alcohol use? [PBIS evaluation brief]. OSEP TA Center on Positive Behavioral Interventions and Supports.

11.

Bates

Mächler

Bolker

Walker

(2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67(1), 1–48. https://doi.org/10.18637/jss.v067.i01

12.

Benner

G. J.

Beaudoin

K. M.

Chen

P.-Y.

Davis

Ralston

N. C.

(2010). The impact of intensive positive behavioral supports on the behavioral functioning of students with emotional disturbance: How much does fidelity matter? Journal of Behavior Assessment and Intervention in Children, 1(1), 85–100.

13.

Botvin

G. J.

Griffin

K. W.

(2004). Life skills training: Empirical findings and future directions. Journal of Primary Prevention, 25(2), 211–232.

14.

Bozzini

A. B.

Bauer

Maruyama

Simões

Matijasevich

(2020). Factors associated with risk behaviors in adolescence: A systematic review. Brazilian Journal of Psychiatry, 43, 210–221.

15.

Bradshaw

C. P.

Mitchell

M. M.

Leaf

P. J.

(2010). Examining the effects of schoolwide positive behavioral interventions and supports on student outcomes: Results from a randomized controlled effectiveness trial in elementary schools. Journal of Positive Behavior Interventions, 12, 133–148. https://doi.org/10.1177/1098300709334798

16.

Bryan

Williams

J. M.

Griffin

(2020). Fostering educational resilience and opportunities in urban schools through equity-focused school–family–community partnerships. Professional School Counseling, 23(1_part_2), 2156759X19899179.

17.

Burnham

K. P.

Anderson

D. R.

(2002). Model selection and multi-model inference: A practical information theoretical approach (2nd ed.). Springer-Verlag.

18.

Chatterjee

(2022, April). Teen drug overdose deaths rose sharply in 2020, driven by fentanyl-laced pills. npr. https://www.npr.org/sections/health-shots/2022/04/12/1092309418/teen-drug-overdose-deaths-rose-sharply-in-2020-driven-by-fentanyl-laced-pills

19.

Ciranka

van den Bos

(2021). Adolescent risk-taking in the context of exploration and social influence. Developmental Review, 61, 100979.

20.

Cleveland

M. J.

Feinberg

M. E.

Bontempo

D. E.

Greenberg

M. T.

(2008). The role of risk and protective factors in substance use across adolescence. Journal of Adolescent Health, 43(2), 157–164.

21.

Clifford

M. E.

Nguyen

A. J.

Bradshaw

C. P.

(2020). Both/and: Tier 2 interventions with transdiagnostic utility in addressing emotional and behavioral disorders in youth. Journal of Applied School Psychology, 36(2), 173–197.

22.

Cohen

(1983). The cost of dichotomization. Applied Psychological Measurement, 7(3), 249–253.

23.

Collins

D. A.

Johnson

K. W.

Shamblen

S. R.

(2012). Examining a home environmental strategy to reduce availability of legal products that can be misused by youth. Substance Use & Misuse, 47(12), 1339–1348.

24.

Compton

W. M.

Jones

C. M.

Baldwin

G. T.

Harding

F. M.

Blanco

Wargo

E. M.

(2019). Targeting youth to prevent later substance use disorder: An underutilized response to the US opioid crisis. American Journal of Public Health, 109(Suppl. 3), S185–S189.

25.

Crone

D. A.

Horner

R. H.

Hawken

L. S.

(2003). Responding to problem behavior in schools: The behavior education program. Guilford.

26.

Curtis

B. L.

McLellan

A. T.

Gabellini

B. N.

(2014). Translating SBIRT to public school settings: An initial test of feasibility. Journal of Substance Abuse Treatment, 46(1), 15–21.

27.

Dishion

T. J.

Nelson

S. E.

Kavanagh

(2003). The family check-up with high-risk young adolescents: Preventing early-onset substance use by parent monitoring. Behavior Therapy, 34(4), 553–571.

28.

Dong

Krohn

M. D.

(2020). Sent home versus being arrested: The relative influence of school and police intervention on drug use. Justice Quarterly, 37(6), 985–1011.

29.

Dowdy

Furlong

Raines

T. C.

Bovery

Kauffman

Kamphaus

R. W.

Dever

B. V.

Price

Murdock

(2015). Enhancing school-based mental health services with a preventive and promotive approach to universal screening for complete mental health. Journal of Educational and Psychological Consultation, 25(2–3), 178–197.

30.

Drevon

D. D.

Hixson

M. D.

Wyse

R. D.

Rigney

A. M.

(2019). A meta-analytic review of the evidence for check-in check-out. Psychology in the Schools, 56(3), 393–412.

31.

Ennett

S. T.

Ringwalt

C. L.

Thorne

Rohrbach

L. A.

Vincus

Simons-Rudolph

Jones

(2003). A comparison of current practice in school-based substance use prevention programs with meta-analysis findings. Prevention Science, 4, 1–14.

32.

Fabelo

Thompson

M. D.

Plotkin

Carmichael

Marchbanks

M. P.

Booth

E. A.

(2011). Breaking schools’ rules: A statewide study of how school discipline relates to students’ success and juvenile justice involvement. Council of State Governments Justice Center.

33.

Faggiano

Vigna-Taglianti

F. D.

Versino

Zambon

Borraccino

Lemma

(2008). School-based prevention for illicit drugs use: A systematic review. Preventive Medicine, 46(5), 385–396.

34.

Fletcher

Bonell

Hargreaves

(2008). School effects on young people’s drug use: A systematic review of intervention and observational studies. Journal of Adolescent Health, 42(3), 209–220.

35.

Folk

J. B.

Hirschtritt

M. E.

McCrary

Q. D.

Kalapatapu

R. K.

(2022). Agreement between youth self-report and biospecimen-confirmed substance use: A systematic review. Substance Use & Misuse, 57(4), 531–538.

36.

Gage

N. A.

Grasley-Boy

Lombardo

Anderson

(2020). The effect of school-wide positive behavior interventions and supports on disciplinary exclusions: A conceptual replication. Behavioral Disorders, 46(1), 42–53.

37.

Gage

N. A.

Whitford

D. K.

Katsiyannis

(2018). A review of schoolwide positive behavior interventions and supports as a framework for reducing disciplinary exclusions. Journal of Special Education, 52(3), 142–151.

38.

Grasley-Boy

N. M.

Gage

N. A.

Lombardo

Anderson

(2022). The additive effects of implementing advanced tiers of SWPBIS with fidelity on disciplinary exclusions. Journal of Positive Behavior Interventions, 24(3), 183–195.

39.

Gregory

Graybill

E. C.

Barger

Roach

A. T.

Lane

(2021). Predictive validity of the student risk screening scale-internalizing and externalizing (SRSS-IE) scores. Journal of Emotional and Behavioral Disorders, 29(2), 105–112.

40.

Gubbels

van der Put

C. E.

Assink

(2019). Risk factors for school absenteeism and dropout: A meta-analytic review. Journal of Youth and Adolescence, 48, 1637–1667.

41.

Harrell

Jr. (2023). Hmisc: Harrell miscellaneous (R Package 4.8-0). https://CRAN.R-project.org/package=Hmisc

42.

Hawken

L. S.

(2006). School psychologists as leaders in the implementation of a targeted intervention: The behavior education program. School Psychology Quarterly, 21, 91–111.

43.

Horner

R. H.

Sugai

(2015). School-wide PBIS: An example of applied behavior analysis implemented at a scale of social importance. Behavior Analysis in Practice, 8, 80–85. https://doi.org/10.1007/s40617-015-0045-4

44.

Horner

R. H.

Sugai

Smolkowski

Eber

Nakasato

Todd

A. W.

Esparanza

(2009). A randomized, wait-list controlled effectiveness trial assessing school-wide positive behavior support in elementary schools. Journal of Positive Behavior Interventions, 11, 133–144.

45.

Horner

R. H.

Todd

A. W.

Lewis-Palmer

Irvin

L. K.

Sugai

Boland

J. B.

(2004). The school-wide evaluation tool (SET): A research instrument for assessing school-wide positive behavior support. Journal of Positive Behavior Interventions, 6, 3–12.

46.

Jordan

C. J.

Andersen

S. L.

(2017). Sensitive periods of substance abuse: Early risk for the transition to dependence. Developmental Cognitive Neuroscience, 25, 29–44.

47.

Knight

J. R.

Sherritt

Shrier

L. A.

Harris

S. K.

Chang

(2002). Validity of the CRAFFT substance abuse screening test among adolescent clinic patients. Archives of Pediatrics & Adolescent Medicine, 156, 607–614.

48.

Kimber

Sandell

(2009). Prevention of substance use among adolescents through social and emotional training in school: A latent-class analysis of a five-year intervention in Sweden. Journal of Adolescence, 32(6), 1403–1413.

49.

Kittelman

Cohen Lissman

Strickland-Cohen

M. K.

McGrath Kato

Flannery

K. B.

Horner

R. H.

Izzard

St. Joseph

Mowery

(2023). CICO-secondary: A single-case experimental study in high school. Journal of Positive Behavior Interventions, 10983007231200542.

50.

Kittelman

McIntosh

Hoselton

(2019). Adoption of PBIS within school districts. Journal of School Psychology, 76, 159–167.

51.

Kittelman

Monzalve

Flannery

K. B.

Hershfeldt

P. A.

(2018). Adaptations of check-in/check-out to meet the needs of high school students. The High School Journal, 102(1), 4–17.

52.

Lee

Gage

N. A.

(2020). Updating and expanding systematic reviews and meta-analyses on the effects of school-wide positive behavior interventions and supports. Psychology in the Schools, 57(5), 783–804.

53.

Lewis

K. M.

Bavarian

Snyder

F. J.

Acock

Day

DuBois

D. L.

Schure

M. B.

Silverthorn

Vuchinich

(2012). Direct and mediated effects of a social-emotional and character development program on adolescent substance use. The International Journal of Emotional Education, 4(1), 56–78.

54.

Lipscomb

Hamison

Burghardt

Johnson

D. R.

Thurlow

(2017). Preparing for life after high school: The characteristics and experiences of youth in special education. Findings from the National Longitudinal Transition Study 2012. Volume 2: Comparisons across Disability Groups: Full Report. NCEE 2017-4018. National Center for Education Evaluation and Regional Assistance.

55.

Lochman

J. E.

Wells

K. C.

(2003). Effectiveness of the coping power program and of classroom intervention with aggressive children: Outcomes at a 1-year follow-up. Behavior Therapy, 34(4), 493–515.

56.

Maggin

D. M.

Zurheide

Pickett

K. C.

Baillie

S. J.

(2015). A systematic evidence review of the check-in/check-out program for reducing student challenging behaviors. Journal of Positive Behavior Interventions, 17, 197–208.

57.

Manikandan

(2010). Data transformation. Journal of Pharma-cology and Pharmacotherapeutics, 1(2), 126–127.

58.

May

S. L.

Ard

W. I.

Todd

A. W.

Horner

R. H.

Glasgow

Sugai

Sprague

J. R.

(2013). School-wide information system. https://www.pbisapps.org/

59.

McDaniel

S. C.

Watkins

Chow

J. C.

Fedewa

Nemer

(2023). Systematic literature review and meta-analysis of coping power: Effects and implications for implementation. Journal of Applied School Psychology, 39(4), 350–374.

60.

McIntosh

Massar

Algozzine

R. F.

George

H. P.

Horner

R. H.

Lewis

T. J.

Swain-Bradway

(2017). Technical adequacy of the SWPBIS Tiered Fidelity Inventory. Journal of Positive Behavior Interventions, 19, 3–13.

61.

McLellan

A. T.

(2017). Substance misuse and substance use disorders: Why do they matter in healthcare? Transactions of the American Clinical and Climatological Association, 128, 112–130.

62.

Meehan

B. T.

Hughes

J. N.

Cavell

T. A.

(2003). Teacher–student relationships as compensatory resources for aggressive children. Child Development, 74(4), 1145–1157.

63.

Molloy

L. E.

Moore

J. E.

Trail

Van Epps

J. J.

Hopfer

(2013). Understanding real-world implementation quality and “active ingredients” of PBIS. Prevention Science, 14, 593–605.

64.

Nelson

S. E.

Van Ryzin

M. J.

Dishion

T. J.

(2015). Alcohol, marijuana, and tobacco use trajectories from age 12 to 24 years: Demographic correlates and young adult substance use problems. Development and Psychopathology, 27(1), 253–277.

65.

Noltemeyer

A. L.

Ward

R. M.

Mcloughlin

C. S.

(2015). Relationship between school suspension and student outcomes: A meta-analysis. School Psychology Review, 44, 224–240.

66.

Poudel

Gautam

(2017). Age of onset of substance use and psychosocial problems among individuals with substance use disorders. BMC Psychiatry, 17, 1–7.

67.

Rimm-Kaufman

S. E.

La Paro

K. M.

Downer

J. T.

Pianta

R. C.

(2005). The contribution of classroom setting and quality of instruction to children’s behavior in kindergarten classrooms. The Elementary School Journal, 105(4), 377–394.

68.

Romer

von der Embse

Eklund

Kilgus

Perales

Splett

J. W.

Suldo

Wheeler

D. W.

(2020). Best practices in universal screening for social, emotional, and behavioral outcomes: An implementation guide (Version 2.0). https://smhcollaborative.org/universalscreening/

69.

Rusby

J. C.

Light

J. M.

Crowley

Westling

(2018). Influence of parent–youth relationship, parental monitoring, and parent substance use on adolescent substance use onset. Journal of Family Psychology, 32(3), 310–320.

70.

Seo

Y. S.

Chang

Y.-P.

(2022). Racial and ethnic differences in e-cigarette and cigarette use among adolescents. Journal of Immigrant and Minority Health, 24(3), 713–720.

71.

Simonsen

Myers

Briere

D. E.

III (2011). Comparing a behavioral check-in/check-out (CICO) intervention to standard practice in an urban middle school setting using an experimental group design. Journal of Positive Behavior Interventions, 13(1), 31–48.

72.

Skiba

R. J.

Horner

R. H.

Chung

Rausch

M. K.

May

S. L.

Tobin

(2011). Race is not neutral: A national investigation of African American and Latino disproportionality in school discipline. School Psychology Review, 40, 85–107.

73.

Splett

J. W.

Perales

Al-Khatib

A. M.

Raborn

Weist

M. D.

(2020). Preliminary development and validation of the interconnected systems framework-implementation inventory (ISF-II). School Psychology, 35(4), 255.

74.

Splett

J. W.

Trainor

K. M.

Raborn

Halliday-Boykins

C. A.

Garzona

M. E.

Dongo

M. D.

Weist

M. D.

(2018). Comparison of universal mental health screening to students already receiving intervention in a multitiered system of support. Behavioral Disorders, 43(3), 344–356.

75.

Tremblay

Baydala

Khan

Currie

Morley

Burkholder

Davidson

Stillar

(2020). Primary substance use prevention programs for children and youth: A systematic review. Pediatrics, 146(3), Article e20192747.

76.

van Amsterdam

Pennings

Brunt

van den Brink

. (2013). Physical harm due to chronic substance use. Regulatory Toxicology and Pharmacology, 66(1), 83–87.

77.

von der Embse

Kim

E. S.

Kilgus

Dedrick

Sanchez

. (2019). Multi-informant universal screening: Evaluation of rater, item, and construct variance using a trifactor model. Journal of School Psychology, 77, 52–66.

78.

Weinberg

N. Z.

(2001). Risk factors for adolescent substance abuse. Journal of Learning Disabilities, 34(4), 343–351.

79.

Weist

M. D.

Eber

Horner

Splett

Putnam

Barrett

Perales

Fairchild

A. J.

Hoover

(2018). Improving multitiered systems of support for students with “internalizing” emotional/behavioral problems. Journal of Positive Behavior Interventions, 20(3), 172–184.

80.

Wolfe

Pyle

Charlton

C. T.

Sabey

C. V.

Lund

E. M.

Ross

S. W.

(2016). A systematic review of the empirical support for check-in check-out. Journal of Positive Behavior Interventions, 18(2), 74–88.