Benchmarks of Equality? School-Wide Positive Behavioral Interventions and Supports and the Discipline Gap

Data Collection and Cleaning

Archival data from the 2015–2016 school year were used from a statewide database from a southeastern state. This database was designed, utilized, and cleaned by a state PBIS project providing schools (N = 1,623) training and ongoing technical assistance (TA; see Florida PBIS Project, 2016, p. 12, for a description of TA provided at the time). Data for each school in the database included district and school identification numbers, school demographic data (verified by the state’s department of education), total and subscale scores from the BoQ, and school-level discipline indicators (e.g., percentage of students receiving ODRs and OSSs).

To be considered for inclusion in this study, a school must have, for the 2015–2016 school year, (a) received TA and implementation monitoring of PBIS, including the submission of BoQ scores, and (b) provided a complete Equity Report of racially disaggregated school enrollment, attendance, and discipline rates (which included the discipline frequency [total number of occurrences, number of students receiving discipline] disaggregated by race). To obtain reliable estimates of discipline risk, included schools also (c) had no less than 10 students in any studied groups (i.e., Black, Latinx, and White). Out of the 1,426 elementary, middle, and high schools meeting the first inclusion criterion (receiving TA for PBIS implementation in 2015–2016), a total of 802 schools (56%) met the second criterion (submitted implementation and discipline data), with 427 of those schools (53%) meeting the third criterion (submitting a complete Equity Report). To ensure reliable estimates of discipline rates, the final criterion (having no less than 10 Black, Latinx, or White students enrolled) resulted in 322 schools being included in the study.

Sample Characteristics

The 322 schools (206 elementary, 73 middle, and 43 high) enrolled a total of 292,490 students. The average enrollment across school levels was 700, 913, and 1,900 for elementary, middle, and high schools, respectively. Of the enrolled students, 54,570 (18.66%) were identified as Black, 82,283 (28.13%) as Latinx, 136,910 (46.81%) as White, and 18,727 (6.40%) as other racial/ethnic identities. ² Analyses revealed the final sample, when compared with all schools receiving project support, to include (a) larger schools that (b) enrolled smaller percentages of students of color and (c) implemented SWPBIS with higher fidelity.

Study Variables and Measures

Data Analysis

Multiple linear regression (MLR) analyses were conducted using Mplus Version 7.31. To account for non-normality in the IVs and DVs, maximum likelihood estimation with robust standard errors (MLR) was utilized. Correlations among the BoQ subscales were expected (Cohen et al., 2007) and were examined for multicollinearity using SPSS (Nie et al., 1970) Version 23.0, but did not exceed .80 (rs ranged from .56 to .77). A Bonferonni correction was used to control for Type 1 error across 10 models, resulting in an alpha level of .005 (Holm, 1979).

Predictors were entered in a base model first. All IVs were entered along with the covariates in each of the 10 models. Each model included a DV of either a risk or risk ratio for a specified group (all students, Black, or Latinx). The IVs of SWPBIS component fidelity (e.g., percent of possible points on Expectations), percentage of students of color, and school size were mean-centered. To investigate the interactions between SWPBIS components and school level, component-by-level interactions for middle schools and high schools were independently added to each of the 10 base models but were not included in the final models due to nonsignificant interactions and minimal R² changes (mean R² change was .004).

SWPBIS Components Prediction of Discipline Risk Rates and Ratios

Classroom Systems predicted lower overall ODR risk (b = −0.15, SE = 0.04, p = .001) and lower ODR risk for Black students (b = −0.22, SE = 0.07, p = .001). Classroom components predicted lower OSS risk for all students (b = −0.08, SE = 0.03, p = .004), for Black students (b = −0.15, SE = 0.04, p < .001), and for Latinx students (b = −0.06, SE = 0.02, p = .005) as well. Conversely, fidelity to Expectations was related to higher OSS risk for Black students (b = 0.13, SE = 0.04, p = .002). Fidelity of the Recognition components was related to more equitable OSS ratios for Black (b = −2.41, SE = 0.82, p < .002) and Latinx students (b = −2.42, SE = 0.81, p < .003). No other components significantly predicted risk rates or ratios for Black or Latinx students.

School Predictors of Discipline Risk Rates and Ratios

Attending a middle or high school predicted discipline risk rates and ratios for most models. ODR risk for Black and Latinx students was positively related to middle (Black b = 28.42, SE = 2.29, p < .005; Latinx b = 15.89, SE = 1.44, p < .005) and high school status (Black b = 30.80, SE = 6.29, p < .005; Latinx b = 16.15, SE = 3.87, p < .005). Similar OSS risk patterns were observed for both groups in middle (Black b = 14.28, SE = 1.50, p < .005; Latinx b = 6.45, SE = 0.66, p < .005) and high school (Black b = 11.83, SE = 2.64, p < .005; Latinx b = 5.31, SE = 1.17, p < .005). For OSS risk ratios, middle school status positively was related to OSS risk ratios for Black (b = 92.86, SE = 22.14, p < .005) and Latinx students (b = 93.18, SE = 22.09, p < .005).

In terms of school size and school racial/ethnic composition, overall ODR (b = −0.007, SE = 0.002, p < .001) and OSS (b = −0.003, SE = 0.001, p < .001) risk was higher in larger schools. In addition, in schools with a higher percentage of students of color, OSS risk was higher for all students (b = 0.07, SE = 0.01, p < .001), Latinx students (b = 0.03, SE = 0.01, p = .005) and Black students (b = 0.06, SE = 0.02, p = .005). However, in schools with a higher percentage of students of color, lower OSS risk ratios were found for Black (b = −2.61, SE = 0.77, p = .001) and Latinx students (b = −2.59, SE = 0.77, p = .001).

Classroom Systems

Consistent with previous research (Childs et al., 2015), when controlling for school demographic variables, only the fidelity of SWPBIS Classroom Systems was related to lower school-wide discipline risk. In addition, results indicated that implementation fidelity to Classroom Systems was related to significantly lower discipline risk for Black students, a phenomenon not directly assessed by previous studies investigating SWPBIS and exclusionary discipline. The lack of significant relationships between BoQ Classroom Systems scores and ratios may be due to differences in instrumentation and scale of investigation. Tobin and Vincent (2011), found evidence of reductions in discipline inequities using the Effective Behavior Supports (EBS) Survey, which may have captured variance in discipline equity that is explained by differences in instructional practices (e.g., instruction’s alignment with student ability, student rates of success) not assessed by the BoQ. In terms of sample size, Cook et al. (2018) found reductions in inequities in a multiple-baseline investigation of a smaller sample of schools receiving direct training in practices similar to Classroom Systems measured by the BoQ.

Expectations

School-level implementation of SWPBIS Expectations demonstrated a significant positive relationship with Black students’ OSS risk—a pattern not found by similar studies in ethnically diverse elementary schools (Tobin & Vincent, 2011). Three hypotheses may be helpful in explaining this finding: climate perception discrepancies, cross-cultural translation, and how Expectations were measured. Regarding climate perception discrepancies, administrators implementing positively stated expectations and rules may be likely to experience a benefit in climate, or organizational health (i.e., resource access, cooperativity, and collegiality; Bradshaw et al., 2008), and assume this benefit to be experienced by all staff and students. However, some evidence (Bottiani et al., 2014) suggests that Black students are less likely than White peers to self-report benefits (i.e., improved teacher–student relationships, fairness) in more organizationally healthy schools. This may lead to administrators being less forgiving of (i.e., more likely to suspend) Black students who defy behavioral expectations considered by administration to be clear.

Second, cross-cultural translation of behavioral expectations may partly explain the association with higher suspension risk for Black students. For instance, “Be Respectful” is a common expectation in schools implementing SWPBIS. However, what “Be Respectful” means likely varies as a function of who is interpreting the expectations. Vincent and colleagues (2011) highlighted that, according to discourse theory, instances of overlapping speech (two persons speaking simultaneously) might be interpreted in some linguistic cultures (i.e., linguistically conditioned sociocultural subtext) as a sign of social engagement, but as a sign of “disrespect” in others. Similar to the authoritarian school discipline models marked by highly specific, White middle-class expectations critiqued by some as “paternalistic” (Whitman, 2008), higher implementation of Expectations might increase the rate at which a mostly White administrative staff (Goldring et al., 2013) evaluates Black students’ behavior with a subtext that is discrepant from the students’ subtext. For example, establishing the behavioral expectation of “Be Respectful” and an associated rule of “listen to instruction” might be understood by administrators to exclude overlapping speech while understood by some students to include overlapping speech, increasing the likelihood that such behavior is disciplined (Emdin, 2016).

Third, the BOQ Expectations subscale measurement design may contribute to the finding. One potential limitation to the BoQ Expectations subscale is that multiple items on this subscale may remain static over time. Four of the subscale’s five items refer to a single, historical implementation event (i.e., developing and posting expectations and associated rules with staff involvement), whereas other subscales capture ongoing practices (e.g., current rates of reinforcement, team meetings, and data analysis). Scores on this subscale may not differentiate between schools with outdated and underutilized expectations posters and those that are actively updating and enhancing behavioral expectations. In fact, the Expectations subscale demonstrated the highest average rate of implementation (90.85) and was significantly negatively skewed. Thus, this potential measurement artifact necessitates some caution in the interpretation of this finding. Regarding Latinx students, other measurement limitations (e.g., equitable OSS rates) may explain the lack of a significant relationship between PBIS Expectations and equitable OSS.

Recognition

This investigation also extends Tobin and Vincent’s (2011) finding of school-based reward and recognition practices being associated with more racially equitable suspension rates. Tobin and Vincent used a single item measuring classroom-based positive-to-negative interaction ratios with a 3-point scale. The BoQ, on the contrary, not only includes an item similar to that used by Tobin and Vincent (2011; “Ratios of acknowledgement to corrections are high”), but also includes six other items that measure the school-wide establishment of a recognition system. These items also focus on (a) the prevalence of recognition across campus, (b) the variety of recognition methods used, (c) the use of common language when recognizing students’ behavior, and (d) student and staff involvement in recognition of students’ behavior.

One potential explanation for recognition systems reducing OSS risk for Black and Latinx students involves trust that may be built. Tobin and Vincent (2011) argued that trusting teacher–student relationships may mediate the relationship between high reinforcement-to-correction ratios and disciplinary equity. Indeed, educational interventions that improve student trust of educators (Yeager et al., 2014) have recently been shown to improve long-term discipline equity (Yeager et al., 2017). This finding may also be due to another mechanism that may increase teacher–student trust: counter-stereotypical acknowledgment. McIntosh and colleagues (2014) hypothesized this to occur in a school-wide commitment to “catch students being good,” which builds positive behavior counter-narratives to counteract the lowered or racialized perceptions of Black students’ behavior (Bates & Glick, 2013; Downey & Pribesh, 2004; Halberstadt et al., 2020) held by a mostly White education workforce (Goldring et al., 2013) and the general public (Starck et al., 2020).

Another explanation of why racial equity in suspensions was associated with Recognition may reflect the BoQ items related to student input into recognitions and rewards, thereby implicitly reflecting culturally— and developmentally—relevant interests. Providing students of color access to culturally relevant rewards for following rules may shape culturally situated peer pressures such as associations of Black students’ identity with “oppositional culture” (Ogbu, 2004). That is, schools obtaining student input into Recognition practices may be disassociating racial identity from behavioral expectations, such that rule-following behaviors are viewed by Black students less negatively as “acting White” (Ogbu, 2004) and more positively as a demonstration of “Black excellence.” A caveat to this hypothesis is that one would expect such a phenomenon to also associate with equitable ODR rates with the practice of Recognition. This study did not find such an association, but this hypothesis warrants further exploration and investigation (Frisby, 2013).

Lessons and Data Analysis

Fidelity of neither Lesson Plans for Teaching Expectations/Rules nor Data Entry and Analysis Plan was related to lower discipline rates or gaps. This finding may suggest that students may only benefit from the lesson plans of behavioral curriculum to the degree to which they are utilized regularly in Classroom Systems. The lack of significant relations between Data Analysis and discipline equity may be due to the BoQ subscale not directly measuring the degree to which a school is using racially disaggregated data to make decisions (Osher et al., 2015; Scott et al., 2012), a necessary but insufficient step toward equity (McIntosh et al., 2020).

Other School-Level Factors

A number of other variables were related to students’ discipline risk and ratios. Consistent with previous literature, higher discipline rates were observed among secondary schools (Raffaele Mendez & Knoff, 2003), larger schools (Finn & Servoss, 2013), and schools with more students of color (Skiba et al., 2014). Secondary schools also were associated with larger OSS risk ratios (Anyon et al., 2014; Martinez et al., 2016), which may relate to experimental research documenting age 10 years as a “milestone” at which adult perceptions of children’s innocence become racialized (Goff et al., 2014) for behaviors as mundane as walking (Neal et al., 2003). Regarding schools with higher percentages of students of color relating to discipline risk, one hypothesis involves that these schools often have less experienced or qualified staff (Darling-Hammond, 2010) who may refer out for behaviors more quickly. However, more equitable suspension rates in schools with more students of color may be due to educators in these schools holding more individualized (less racialized) views of student behaviors (Hugenberg et al., 2010). Finally, larger schools may be associated with increased discipline risk due to less opportunity for staff and students to develop relationships and trust, given the staff to student ratios (Bottiani et al., 2014).

Limitations

Limitations exist regarding the potential generalizability of findings. The participating schools all were in one southeastern state, receiving TA on SWPBIS. Furthermore, their submission of racially disaggregated data may be indicative of a sample of schools significantly more interested in addressing discipline inequity; however, the 2015–2016 project data indicate that fewer than 10 schools received equity-related TA (Florida PBIS Project, 2016). Moreover, the final sample included schools with larger size, fewer students of color enrolled, and higher implementation fidelity levels than those that were excluded from the original data set. Although school size, percentage of students of color enrolled, and fidelity levels of the SWPBIS implementation components were included as variables in the analyses, findings may not generalize to smaller schools, school with higher proportions of students of color, and schools that implement SWPBIS with lower levels of fidelity.

Higher levels of implementation fidelity may have limited power to detect relationships due to less variability in the sample of included schools. Other threats to internal validity involve the correlational, cross-sectional design from only one school year, namely, 2015–2016. Longitudinal investigations with more recent data are needed to investigate how risk and risk ratios change over time as they relate to SWPBIS implementation fidelity. This investigation also did not include data regarding a number of factors related to the discipline gap, including school administrator variables (Skiba et al., 2014), socioeconomic status (Wallace et al., 2008), and gender identity (Gilliam, 2005; Skiba et al., 2011), which may have accounted for some variance in disciple risk and risk ratios.

Finally, internal validity may be impacted by a couple of measurement issues. The length of an OSS (e.g., 1–10 days) was not included in this analysis as the purpose was to look at risk of the presence of discipline; however, it is possible that discipline equity may also differ in terms of length of discipline actions. Moreover, the average and range of scores in the sample may be partially inflated due to the tendency for self-report measures to produce a positive response bias when completed without an external evaluator (McIntosh et al., 2017).

Implications for Research and Practice

Because many questions remain regarding how to eliminate inequity in schools implementing SWPBIS, the implications below are discussed in the context of both research and practice. A case is made that differential access, selection (e.g., ODRs), and processing (e.g., OSSs) of Black students (Gregory et al., 2017) can be countered by SWPBIS research and practice frameworks integrating (a) rigorous instructional practices to promote equitable access, (b) trusting relationships to promote equitable selection, and (c) culturally responsive models to promote equitable processing.