PROTOCOL: Effects of trauma-informed approaches in schools

The problem, condition or issue

Childhood trauma has been receiving increased attention and it is increasingly being recognized as a significant public health concern (Lang, Campbell, & Vanerploeg, 2015). Trauma exposure involves “actual or threatened death, serious injury, or sexual violence” that is either directly experienced or witnessed, or learning that any traumatic experiences have happened to a loved one, or having repeated exposure to details of traumatic events (APA, 2013, p. 271). Prevalence estimates of trauma experienced in childhood or adolescence vary by type of traumatic event (e.g., physical abuse, neglect, sexual abuse, witnessing violence, natural disasters) and how and when the traumatic experience is measured, but can range between 4% and 71% (Finkelhor et al., 2015; McLaughlin et al., 2013; Saunders & Adams, 2014). Exposure to traumatic events can disrupt brain development and can have immediate and lifelong adverse effects on social, emotional and physical wellbeing, including deficits executive functioning, developmental delays, behavioural health problems, difficulty regulating emotions and behaviour, academic performance and IQ, school behaviour problems, delinquency, substance abuse, and mental health and psychiatric disorders (Anda et al., 2006; Delaney-Black et al., 2002; DePrince et al., 2009; Flannery et el., 2004; Lang, Campbell, & Vanerploeg, 2015; Lansford et al., 2002). In a systematic review specifically examining school-related outcomes of traumatic event exposure, Perfect and colleagues (2016) identified 44 studies that examined cognitive functioning, 34 that examined academic functioning and 24 that examined social-emotional-behavioural functioning. Their findings suggest that youth who have experienced trauma are at significant risk for impairments across various cognitive functions, including IQ, memory, attention and language/verbal ability; poorer academic performance and school-related behaviours such as discipline, dropout and attendance; and higher rates of behavioural problems and internalizing symptoms.

Exposure to different types of trauma have been associated with varying types and complexity of adverse outcomes. Kira and colleagues’ (2012) study of African American and Iraqi refugee youth found that different types of trauma differentially impact different components of cognitive functioning, including perceptual reasoning, working memory, processing speed and verbal comprehension. In another study examining effects of different types of trauma, exposure to violence was found to be associated with depression, separation anxiety and conduct problems, whereas exposure to non-interpersonal traumatic events was associated with phobic anxiety (Briggs-Gowan et al., 2010). Moreover, there is some evidence that the effects of trauma are cumulative, thus youth who experience a greater number of traumatic events are more at risk for adverse outcomes and more complex symptoms through adulthood (Chartier et al., 2010; Cloitre et al.; Hodges et al., 2013). Duke and colleagues (2010) analyzed data from respondents to the 2007 Minnesota Student Survey (n = 136,549), and identified “a significant positive relationship between each adverse event and delinquent behaviors for girls and boys,” (p. e782). While exposure to traumatic or potentially traumatic experiences are associated with a range of short and long-term outcomes, there are multiple pathways through which trauma can impact various domains across the life course. Moreover, not all youth will experience the same traumatic events in the same way and not all youth will develop symptoms following a traumatic experience (Layne et al., 2009).

Given the growing research on trauma and increased knowledge about the prevalence, consequences and costs associated with trauma, there have been increased efforts at the local, state and federal levels to make systems “trauma-informed” (Lang, Campbell, & Vanerploeg, 2015). In an effort to examine the extent to which federal legislation has been proposed to advance trauma-informed practice, Purtle & Lewis (2017) conducted a policy mapping study of federal legislative proposals from 1973 to 2015 that explicitly mentioned trauma-informed practice. The authors identified 49 bills introduced, beginning in December 2009 with the Domestic Minor Sex Trafficking Deterrence and Victims Support Act, and observed a dramatic increase over time with 28 bills being introduced in 2015 alone. Altogether, the 49 bills contained 71 sections that included trauma-informed language, with the highest proportion of those specifically targeting youth in primary and secondary schools (16 sections, 22.5%). For example, the United States Congress established the National Child Traumatic Stress Network (NCTSN) in 2000 through a congressional initiative that is funded by the Substance Abuse and Mental Health Services Administration (SAMHSA). The NCTSN (n.d. a) is a growing network of providers, researchers, and families with a broad mission to improve care and access to services for traumatized children, their families, and communities. The NCTSN offers training, support, and resources aimed at treatment, intervention development, program evaluation, systems change, and the integration of trauma-informed and evidence-based practices in all child-serving systems. The Administration for Children and Families, Center for Medicare and Medicaid Services, the Department of Justice, and the Department of Education have also recognized the impact of child trauma on youth well-being and development and are launching initiatives and implementing policies designed to promote the use and expansion of trauma-informed systems and programs across child-serving organizations and agencies (Lang, Campbell and Vanderploeg, 2015; SAHMSA, 2014).

The promotion and provision of trauma-informed approaches in school settings is growing at a rapid rate across the United States. At least 17 states have been identified in which trauma-informed approaches have been implemented at the school, district, and even state-wide levels (Overstreet & Chafouleas, 2016). This rapid increase in the growth of trauma-informed approaches in schools has been fuelled by a number of local, state, and federal initiatives and increasing support by education related organizations. For example, there are explicit provisions for trauma-informed practices in the Every Student Succeeds Act (ESSA, 2015), the legislation that replaced No Child Left Behind, including training of school personnel in understanding when and how to refer students affected by trauma, and grant programs that provide funding to support services that are based on trauma-informed practices that are evidence-based (section 4108). The promotion of trauma-informed schools is also supported by the National Education Association and state-level agencies have been spearheading efforts to develop guidelines and implement change within and across school systems.

The intervention

Trauma-informed approaches are being promoted and used across child serving systems and constitute a relatively new approach to trauma care for children and youth being served within the child welfare, juvenile justice, mental health and education systems. While trauma-specific interventions, such as Trauma-Focused CBT, are well known and widely used to treat trauma-related symptoms and disorders in both adults and children, trauma-informed approaches to care are distinct from trauma-specific interventions. However, what is essential to a “trauma-informed approach” has not always been clearly operationalized, and the approach and variations of the approach have been referred to in varying ways, e.g. “trauma-informed care,” “trauma-sensitive,” “trauma-informed system” (Hanson & Lang, 2016). To date, there is no consensus on the use of these terms, which makes efforts to both implement and study trauma-informed approaches to care challenging.

While there is much confusion, overlap and misuse of the various terminologies in this rather nascent area of practice and research, we are drawing from SAMHSA and the NCTSN to define trauma-informed approaches for the purpose of this review. SAMHSA (2014) defines trauma-afformed approaches (which the agency uses interchangeably with “trauma-informed care”) as incorporating “key trauma principles into the organizational culture” of the program, agency, or system (p. 9). A trauma-informed approach is thus more akin to a multi-tiered framework such as School-Wide Positive Behavioral Supports (SWPBIS; Chafouleas et al., 2016), and is based on incorporating four key assumptions and six key principles, generalizable to any setting, that are infused across all levels of an organization rather than implementing a prescribed set of practices or interventions (SAMHSA, 2014; see Figure 1).

OPEN IN VIEWER

Figure 1

SAMHSA's trauma-informed approach (Lang et al., 2015)

A trauma-informed program, organization, or system is one that (SAMHSA, 2014, p. 9): 1.

Realizes the widespread impact of trauma an understands potential paths for recovery

The six key principles of a trauma-informed approach include safety; trustworthiness and transparency; peer support; collaboration and mutuality; empowerment, voice and choice; and cultural, historical and gender issues (SAMHSA, 2014).

The NCTSN (n.d. b) adapts the model outlined by SAMHSA to a context specific to youth and families, and describes their model as a “trauma-informed child- and family-service system,” in which:

In essence, a trauma-informed approach is not a standalone intervention that can be delivered in isolation, but rather a framework to guide systems. A trauma-informed approach can include trauma-specific interventions, but trauma-specific interventions alone are not seen as sufficient for achieving optimal outcomes or to influence service systems (SAMHSA, 2014). Hanson & Lang (2016) identify three core domains essential to trauma informed care that they derived from analysing definitions and components across several organizations and authors, including SAMHSA and NTCSN: (a) workforce/professional development, (b) organizational changes, and (c) practice changes.

Examples of trauma-informed approaches implemented in schools

As described above, trauma-informed approaches are complex interventions and involve a number of components at various levels, thus providing some examples of such programs can be helpful in elucidating this complexity. One example of a trauma-informed approach is the Healthy Environments and Response to Trauma in Schools (HEARTS) program that was implemented in elementary schools (K-8^th grade) in the San Francisco Unified School District. HEARTS is a whole-school program developed by the University of California, San Francisco to promote school success for students who have been impacted by trauma (Dorado et al., 2016). This whole-school approach used the response to intervention three tier framework of universal, selected and targeted interventions and included supports at the system, adult (teacher/staff) and student levels at each tier. Activities involved changes in school policies and school-wide practices; training, professional development and consultation for all school staff around trauma-sensitive practices and stress, burnout and secondary trauma; and use of evidence-informed universal, secondary and targeted trauma-informed interventions.

Trauma-informed approaches are also used at the pre-school level. A multi-organization collaboration implemented a trauma-informed approach in head start programs in the Appalachian region of the US (Shamblin, Graham, & Bianco, 2016). This comprehensive, 3 tier model involved the use of trained consultants in the classroom to provide three tiers of services. The first tier was universal consultation, intended to build teacher capacity to deliver an evidence-based social-emotional curriculum (Second Steps or Incredible Years) to children and help teachers understand trauma-informed principles through training and mentoring of teachers. The second tier involved targeted consultation of teachers to develop behaviour plans and specific strategies to address challenging behaviours of individual children in the classroom which take into account the child's trauma experience. The third tier included the provision of intensive services wherein the consultant provided on-site mental health assessment and treatment to children and their families. For children who had experienced trauma, the consultant provided Trauma-Focused Cognitive Behaviour Therapy and/or Parent-Child Interaction Therapy. Additionally, the collaboration provided workforce development training to preschool teacher and other child service providers at various times during the year.

While schools may be implementing trauma-informed approaches, it is unclear to what extent or how much variation there is in what schools are implementing, how much emphasis they are putting on various components (e.g., workforce development versus organizational change versus practice changes) and whether schools may implement trauma-informed approaches differently based on the characteristics of their students, neighbourhood, country or other contextual factors.

How the intervention might work

One out of every four children attending school has been exposed to a traumatic event, which can impact school performance, impair learning, and cause physical and emotional distress (NCTSN, 2008). Moreover, the majority of youth who have experienced trauma do not receive services, and those who are exposed to a potentially traumatic event do not necessarily need an intervention (Layne et al., 2009). Due to the relatively high rates of youth exposed to traumatic events and the negative impacts of those traumatic experiences on academic achievement and life course outcomes, schools represent a natural system in which to help prevent and reduce the adverse effects of trauma and more effectively engage students in the learning process (Chafouleas, Johnson, Overstreet, & Santos, 2016). Trauma-informed schools adopt the trauma-informed approach to “create educational environments that are response to the needs of trauma-exposed youth through the implementation of effective practices and system-change strategies” (Overstreet & Chafouleas, 2015, p. 1). A trauma-informed approach in schools is designed to create a systematic model for schools to decrease the impact of trauma on students (Wiest-Stevenson & Lee, 2016) and more appropriately address academic, behavioral and socio-emotional problems by recognizing and responding to student behavior from a trauma-informed perspective. This is done through a multi-level approach intended to improve the school environment through implementing trauma-informed policies and procedures; increase the ability of school staff to recognize and more effectively respond to students through professional development; and prevent, mitigate and reduce trauma-associated symptoms through evidence-informed practices, leading to improved student academic, behavioral, and socio-emotional outcomes.

As described above, a trauma-informed approach involves strategies implemented at various levels in the school and includes workforce/professional development, organizational change, and practice change using evidence-informed practices (Hanson & Lang, 2016).

Why it is important to do the review

Although one could argue about the necessity or value of schools adopting a trauma-informed approach, trauma-informed approaches are being promoted and used across child serving systems, and the number of states and school districts adopting trauma-informed approaches in schools is growing rapidly (Overstreet & Chafouleas, 2016). While the intent of creating trauma-informed approaches in schools is a noble one, there is relatively little known about the benefits, costs, and how trauma-informed approaches are being defined and evaluated (Berliner & Kolko, 2016). Indeed, it is unclear whether schools adopting a trauma-informed approach (i.e., being a “trauma-informed school”) are effective in reducing trauma symptoms or affecting behavioural or academic outcomes, as the proponents of the movement propose. Adopting a trauma-informed approach in a complex system such as a school building or district is a time consuming and potentially costly endeavour and thus it is important to assess the effects of this approach to inform policy and practice. Also, from our cursory review of the literature, the description of the trauma-informed approaches being implemented in schools vary in terms of the types of strategies used in each of the three areas and the relative emphasis on the three areas (policy/procedures, professional development and practices), thus it is important to conduct a systematic inventory and description of the trauma-informed approaches implemented in schools to more fully understand how this approach is being utilized in schools, and whether variations in the components impact outcomes.

To date, we have not located any systematic reviews specifically examining effects of trauma-informed approaches in schools. We have identified reviews that have examined trauma-specific interventions for adolescents (e.g., Black, Woodworth, Tremblay & Carpenter, 2012; Cary & McMillen, 2012) and those that examine school-based interventions for specific trauma-related disorders, such as PTSD (e.g., Rolfsnes & Idsoe, 2011); however, these reviews are examining effects of trauma-specific interventions rather than trauma-informed approaches. By virtue of these studies’ primary research questions and inclusion criteria, the scope of these reviews were not designed to evaluate other domains of trauma-informed approaches (professional development and organizational changes) and thus cannot comprehensively inform decisions related to whether a trauma-informed approach is effective.

Criteria for including and excluding studies

Procedures for applying inclusion criteria

One reviewer will conduct the initial search in all sources and will save the search results in an electronic format. At this stage, the reviewer will examine titles and abstracts and will discard results that are obviously ineligible (non-empirical report, book review, editorial, adult participants, prior to 2006 etc.). For those that are not obviously ineligible, the reviewer will retrieve the reports, save them in an electronic file, and document the bibliographic information, source, and date retrieved in a database. Two reviewers will then independently screen each of the reports for eligibility using a screening instrument (see Appendix A). The two reviewers will compare the coding and identify all discrepancies. The review team will discuss discrepancies and will resolve all discrepancies through consensus.

Search strategy

We propose to include all studies that meet the inclusion criteria outlined above. We will attempt to identify and retrieve both published and unpublished studies through a comprehensive search that includes multiple electronic databases, research registers, grey literature sources, and reference lists of reviews and relevant studies. 1)

Electronic Databases a

. Academic Search Complete

Search terms and keywords

We will use combinations of terms related to the intervention, population, study design, and setting to search the electronic databases. Database-specific strategies will be explored for each database, including the use of truncation and database-specific limiters and thesauri will be consulted to employ more precise search strategies within each database. Below are examples of the types of terms we anticipate using: 1)

Intervention/condition: trauma-informed OR trauma-sensitive OR “trauma services” OR trauma OR PTSD OR “post-traumatic stress disorder”

AND

Data extraction and study coding procedures

For all studies that pass the eligibility screening process described above, two reviewers will independently code all eligible studies using a structured data extraction form (see Appendix B). The data extraction form includes items related to bibliographic information and source descriptors (e.g., report type, country); study methods (e.g., assignment to condition, outcomes reported); characteristics of intervention (e.g., description of components; role of evaluator, fidelity assessment); participant and setting characteristics (e.g., grade level, race/ethnicity of participants; type/location of schools); and outcome data needed to calculate effect sizes.

Two trained coders, Brandy Maynard and Anne Farina, will code all included studies. If greater than 20 studies are eligible for inclusion in the review, a third coder may be recruited to assist with coding. Maynard is a trained Campbell reviewer and has substantial experience coding intervention studies for systematic reviews. Farina is a doctoral student who also has experience coding intervention studies. Both coders have been involved in the development of the data extraction form and protocol. Moreover, coders will pilot test the code form together using diverse types of studies and will discuss any items that are unclear and ensure mutual understanding of all items. Following pilot testing of the form, two coders will independently code 100% of the included studies. Coders will compare coding and will identify and discuss discrepancies, which will be resolved through consensus. If consensus cannot be reached between the two coders, a third member of the review team will be consulted to resolve the discrepancy. Initial discrepancies will be recorded and inter-rater reliability will be reported.

Criteria for determination of independent findings

We are interested in four primary outcome constructs: trauma symptoms/mental health, academic performance, externalizing behaviors, and socio-emotional functioning. We anticipate that included studies will use multiple measures for each outcome, multiple reports of the same outcome measure, multiple follow-up time points, and possibly more than one counterfactual condition. These circumstances create statistical dependencies that violate assumptions of standard meta-analytic methods. In order to ensure independence of study-level effect sizes, we will include only one effect size estimate from each independent sample on each outcome construct.

For cases in which a study uses multiple measures (i.e., observation and a standardized instrument) of the same construct, we will code data for each measure and create a study level average across the measures. In cases of multiple reports on the same outcome (i.e., parent and child report), we will code data for each report and take an average of all reports. In cases where multiple points of follow-up are provided, we will code follow-up points to conduct a separate analysis for effect sizes comparing studies with similar points of follow-up. In the case of multiple counterfactual conditions, we will select the comparison condition that is most similar to those in the other included studies.

Risk of bias

Two review authors will independently assess risk of bias in all included studies using the Cochrane Collaboration's risk of bias tool (Higgins et al., 2011) for RCTs and the Risk of Bias in Non-Randomized Studies of Interventions (ROBINS-I) -NRSI (Sterne et al., 2014) for QED studies. We will follow procedures for each of the RoB tools to code risk of bias for each study. Following independent coding by two authors, coders will meet to identify any discrepancies, and all discrepancies will be resolved through consensus. If consensus cannot be reached between the two reviewers, a third member of the review team will be consulted.

Risk of bias in each domain will be reported within and across studies in the results section using narrative and graphs. We anticipate that most studies included in this review will be at high risk of bias; thus, we do not plan to restrict analyses based on risk of bias nor do we anticipate using risk of bias as a moderator variable. We plan to present all included studies and provide a narrative discussion of risk of bias to include discussion of the potential limitations of the review as well as implications of bias in the interpretation of the results in the Discussion section of the review.

Statistical procedures and conventions

We will conduct descriptive analyses on variables of interest from all included studies to provide information regarding: •

Study participants

Following descriptive analysis, we will estimate effect sizes for each included study. We will calculate the magnitude of effect using the standardized mean difference effect size with Hedges’ g correction for continuous outcomes and odds ratios for outcomes presented as dichotomous variables. We anticipate that outcomes within a category will be measured using similar metrics and therefore, effect sizes within each category will only use either Hedges’ g or odds ratio metrics in the analysis. If, however, outcomes within one of the outcome categories are measured in different metrics, we will convert them to Hedges’ g using CMA software. For studies in which the unit of assignment (e.g., classroom, school) does not match the unit of analysis (e.g., student) and the primary study authors did not account for clustering in their analysis, we will follow recommendations in the Cochrane Handbook (Higgins et al., 2011, 16.3.4) to derive approximately correct analysis by reducing the size of each trial to its ‘effective sample size’ (Rao, 1992) by dividing the sample size for each group by the ‘design effect’ (1 + (M-1)ICC). We will apply this correction only to findings that are statistically significant as nonsignificant findings will remain nonsignificant without the correction (WWC manual).

Following the estimation of individual study level effects, we will conduct separate meta-analyses to pool studies for each outcome construct. Within each of these domains, included studies may use multiple measures and/or multiple reports (e.g., parent, teacher) of the same measure. These circumstances created statistical dependencies that violate assumptions of standard meta-analytic methods. In order to ensure independence of study-level effect sizes, the mean of different measures of the same construct per reporter (e.g., parent, child) will be taken to estimate a study-level average across the measures by reporter type (child reported measures will be pooled separately from parent reported measures). We will follow standard procedures in CMA version 3.0 to use the mean of the selected outcomes for studies with multiple measures of the same outcome construct. In cases where data for multiple points of follow-up are reported, we will code effect size data at each follow-up point to conduct a separate analysis for effect sizes comparing studies with similar points of follow-up.

To synthesize effects across studies, a weighted mean effect will be calculated by weighting each study level effect size by the inverse of its variance. Random effects statistical models will be used throughout unless a compelling case arises for fixed effect analysis. RCT and QED studies will be pooled to allow for greater statistical power unless there are statistically significant differences between the study design types.

Following the estimation of summary effects, we will conduct a test of homogeneity (Q-test) and the I² statistic will also be used to describe the percentage of total variation across studies due to the heterogeneity rather than chance. We will also construct a forest plot displaying study-level mean effect sizes and 95% confidence intervals for the included studies to provide opportunity for visual analysis of the precision of the estimated effect sizes, detection of studies with extreme effects, and information regarding heterogeneity of studies.

Provided there are a sufficient number of studies, we will conduct moderator analysis to examine characteristics of the study methods, interventions, and student samples that may be associated with effect size. The approach to moderator analysis will be dependent upon the available data. If there are sufficient studies available, meta-regression will be used. If we do not have adequate statistical power for meta-regression, we will use the analog to the ANOVA to conduct moderator analysis using the following variables: 1) student grade level (elementary, middle, high school); 2) program variations (used all three trauma-informed components or only 2). Sensitivity analysis will be conducted to examine the potential effects of outliers and studies not reporting or controlling for group differences. If necessary, additional sensitivity analysis will be conducted if other issues arise that may impede our confidence in the estimated pooled effect size estimates. Moderator analysis will be conducted using Comprehensive Meta-Analysis 2.2 (CMA). We will also assess for publication bias using funnel plots and the Egger's test, both conducted in CMA v. 3.

If there is not a sufficient number of studies to conduct meta-analysis, we will present each of the included studies descriptively and identify deficiencies and gaps in the research base. When possible, we will calculate an effect size per each relevant outcome and provide the point estimate and confidence intervals.

Treatment of qualitative research

We do not plan to include qualitative research.