Protocol for a Systematic Review: Policing Schools Strategies to Reduce Crime,Increase Perceptions of Safety,and Improve Learning Outcomes in Primary and Secondary Schools

The Problem, Condition or Issue

The presence of gangs, drugs, bullying, weapons, and other criminal activity is associated with negative effects on school culture and student learning (e.g., Henry, 2000). Schools and school districts have long turned to police interventions to address these and other challenges to improve school safety (Shaw, 2004). In turn, police departments have also recognized the importance of developing strategies in collaboration with schools, such as special truant units or police-school liaisons, to combat school-based crime and disorder or to develop more positive relationships with youth (Shaw, 2004).

Although school-based police interventions are controversial (see below), American educators and the public are increasingly open to their implementation (Cook, Gottfredson, & Na, 2009). Police presence on school grounds has grown dramatically following several high-profile shootings (Brown, 2006). In particular, the recent massacre at the Sandy Hook Elementary School in December, 2012, has renewed calls for deploying armed police in schools, a proposal offered by both President Obama and the National Rifle Association. Increased police presence and intervention at schools has also been reported in the United Kingdom and other European nations (Smith, 2003).

Research suggests that juvenile violence has been on the decline in the United States since the mid-1990s (Gottesman & Schwarz, 2011), and schools remain safe places for students (relative to their time outside of the school setting). However, a considerable amount of crime and misbehavior has always occurred in educational settings—a result of placing large numbers of children together in one setting. For example, the U.S. Departments of Education and Justice report that during the 2009–2010 school year, 85% of public schools reported that at least one violent crime, theft, or other crime occurred at their school (Robers, Zhang, & Truman, 2012). Over half of administrators surveyed in the 2009–2010 School Survey on Crime and Safety reported at least one student threat to physically attack another person, with or without a weapon (Neiman, 2011). Moreover, in 2007, 8% of students in high school reported being threatened or injured with a weapon during the previous 12 months (Dinkes, Kemp, & Baum, 2009).

The problem of school violence, crime, and disorder is not just an American issue. Estimates of school-related crime from other countries are more difficult to obtain, but the European Observatory on School Violence (EOSV) has been collating data from nations such as the U.K., Germany, France, and Spain (Debarbieux, 2003). For example, the EOSV has found that nearly 6% of U.K. teachers report being threatened or attacked at school, 20% of Spanish students report that they have committed some form of vandalism at school, between 15–46% of German students report carrying a weapon to school, and over 25% of French students report being involved in a physical assault on school property (Debarbieux, 2003).

These international estimates dovetail with a recent World Health Organization (WHO) report that listed violence as the world's priority public health issue (Krug, Mercy, Dahlberg, & Ziwi, 2002). The WHO report has generated a renewed focus on youth violence and, more specifically, on school violence, in such organizations as the United Nations Economic, Social and Cultural Organization (UNESCO, 2007). The European Observatory's offspring, the International Observatory on Violence in the School Environment, was inspired in part by growing interest in this issue by nations around the globe.

The Intervention

Given the high levels of crime and disorder present at schools in the U.S. and in other nations, it is not a surprise that police and educators would collaborate in order to combat it. Shaw's (2004) international review of police-school practices categorized such strategies into three groups: (1) school-based police officer programs; (2) police as ‘teachers’ (i.e., curriculum approaches such as the police officer teaching the students about drug, gang, or sexual abuse prevention); and (3) comprehensive or broad-based liaison programs in which police and other social service agencies are involved with the schools. Shaw reported that non-curricular policing programs (e.g., police patrol strategies, problem-oriented or community policing tactics, assigning school resource officers or police-school liaison offices, or other measures) have been implemented in a wide range of countries, including the U.S., U.K., Canada, Germany, Denmark, Australia, the Netherlands, Belgium, Estonia, Poland, and South Africa, at least through 2003. Shaw's review was globally comprehensive but not systematic; she concluded her synthesis (2004, p. 26) with a future research agenda that includes “a more comprehensive, comparative review of police-school interventions” and “the systematic collection and documentation of good practice models and tools.”

Why it is Important to do the Review

To our knowledge, there has not previously been a systematic review of evaluative evidence specific to the “policing schools” area, with the exception of police-led curriculum efforts such as Drug Abuse Resistance Education (D.A.R.E) (Ennett, Tobler, Ringwalt, & Flewelling, 1994; West & O'Neal, 2004). In other words, while police departments maintain a significant presence in many schools, there is not yet a research-based consensus on whether School Resource Officer or special police-school liaison programs, or other policing schools strategies make schools safer and reduce criminal or delinquent behavior and non-criminal student misbehavior (such as bullying and harassment). There is also no agreement on whether the presence of police enhances perceptions of safety by students or staff.

Critics of school-based policing argue that police in schools represent an unnecessary expense and may have unintended negative consequences for students, teachers, and staff (e.g., Petteruti, 2011). For example, police presence in the schools could lead to behavior normally handled informally by schools to be criminalized by arrest and formal charges (e.g., Petteruti, 2011). The National Association of School Psychologists (2013) recently issued a brief that summarized a number of potential harms from increasing police and security at schools, including sending a signal to students that their campus is unsafe, increasing fear, and undermining school climate.

In 2012, Petrosino, Guckenburg and Fronius reported on a scoping study using systematic review techniques to assess the state of the evaluation evidence in this area. They located 11 quasi-experiments available that had a comparison group. Only one would have met the minimum evidentiary standards of the What Works Clearinghouse in education, and most of the studies would have been scored a “3” by the Maryland Scale of Scientific Methods: a common approach to rating evaluations in justice. However, they did not attempt any meta-analysis, and their review covered the literature through 2009.

This proposed systematic search is designed to build upon Shaw's (2004) work and recommendations, and the earlier Petrosino, Guckenburg and Fronius (2012) scoping study, to provide a systematic review of research on the effects of policing schools strategies.

How the Intervention Might Work

A general theory of change for policing schools strategies is provided in the Figure in Appendix A. As the Figure denotes, there are a number of anticipated interventions that we expect fall under the “policing schools” strategies, including various police patrol strategies, school resource officer allocation, police-school liaisons, safe routes/corridor programs, and community and problem-oriented policing strategies specifically targeting the schools. Programs that solely involve the police officer teaching a specific prevention curriculum such as D.A.R.E., G.R.E.A.T., or sexual abuse prevention are not eligible, though interventions that combine curricular and non-curricular strategies would be included. Policing programs that involve providing information on school crime or data to schools would be included, if the police are not teaching a curriculum or lesson plan.

These programs and strategies are expected to increase police presence activities and interaction with students and staff at the schools. Police presence serves as a deterrent to crime and misbehavior, and police establish relationships that allow them to gather information to prevent or intervene with students who may be planning a crime. In turn, this should reduce criminal behavior and non-criminal student misconduct; increase perceptions of safety by students, staff, and parents; and lead to more positive relationships between police with young persons and with the schools. This should also result in increased attendance, test scores, and grades, and improved perceptions of school climate. However, there is the possibility of unintended negative consequences that police presence leads to heavy surveillance, resulting in police coming across more misbehavior by youth, leading to overuse of arrest for behavior normally handled informally by schools and to more push-outs of students (disproportionately affecting racial and ethnic minorities, and youth with disabilities). Police presence could also lead to increased fear of crime and misbehavior by sending a signal to the school community that the building is unsafe.

The types of learning outcomes we anticipate are grades, attendance/absenteeism, dropout, test scores, and perceptions of school climate. The types of non-educational outcomes we anticipate are measures of safety such as police reports, calls for service, arrests, school disciplinary reports, self-reports of student behavior, self-reports by staff of safety and fear, self-reports by students of safety and fear, and measures of parent fear for their youth at school.

Potentially positive effects

What are the effects of policing strategies in primary and secondary schools (K-12 in USA) on measures of crime and non-criminal student misbehavior (e.g., certain levels of bullying, harassment, etc.) in and around schools?

Potentially negative, unintended consequences

4

What are the effects of policing strategies in primary and secondary schools on arrests, formal sanctions, or push-outs of students?

Criteria for including and excluding studies

For this project, we will only include those studies that have the following characteristics:

The study must evaluate a policing strategy whose primary approach does not involve police officers teaching a curriculum. The program must focus on a goal such as reduced crime or increased school safety. This would include strategies such as increasing the number of police patrols on school grounds. Police must either lead or be a strong partner in the strategy. Broader programs that involve multiple agencies with minimal police involvement (e.g., police just attend a meeting of concerned agencies about a school problem) will not be included. We will not include evaluations of programs that include the police along with many other components (e.g., police collaborations with health and other social service agencies). In such studies, one cannot isolate the effects of the police action. The review will not include studies of the impact of academy or educational training on police performance.

There have been meta-analyses of police-taught curricula such as D.A.R.E. (e.g., see Ennett et al., 1989). The role of the officer (as a teacher) is very different in such strategies in that the focus is to deliver universal prevention strategies to students, not to gain immediate crime prevention and safety benefits at the schools. Indeed, there have been over a dozen high-quality studies of D.A.R.E., but not one collected crime outcomes; instead, they focused on knowledge about drugs, attitudes and perceptions of youth, and self-reported drug use (Petrosino, 2005). Thus, evaluations of police-led education programs such as D.A.R.E., G.R.E.A.T., or sexual abuse prevention curricula will not be included. However, programs that involve the combination of a curriculum component with a non-curricular policing strategy will be included.

Search strategy for identification of relevant studies

We will rely on five major strategies to identify eligible evaluations published through 2013. These are:

Electronic searches of bibliographic databases. Researchers will use available online resources and databases at WestEd, the University of Pennsylvania, George Mason University, and Bridgewater State University, including Criminal Justice Abstracts, the National Criminal Justice Reference Service (NCJRS), Sociological Abstracts, and the Education Resource Information Center (ERIC). See Appendix B for a listing of electronic databases that will be searched.

Retrieving and final screening of studies

Search methods will usually result in a large number of citations and abstracts. The review author conducting the search will save the results from the searches described earlier and mark those for which full-text should be retrieved. This will be examined by at least one other co-author to ensure that there is agreement on the selections. Since all three authors are experienced in conducting reviews of this type, no training is necessary.

Many of these will be easily excluded as not being relevant to the proposed review. In some cases, however, they will identify potentially eligible studies. The full text documents of those potentially eligible studies will be retrieved and screened by the co-authors before the study can be formally included. We expect to retrieve the full text of most articles electronically. We also have access to interlibrary loan through Bridgewater State College. When a full text report is received, two of the review co-authors will read it to ensure that it meets the eligibility criteria. If there is no agreement on the inclusion of a particular study, it will be excluded and the exclusion documented in the final report.

We will also establish a bibliographic reference database to maintain a log of all included and excluded studies. The log includes a field that allows the research team to document the reason for exclusion.

Extracting information from each study

We have designed a preliminary instrument to guide us in recording information from each study (see Appendix C). Although the instrument contains several open-ended items, these will be collapsed when appropriate into a smaller number of categories to permit further analysis. For example, items such as “how equating was performed” can be collapsed into three or four larger categories representing the most frequent responses (e.g., discontinuity, covariate matching, propensity score, post-hoc statistical matching) and an “other” category that captures all those responses that do not fit into the most common methods of equating in this set of studies. The instrument has items in the following areas:

Statistical procedures and data synthesis

Data will be entered into the Comprehensive Meta-Analysis (CMA), version 2. We will use CMA to statistically combine results from the evaluations.

We will report standardized mean differences (Hedges g), which is similar to Cohen's effect size measure, but it includes a small sample bias correction for d, for each main effect size analysis. Hedges g (like Cohen's d) is a very flexible effect size metric, and many formulae are available to estimate it from information often reported in evaluation articles (e.g., statistical test data, probability levels). The effect sizes will be inverse-variance weighted when combined; procedures are available in CMA to do this. Forest plots will be used to display the results from the effect sizes. The plot will display, for each study, the effect size, confidence intervals, and significance level. The plot will also display the same for the average effect across studies. Note that all analyses will be reported assuming a random effects model, and the estimate will be weighted by the inverse variance. When describing results in the text, we will report the effect size, the confidence intervals, and whether the analysis indicates that the result is statistically significant. Note that a positive effect size indicates that the strategy had a beneficial impact on the outcome (e.g., a decrease in crime or an improvement in grades); a negative value effect size indicates a harmful effect for intervention (0 means there is absolutely no difference in values of the outcome measure between treatment and control groups).

Because of the likely heterogeneity in interventions, samples, and settings, we will assume random effects models in our analyses, which tends to be more conservative than the fixed effects approach. For our analyses, we will conduct tests for heterogeneity to determine if the average effect size is a good representation of the sample of studies being used in the analysis. We anticipate the heterogeneity will be present, given the suspected variations in intervention type, sample populations, and the like in these studies.

Using CMA, we will confirm heterogeneity in each summary analysis (of each outcome at each of the two time intervals: first and longest) through the Q-Value, which is reported as a summary indicator of the extent of variance across studies in the sample. We will also report I-squared and tau-squared values as additional checks on heterogeneity. However, moderator analyses have to be approached carefully, as they are often based on small numbers of studies (the “small cell” problem), and such analyses can be significant by chance if large numbers of variables are considered (the “capitalizing on chance” problem). We anticipate examining a small number of moderating variables as a source of heterogeneity, by comparing the effect sizes (assuming a random effects model) for the following potential moderators:

Intervention type. An important policy question is whether communities and schools are getting more “bang for the buck” using one particular approach or another. We will recode the detailed intervention categories into discrete groups for analysis and compare the average effect sizes for these groups.

We are open to including other moderating analyses, if appropriate to conduct (i.e., there are enough studies to warrant subgroup analyses), such as looking at the relationship between the characteristics of the schools and outcomes.

Treatment of non-experimental research

As mentioned earlier, we will collect non-experimental evaluations and correlation analyses. Our plan is to summarize them in a separate section, and in particular, to use them to highlight the context, theory, implementation, process, mechanisms, and challenges to policing schools programs.

Treatment of qualitative research

It is very unlikely, because of our focus on experiments and quasi-experiments, and our focus on quantifiable outcomes that can be converted into an effect size metric, that we will uncover much qualitative research. However, we will code the presence or absence of ancillary qualitative studies, what the studies focused on, and what the main findings are. Certainly, qualitative data from the experiments and quasi-experiments will be used to illuminate three particular areas: (1) the context for the intervention; (2) the theory or mechanisms by which the program is supposed to impact the ultimate outcomes; (3) the quality and nature of the intervention and comparison condition; and (4) potential issues that may be important to flush out and discussed in relation to the outcome or moderating variable analyses.

Treatment of economic data

We will report on any economic data included in the primary studies that are included in the review. Our prior experience in conducting reviews is that cost data are rarely included, but it remains worthwhile to include. This includes information on the costs of the program, any analysis of the cost-effectiveness of the intervention (e.g., the cost per school), and cost-benefit studies (e.g., the sum costs and benefits of the program). It is important that this information be linked in some way to the primary outcome studies so that it can be retrieved during the searches.

Authors' responsibilities

By completing this form, you accept responsibility for preparing, maintaining and updating the review in accordance with Campbell Collaboration policy. The Campbell Collaboration will provide as much support as possible to assist with the preparation of the review.

A draft review must be submitted to the relevant Coordinating Group within two years of protocol publication. If drafts are not submitted before the agreed deadlines, or if we are unable to contact you for an extended period, the relevant Coordinating Group has the right to de-register the title or transfer the title to alternative authors. The Coordinating Group also has the right to de-register or transfer the title if it does not meet the standards of the Coordinating Group and/or the Campbell Collaboration.

You accept responsibility for maintaining the review in light of new evidence, comments and criticisms, and other developments, and updating the review at least once every five years, or, if requested, transferring responsibility for maintaining the review to others as agreed with the Coordinating Group.

Publication in the Campbell Library

The support of the Coordinating Group in preparing your review is conditional upon your agreement to publish the protocol, finished review, and subsequent updates in the Campbell Library. The Campbell Collaboration places no restrictions on publication of the findings of a Campbell systematic review in a more abbreviated form as a journal article either before or after the publication of the monograph version in Campbell Systematic Reviews. Some journals, however, have restrictions that preclude publication of findings that have been, or will be, reported elsewhere and authors considering publication in such a journal should be aware of possible conflict with publication of the monograph version in Campbell Systematic Reviews. Publication in a journal after publication or in press status in Campbell Systematic Reviews should acknowledge the Campbell version and include a citation to it. Note that systematic reviews published in Campbell Systematic Reviews and co-registered with the Cochrane Collaboration may have additional requirements or restrictions for co-publication. Review authors accept responsibility for meeting any co-publication requirements.

I understand the commitment required to undertake a Campbell review, and agree to publish in the Campbell Library. Signed on behalf of the authors:

Form completed by: Date: 12/20/13