PROTOCOL: Broken Windows Policing to Reduce Crime: A Systematic Review

Linked article

Systematic review

Sources of Support

Background for the Review

Crime policy scholars, primarily James Q. Wilson and George L. Kelling, and practitioners, such as Los Angeles Police Chief William J. Bratton, have argued for years that when police pay attention to minor offenses—such as aggressive panhandling, prostitution, and graffiti—they can reduce fear, strengthen communities, and prevent serious crime (Bratton & Kelling, 2006; Wilson & Kelling, 1982). Spurred by claims of large declines in serious crime after the approach was adopted in New York City in the early 1990s, dealing with physical and social disorder, or “fixing broken windows,” has become a central element of crime prevention strategies adopted by many American police departments (Kelling & Coles, 1996; Sousa & Kelling, 2006).

In their seminal “broken windows” article, Wilson and Kelling (1982) argue that social incivilities (e.g., loitering, public drinking, and prostitution) and physical incivilities (e.g., vacant lots, trash, and abandoned buildings) cause residents and workers in a neighborhood to be fearful. Fear causes many stable families to move out of the neighborhood and the remaining residents isolate themselves and avoid others. Anonymity increases and the level of informal social control decreases. The lack of control and escalating disorder attracts more potential offenders to the area and this increases serious criminal behavior. Wilson and Kelling (1982) argued that serious crime developed because the police and citizens did not work together to prevent urban decay and social disorder.

The available research evidence on the connections between disorder and more serious crime is mixed. In the Netherlands, Keitzer et al. (2008) conducted six field experiments examining the links between disorder and more serious crime and concluded that dealing with disorderly conditions was an important intervention to halt the spread of further crime and disorder. Skogan's (1990) survey research found disorder to be significantly correlated with perceived crime problems in a neighborhood even after controlling for the population's poverty, stability, and racial composition. Further, Skogan's (1990) analysis of robbery victimization data from thirty neighborhoods found that economic and social factors' links to crime were indirect and mediated through disorder. In his reanalysis of the Skogan data, Harcourt (1998, 2001) removed several neighborhoods with very strong disorder-crime connections from Newark, New Jersey, and reported no significant relationship between disorder and more serious crime in the remaining neighborhoods. Eck and Maguire (2006) suggest that Harcourt's analyses do not disprove Skogan's results; rather his analyses simply document that the data are sensitive to outliers. Indeed, the removal of different neighborhoods from Harcourt's analysis may have strengthened the disorder-crime connection (Eck & Maguire, 2006).

In his longitudinal analysis of Baltimore neighborhoods, Taylor (2001) finds some support that disorderly conditions lead to more serious crime. However, these results varied according to types of disorder and types of crime. Taylor (2001) suggests that other indicators, such as initial neighborhood status, are more consistent predictors of later serious crimes. Using systematic social observation data to capture social and physical incivilities on the streets of Chicago, Sampson and Raudenbush (1999) found that, with the exception of robbery, public disorder was not significantly related to most forms of serious crime when neighborhood characteristics such as poverty, stability, race, and collective efficacy were considered. Sampson and Raudenbush's findings have been criticized because their social observation data on disorder were collected during the day rather than at night (Sousa & Kelling, 2006) and based on their decision to test a model in which disorder mediates the effects of neighborhoods characteristics on crime rather than neighborhood characteristics mediating the effects of disorder on crime (Jang & Johnson, 2001). In another analysis, Xu et al. (2005) point out that Sampson and Raudenbush (1999)'s results actually are supportive of broken windows theory.

The scientific research evidence on the crime control effectiveness of broad-based broken windows policing strategies, such as quality-of-life programs and order maintenance enforcement practices, is also mixed. However, there seems to be more research evidence supporting the crime prevention value of broken windows policing strategies than refuting it. The New York City Police Department (NYPD) provides the best known example of a macro policy of order maintenance, as it is well documented that officers were more aggressive in making arrests for minor offenses (Sousa & Kelling, 2006). Using misdemeanor arrests as a proxy for order maintenance activities, Kelling and Sousa (2001) found that the NYPD strategy was associated with a significant reduction in violent crime in the 1990s, after controlling for economic, demographic, and drug use variables. A similar analysis by Corman and Mocan (2002) found that increased misdemeanor arrests in New York City during the 1990s had a significant impact on robbery and motor vehicle theft controlling for economic and criminal justice factors.

Two more recent studies of the impact of order-maintenance policing in New York City support the idea that policing disorder prevents more serious crime. Rosenfeld, Fornango, and Renfigo (2007) analyzed the effects of order-maintenance arrests on precinct-level robbery and homicide trends in New York City between 1988 and 2001, and concluded that the approach generated small but significant crime reduction gains. Using a different analytic approach, Messner and his colleagues (2007) analyzed homicide trends in 74 New York City police precincts between 1990 and 1999, and found that misdemeanor arrests generated significant reductions in total homicide rates with the largest impacts on gun homicide rates. This is consistent with Fagan, Zimring, and Kim's (1998) observation that the kinds of changes in policing associated with broken windows policing might be effective, in part, by taking more guns off the streets through increased police-citizen contacts. More recently, Zimring (2012) suggests that the police action in reducing crime in New York City during the 1990s was not the broken windows policing described by Kelling and Coles (1996) rather it more closely resembled a tight police focus on crime hot spots.

There are also policy evaluations implemented in other jurisdictions that support the perspective that dealing with disorderly conditions generates crime control gains. Two separate randomized controlled trials of disorder policing strategies implemented within a problem-oriented policing framework found the strategy resulted in significant reductions in calls for service to the police in Jersey City, New Jersey (Braga et al., 1999) and Lowell, Massachusetts (Braga & Bond, 2008). The Safer City Initiative, an intervention launched by the Los Angeles Police Department to reduce homeless-related crimes by addressing disorderly conditions associated with homeless encampments, generated modest reductions in violent, property, and nuisance street crimes (Berk & MacDonald, 2010). Other macro-level analyses have generated results supportive of broad-based policing disorder strategies. In California, controlling for demographic, economic, and deterrence variables, a county-level analysis revealed that increases in misdemeanor arrests was associated with significant decreases in felony property offenses (Worrall, 2002). Finally, an analysis of robbery rates in 156 American cities revealed that aggressive policing of disorderly conduct and driving under the influence reduces robbery (Sampson & Cohen, 1988).

Many observers, however, argue that it is very difficult to credit a generalized order maintenance strategy with the crime drop in New York in the 1990s. The NYPD implemented the broken windows strategy within a larger set of organizational changes framed by the Compstat management accountability structure for allocating police resources (Silverman, 1999). As such, it is difficult to establish the independent effects of broken windows policing relative to other strategies implemented as part of the Compstat process (Weisburd et al., 2003). Other scholars suggest that a number of rival causal factors, such as the decline in the city's crack epidemic, played a more important role in the crime drop (Blumstein, 1995; Bowling, 1999). Some academics have argued that the crime rate was already declining in New York before the implementation of any of the post-1993 police reforms, and that New York's decline in homicide rates were not significantly different from declines experienced in surrounding states and in other large cities that did not implement aggressive enforcement policies during that time period (Karmen, 2000; Eck & Maguire, 2006).

Other evaluations have not found significant crime prevention gains associated with broad-based policing disorder strategies. A recent reanalysis of the Kelling and Sousa (2001) data did not find that a generalized broken windows strategy, as measured by increased misdemeanor arrests, yielded significant reductions in serious crimes in New York City between 1989 and 1998 (Harcourt & Ludwig, 2006). A quasi-experimental evaluation of a quality-of-life policing initiative focused on social and physical disorder in four target zones in Chandler, Arizona did not find any significant reductions in serious crime associated with the strategy (Katz et al., 2001). An evaluation of a one-month police enforcement effort to reduce alcohol and traffic-related offenses in a community in a Midwestern city did not find any significant reductions in robbery or burglary in the targeted area (Novak et al., 1999). Similarly, a randomized controlled experiment of broken windows policing in three towns in California (Redlands, Colton, and Ontario) found no significant effects on fear of crime, police legitimacy, collective efficacy, or perceptions of crime and social disorder (Weisburd et al., 2011).

Given the mixed policy evaluation findings, a systematic review of the existing empirical evidence is warranted.

Objectives for the Review

This review will synthesize the existing published and non-published empirical evidence on the effects of broken windows policing interventions and will provide a systematic assessment of the crime reduction value of broken windows policing in neighborhoods. It is anticipated that this review will help inform policy makers and police department decision makers regarding the continued use of broken windows policing interventions to reduce crime in neighborhoods. Many police agencies in the United States, United Kingdom, Australia, and other nations currently use broken windows policing as a core crime control strategy and a critical examination of the existing evidence is warranted.

Methods

Criteria for Inclusion and Exclusion of Studies in the Review

Search Strategies for Identification of Studies

To identify the studies meeting the criteria of this review, the following nine search strategies will be used: 1.

A keyword search will be performed on an array of online abstract databases (see lists of keywords and databases below).

Finally, an information specialist will be engaged at the outset of our review and at points along the way in order to ensure that appropriate search strategies were used to identify the studies meeting the criteria of this review. ³ For instance, we will work with the information specialist to conduct an extensive Google search for eligible studies that will seek to identify eligible studies by using the search terms below as well as including words such as “research,” “evaluation,” and “program analysis.” The information specialist will be consulted on the use of Google Scholar to identify studies that cite seminal broken windows studies (e.g. Wilson & Kelling, 1982; Kelling & Coles, 1996) and on the use of publisher databases and indexes (e.g. Wiley, Sage, and Springer) to identify potentially eligible studies.

The following fifteen databases will be searched: 1.

Criminal Justice Periodical Index

The following terms will be used to search the fifteen databases listed above: 1.

Broken windows AND police

In addition, two existing registers of randomized controlled trials will be consulted. These include (1) the “Registry of Experiments in Criminal Sanctions, 1950-1983 (Weisburd et al., 1990) and (2) the ”Social, Psychological, Educational, and Criminological Trials Register“ or C2 SPECTR being developed by the United Kingdom Cochrane Centre and the University of Pennsylvania (Turner et al., 2003). Two additional online databases of rigorous studies in policing will be reviewed: the Evidence-Based Policing Matrix (http://www.policingmatrix.org) and the US Office of Justice Program's CrimeSolutions.gov website.

Selection of studies

The reviewers will screen abstracts and leads to potentially eligible studies and decide which full-text reports should be acquired. Only the full-text papers of titles and abstracts indicating, or potentially indicating, an evaluation of a broken windows policing or an empirical analysis of the theoretical connections between disorder and crime will be obtained. In cases of ambiguity, the full text of the study will be obtained in order to properly determine whether an eligible study design was used. Studies that use randomized controlled or rigorous quasi-experimental designs will be considered for inclusion in the main review. Correlation and observational studies without control groups will be noted and analyzed separately. These descriptive studies will not be included in the formal analysis reporting the findings of the review.

Assessment of Methodological Quality and Bias

Studies meeting the eligibility set forth above will be coded for a range of characteristics related to methodological quality including the definition criteria used to identify the units of analysis, the statistical tests used to determine crime prevention effectiveness, the measurement of displacement, the violation of randomization procedures, case attrition from the study, and the subversion of the experiment by participants. Farrington (2003) proposes five easily understood methodological criteria to assess the methodological quality of evaluation studies. These criteria include statistical conclusion validity, internal validity, construct validity, external validity, and descriptive validity.

As appropriate and possible, the role of the various methodological factors on the observed empirical results will be assessed. The methodological quality information will be reported in tabular form and presented along with the effect size information. If a meta-analysis is performed, effect sizes will be compared on the above dimensions to assess for potential bias. However, it is important to recognize that eligible studies may not detail or even mention implementation issues. Indeed, all field experiments face implementation difficulties and care will be taken not to artificially downgrade the value of certain studies simply because one provided an open account of potential process problems.

Publication bias presents a strong challenge to any review of evaluation studies (Rothstein, 2008). Campbell reviews, such as ours, take a number of steps to reduce publication bias. Our extensive search procedures, the use of an information retrieval specialist (Phyllis Schultze), and the mobilization of an extensive network of police scholars make it unlikely that relevant unpublished works would remain hidden from this review. Nevertheless, we will use the trim-and-fill procedure (Duval & Tweedie, 2000) to estimate the effect of potential data censoring, such as publication bias, on the outcome of the meta-analyses.

Data Management and Extraction

The reviewers, with the help of a trained research assistant, will extract information from the full text report on the characteristics of the study using a carefully designed data extraction instrument (see coding instruments included in Appendix A). A content analysis will be conducted on the full text of the report and the data extraction instrument will capture data on the relevant dimensions of this review. These dimensions include: a complete description of the treatment, methods used to define and identify targeted areas, research design and statistical techniques, threats to the research design, crime outcome measures, and alternative outcome measures. When important information is missing from available study reports, the original researchers will be contacted, if possible, to determine if they can supply that information.

The two review authors, Braga and Welsh, will independently code each eligible study. The authors will then compare the two completed coding instruments for each eligible study to identify any divergent coding of study material. Where there are discrepancies, the authors will jointly review the study and determine a final coding decision. This process will identify and resolve any coding differences.

Finally, pre-test information will be coded for eligible quasi-experimental studies when available. This will allow the calculation of pre-post effect sizes for eventual inclusion in the meta-analyses described below.

Determination of Independent Findings

A single evaluation of broken windows policing intervention may provide data on multiple outcome measures. For example, the randomized controlled trial of the Lowell policing disorder intervention presents an array of outcome measures including assault, robbery, burglary / breaking and entering, larceny / theft, disorder / nuisance, and total citizen calls for service data (Braga & Bond, 2008). A separate study reported on the community perceptions of the value of the policing disorder intervention in addressing neighborhood crime and safety conditions (Braga & Bond, 2009). Policing disorder interventions are often targeted at underlying environmental conditions that may be responsible for generating a variety of offenses such as violent crimes, property crimes, and drug crimes. The treatment could have varying effects on offending trends and patterns in different crime categories.

For cases such as this with multiple findings from the same sample, each will be examined independently to decide how to either combine the findings or to choose the one that best represents the study. Some policing disorder interventions may be designed to deal with a specific problem, but other programs may also target some secondary problems and report outcomes for these as well. In these cases the effect size for the primary problem will be reported. For instance, while multiple outcomes are reported, the Lowell evaluation clearly identifies the pre-test and post-test counts of total citizen calls for service in the treatment hot spots relative to the control hot spots as the primary outcome measure (Braga & Bond, 2008).

However, some studies might have multiple primary outcomes. Analyzing these separately would clearly lead to problems regarding statistical dependence of outcomes. As such, we will code a maximum of three primary outcomes, with the criteria of choosing the maximum, moderate/median effect size and minimum effect size to offer flexibility in calculating an overall effect size for such studies. The same strategy will be used for any studies reporting the same outcome multiple times with different types of data (i.e. a study evaluating the impact of a policing disorder on “robbery” may use robbery incidents and robbery calls for service as primary outcome measures). Finally, some studies may involve multiple sites—i.e. policing disorder program delivered by one police department/taskforce to specific problems in multiple areas within a city. Such cases will be treated as one study with sub units, and independent effect sizes for primary outcomes will again be created in the same manner as above.

Statistical Procedures and Conventions

Analysis of outcome measures across studies will be carried out in a uniform manner and, where appropriate and possible, involve quantitative analytical methods. Meta-analytic procedures will be used to combine data from studies. We will be limiting our meta analyses to studies that evaluate the impacts of broken windows policing on crime outcomes at the area level. As described by Lipsey and Wilson (2001), it is problematic to combine effect sizes from studies with very different units of analysis (such as combining studies focused on people with studies focused on places).

For eligible studies, with enough data present, effect sizes will be calculated using the standardized measures of effect sizes as suggested in the meta-analytic literature (e.g. see Lipsey & Wilson, 2001). Mean effect sizes will be computed across studies and we will use a correction such as the inverse variance weight for computing the associated standard error. The odds-ratio will be the effect size of choice for all outcomes of a dichotomous or binary nature. For eligible studies that contrast two groups that have a continuous underlying distribution, the standardized mean difference effect size (also known as Cohen's d; see Rosenthal, 1994) will be used in our meta-analysis to synthesize results. This effect size measure would be appropriate for eligible evaluations that contrast an outcome measure, such as mean incident counts, for treatment and control areas over pre-test and post-test observation periods. The Effect Size Calculator, developed by David B. Wilson and available on the Campbell Collaboration's web site, will be used to calculate odds-ratio and standardized mean difference effect sizes for reported outcomes in each study. ⁴ Biostat's Comprehensive Meta Analysis Version 2.2 will then be used to conduct the meta-analysis of effect sizes. In our proposed meta analyses, we will use a random-effects model. ⁵

We will examine four contextual or moderating features of broken windows policing approaches. To assess these contextual or moderating features, we will use the analog to the ANOVA method of moderator analysis (see Lipsey & Wilson, 2001) for categorical moderator variables and meta-analytic regression analysis for continuous moderator variables or analyses involving multiple moderators.

First, consistent with other systematic reviews of policing interventions (e.g. Braga & Weisburd, 2012), we will examine research design type as a moderator of program effect sizes. Relative to quasi-experimental designs, randomized controlled trials have been revealed to generate more conservative effect sizes (Weisburd et al., 2001; Welsh et al., 2011). Second, we will examine policing disorder program type as a moderator of program effect sizes. As our literature review suggests, there seems to be varying approaches that police departments take to controlling disorder that could produce different effects on crime (e.g., zero-tolerance misdemeanor arrest strategies v. community problem-solving approaches to recurring disorder problems). Third, we will conduct a moderator analysis of the impacts of policing disorder programs on different types of crime outcomes (e.g. violent crimes v. property crimes). It is possible that a police focus on disorder could produce disparate impacts on particular types of crimes.

Fourth, and finally, we will examine the unit of analysis as a moderator of program effect sizes. It is possible that the types of areas (e.g. small crime places vs. larger neighborhoods) subjected to policing disorder strategies will matter in program effect sizes. Consistent with the literature on hot spots policing (Braga, 2007), broken windows interventions concentrated in small areas may generate stronger crime reduction impacts relative to treatments diffused across larger geographic units. Therefore, we will conduct separate analyses based on the geographical unit (e.g. city-wide, neighborhood, precinct, or defined hot spot place such as street corners and block faces).

Treatment of Qualitative Research

Qualitative research on crime and disorder outcomes will not be formally included in this systematic review. Qualitative insights on the crime prevention value of broken windows policing will be included as descriptive information in the review report. If the search strategies reveal a number of qualitative studies, the authors will engage a qualitative researcher to assist in future updates to this review with a synthesis of qualitative evaluation measures.

Timeframe

Updating the Review

In accordance with Campbell Collaboration guidelines, we will update this review once every three years.

Conflict of Interest

With colleagues, Braga has conducted two randomized controlled trials that found disorder policing, implemented within a problem-oriented policing framework, to be effective in reducing citizen calls for service to the police in crime hot spots (see Braga et al., 1999; Braga & Bond, 2008). Although Braga doesn't have an ideological bias towards the effectiveness of broken windows policing interventions, it may be uncomfortable for him to report findings in this review that contradict the findings of his evaluations or related evaluations conducted by his colleagues. Welsh does not have any conflicts of interest in conducting this review.