Tracking Quality of Police Actions in a Victim Contact Program

Abstract

In policing, quality of implementation—not just whether a policy is implemented, but how it is implemented—often means the difference between achieving the desired outcomes or not. Police leaders can respond to tracking evidence that shows poor quality of implementation by either improving officer compliance with policy, improving the policy itself, or both. We report a case study of the tracking of implementation quality in a randomized controlled trial of a police policy for contacting victims, in which the first author was a participant-observer. We show that when the results of tracking were fed back to officers to improve compliance, and to managers, who then redesigned policy and training in repeated iterations, the quality of implementation and victim satisfaction improved substantially. This evidence-based, training-tracking-feedback strategy of implementation can be applied more generally to improve the quality of police services and outcomes.

Keywords

tracking implementation victims diversion randomized experiments evidence-based policing

Introduction

A growing body of evidence identifies a range of police and other criminal justice practices that are effective at reducing crime. Yet even practices with strong evidence may fail if not implemented in effective ways (Sherman, 2013). The question of how best to produce high-quality decision making by police officers is central to the development of evidence-based policing.

This question is the subject of an ongoing debate about whether police decision making should be based on high control (tightly prescribed policies, with little room for officer discretion) or high discretion (entrusting professional decision-making skills of officers; see, e.g., Bronitt & Stenning, 2011; Mastrofski, 2004; National Research Council, 2004). While high-control police policies may create consistent outcomes, they do not allow for adjustment to individual circumstances. While high-discretion approaches allow officers to adjust to different situations, they may produce inconsistent, biased decisions with poor outcomes. This dilemma is best resolved by an alternative approach.

A third way to improve quality in police decisions captures the benefits and minimizes the risks of both discretion and control: bounded decision making through decision support. This “third way in officer decision-making” (Slothower, 2014a, p. 345; see also Goldstein, 1979) is informed by the way professional discretion in the field of medicine is set up, bounded, and supported (see, e.g., Grol & Grimshaw, 2003). In medicine, professional discretion is strengthened by targeting discretion where it is useful and important, and by removing it where it is not necessary. Decision-support information technology (IT) tools—designed to deliver information about what is likely to be most effective at any time, and provide feedback on various past decisions to improve over time—can support medical practitioners to make better professional decisions. This enhances their professional discretion, rather than diminishing it (Kawamoto, Houliahan, Balas, & Lobach, 2005). Discretion tracking and feedback systems are one discretion-support tool that have a key role to play in bounding and strengthening officer discretion.

Police measurement has traditionally focused on “how many.” How many arrests? How much patrol time in hot spots of crime (Sherman et al., 2014)? How many convictions? Detection rates, for example, drive police performance across much of the United States and United Kingdom (Moore, 2002). If police leaders want more, they ask for more, and often they get more. But for innovative policies with important qualitative dimensions, numbers alone are not going to tell police managers and executives all the information they need to know. These numbers may tell police leaders how much of something police officers are doing, but not how well they are doing it.

In the last decade, the once popular “tick-box” culture of management in policing has widely fallen out of favor, with widespread recognition that there are perverse outcomes to simple metrics, targets, and quotas (Neyroud, 2008). Some policing areas have moved toward removing measures and relying on values-based officer discretion, but substantial drawbacks have been found from the subjective (and widely inconsistent) judgment of individual officers. Instead of focusing on counts, or stopping measurement altogether, more nuanced approaches to monitoring the implementation of various policies are required.

This article uses a case study of tracking the quality of decision making in a randomized controlled trial of deferred prosecution program for first offenders. It explores the role of police implementation tracking systems as an approach to supporting and bounding the decision making of officers, strengthening professional discretion, and improving the quality of implementation. As Sherman (2013, p. 60) observes, “The greatest value of measuring police performance comes from leaders taking immediate corrective action.” Corrective action may be necessary because of issues with the policy or its implementation, or because the crime problems and contexts have changed. Tracking allows managers who set police policies to have an accurate sense of the “daily delivery and effects of those policies” (Sherman, 2013, p. 7). It gives managers a close eye on whether policies are being implemented as planned, and whether the intended impacts are observable on a range of outcomes—crime, cost, complaints, public opinion, and so on. It also allows managers to detect problems such as lack of implementation, lack of effect, or unintended negative effects. If tracking uncovers a lack of compliance by officers producing any of these problems, “the leader can take stronger action to ensure that the policy is carried out” (Sherman, 2013, p. 60).

Effective tracking tools thus allow managers to take two important types of “strong action” in reaction to negative results identified through tracking: (1) adjust officers’ implementation/adherence to the policy and (2) adjust the policy, including the hard-to-pin-down but important details about how the policy specifies it is to be implemented. Using prior research as well as the case study of a randomized controlled trial in Birmingham, United Kingdom, this article illustrates how the best approach may be to adjust both the policy and the way officers are adhering to the policy. The adjustment described in the case study was to provide training supported by a computerized implementation tracking system, enabling feedback to officers.

Part I of this article analyzes previous research on the use of tracking to react to negative results to improve quality of implementation. Part II presents a case study of police interaction with victims a randomized controlled trial of deferred prosecution in Birmingham, United Kingdom. The case study compares three sequential phases of the study, namely officers making decisions based on their own discretion, officers making decisions after training only, and officers making decisions under an IT-supported, “train, track, and feedback” system. Part III discusses the case study to highlight more general implications of a model of implementation that combines training, tracking, and feedback (TTF).

Tracking: A Tool for Improving Implementation

Reaction 1 to Negative Results: Train, Track, and Feedback

This section demonstrates that implementation quality is a problem, that tracking can measure that problem, and that TTF should be a more effective method of improving implementation quality than merely a “train-and-hope” approach that fails to either track or feedback.

Quality of implementation has been shown to make or break a wide range of even the most proven, effective crime policies. Problem-oriented policing, a policing strategy that measures and strategizes around “problems” that are sources of crime rather than just responding to individual calls or crime incidents, is found to be effective across studies, but not when these approaches are implemented with low fidelity to the intended policy (Weisburd, Eck, Hinkle, & Telep, 2008). It is not just whether or not police focus patrol on crime hot spots, it’s how they patrol crime hot spots—how precisely and accurately they define hot spots, how long officers stay there, how often they go, and possibly what they do when they are there—that determines whether or not they prevent crime and impact perceptions of police legitimacy (Weisburd et al., 2008). Similarly, while police stop and search may reduce weapons crime (Sherman & Rogan, 1995), there is also evidence that how respectfully they treat the person they search determines that person’s compliance and perception of police legitimacy (e.g., Bradford et al., 2013; Hasisi & Weisburd, 2011; Mazerolle, Bennett, Davis, Sargeant, & Manning, 2013; Tyler, 2006a, 2006b; Tyler & Huo, 2002). Body-worn video has been shown to reduce complaints against the police and police use of force (Ariel, Farrar, & Sutherland, 2014), but it is not enough to put body-worn video cameras on officers—they don’t work if they are turned off, and systems have to be in place to use the information they record (Drover & Ariel, 2015). Other crime policies that have resulted in approximately 10–50% reductions in crime in high-quality studies when implemented well have produced little, no, or even backfiring effects when implemented poorly or with the wrong populations (Andrews & Dowden, 2005; Barnoski, 2002, 2004; Dodge, 2001; Goggin & Gendreau, 2006; Karoly et al., 1998; Landenberger & Lipsey, 2005; Latimer, 2001; Lipsey, 1999, 2009; Lipsey & Wilson, 1998; Lösel & Schmucker, 2005; Lowenkamp et al., 2006; Nesovic, 2003; Real & Poole, 2005; Schoenwald, 2008; Strang, Sherman, Mayo-Wilson, Woods, & Ariel, 2013; Welsh, Sullivan, & Olds, 2010).¹ In a systematic review of all juvenile justice programs, the quality of implementation was the single most important driver of effectiveness in reducing reoffending, aside from seriousness of the offense (Lipsey, 2009).

Despite the importance of high-quality implementation, much evidence suggests that most crime policy implementation is in fact low quality. Wide scale assessments of correctional² and violence prevention³ programs have found that only about a quarter to half of programs are implemented with reasonable fidelity. Within policing, there has been great disappointment among police leaders who order policies that have been found to work in studies, only to find few returns on their investments, or new unintended consequences. For example, in Trinidad, police leadership directed the police force to implement hot spots policing in a high-quality manner⁴ but found that while the Trinidad hot spots experiment produced dramatic results while research staff kept pressure on police personnel (Sherman & Strang, 2014), as soon as the researchers left, practice and results went back to normal. Implementation quality only increased again when research staff returned and supported the persistent leadership of the Commissioner. Based in part on the experimental evidence showing the effect of restorative justice conferences on reduced reoffending and increased victim satisfaction (Strang et al., 2013), there was widespread training of police officers and police-affiliated partner organizations across the United Kingdom costing millions of pounds. However, it appears that this training was all but ignored, with very few cases nationally going to conferences (Meadows et al., 2014; Sherman & Strang, 2014). Some qualitative evidence suggests that whether or not managers engaged in certain management strategies may determine levels of use of restorative justice conferencing among officers (Slothower & Joyce, 2014). In terms of individual officer implementation, despite widespread training only some officers are effective in getting participants to agree to conferences and in facilitating effective conferences (Sherman, Strang, & Woods, 2003; Woods, 2009). Although a wide variety of other practices have grown up in the United Kingdom that draw on the evidence supporting restorative justice conferences, Sherman and Strang (2014) point out that the existing restorative justice research should not be construed to provide evidence for approaches that are substantially different from those that were tested.⁵

One key challenge in implementation of crime control policies is that often, the make-or-break factors seem to be related “soft skills,” which are more difficult to track and manage than traditional metrics. More evidence is emerging that effectiveness is sometimes related to certain characteristics about the way in which officers and other practitioners talk to people. Even when practitioners deliver program or policy tasks, if they fail to exhibit certain characteristics such as sociability, or approaching conversations from some frameworks and not others, this can reduce or eliminate the effectiveness of the policy (Sherman et al., 2003; St. Pierre, Osgood, Mincemoyer, Kaltreider, & Kauh, 2005; Tyler & Huo, 2002; Woods, 2009). However, ensuring staff exhibit these skills on the job is not an easy task.

Although not easy, the task of implementing nuanced policy or program initiatives with high implementation quality at a large scale has been successfully accomplished in the past (Fagan, Hanson, Hawkins, & Arthur, 2008). The key to successful implementation appears to be in the ongoing management processes that support the policy or program. In particular, tracking officer compliance with managerial policy is a key means of assessing, managing, and correcting implementation, as Sherman emphasizes:

All too often, police leaders are flying blind about whether policies or operations are being implemented. Tracking evidence provides independent audits on a timely basis about whether the plans are being delivered. If they are not, then commanders can be transferred and tighter control can be imposed. If they are, then rewards can be given and medals can be pinned. (Sherman, 2013, p. 60)

Research from across the public sector finds that training alone is not enough to change practitioner behavior and achieve high-quality implementation. A systematic review of experimental studies of public sector implementation evaluation reported that, “Guidelines, policies, and/or educational information alone or practitioner training alone are not effective…as has been shown in a variety of settings, the train-and-hope (Stokes & Baer, 1977) approach to implementation does not work” (Fixsen et al., 2005, pp. 91–92). Instead, training plus on-the-job feedback (either through live on-the-job coaching or other approaches to tracking and feedback described subsequently) appears to be the key to effective improvement of desired behaviors and professional skills. This evidence suggests that the TTF approach is a more promising alternative to the “train-and-hope” approach. As pointed out in Fixsen, Naoon, Blase, Friedman, and Wallace’s (2005) systematic review of implementation science, the effectiveness of TTF has been found across a range of experimental studies, contexts, and systematic reviews (among teachers, Joyce & Showers, 2002; medical practitioners: Davis, 1995; behavior-change therapists: Ager & O’May, 2001, Maloney et al., 1975; business, where an estimated 10% of training transfers to on-the-job practice: Rogers, 2002). In contrast, as one researcher points out, “there are no known…training materials that will achieve their goals in the absence of…managerial support” (McGuire, 2001, p. 34). A meta-analysis of studies of the effect of teacher training (Joyce & Showers, 2002) found that in no format did training alone (including training focused on theory and discussion, demonstrations in training, and practice and feedback exercises in training) produce substantial use of skills in the classroom (although some did produce knowledge and the ability to demonstrate the skills). However, when in-classroom observation and feedback was added to follow up training, later use in classroom reversed, such that almost all participants displayed the skills.

Drawing on policing and other fields (sports, medicine, business management, teaching, corrections program implementation, therapist quality control, etc.), a range of mechanisms have been successful in tracking the how (i.e., tracking how officers implement policies), ranging from IT tools to structured verbal debriefs to interviews with citizens involved in encounters to video playback. Police officers can even track themselves, with new technologies such as body-worn video. In the Rialto Police body-worn video study (Ariel et al., 2014), officers regularly downloaded their own video records to review their interactions with citizens. Just like pro athletes spend countless hours alongside their coaches reviewing footage of themselves in action to improve their technique and reach the top of their game, these officers had the opportunity to reflect on what worked and did not work, and could give themselves “auto-feedback” on how to improve. In hot spots policing, tracking can take the form of Global Positioning System monitoring of vehicles to see how much time officers spend in hot spots, body-worn video to review the procedural fairness and respect officers exhibited, as well as follow-up surveys with those who police encounter (Sherman et al., 2014).

Reaction 2 to Negative Results: Improve the Policy

This section demonstrates that sometimes the TTF approach can reverse direction, feeding back to managers that the policy needs some adjustment in order to better accomplish the goal. It suggests that tracking can inform managers as well as their supervisees that there may be better tactics or systems for supporting the frontline work of the organization.

Revisiting what was asked of officers in the first place is an alternative to improving implementation of the current policy. Under this approach, police managers or policy architects can respond to negative results by using tracking to learn how to best improve policy. Tracking can identify areas where a policy is not quite right and needs continued adjustment. Rather than focusing on compliance with a policy that is had to implement, managers can make the policy itself easier to comply with. Tracking allows leaders to rethink the policy itself, tweaking and adjusting based on the feedback tracking offers.

The potential for this approach may depend on what measures are tracked. Sherman points out that it is relatively standard for police agencies to track stops of citizens, but the perceived procedural fairness of those stops is usually not tracked:

For many police agencies, stop-and-question is an embedded part of patrol practice in high-crime areas. Its use in crime hot spots in New York and London is highly visible and highly controversial. Few debates distinguish between the questions of how stops should be done and whether they should be done at all (30)…. Tracking may raise the question of police legitimacy. Are complaints up since stops have increased? Are public surveys needed? Should people who have been stopped be interviewed to assess their reactions?…. Tracking can often raise questions that would otherwise never be asked. (Sherman, 2013, p. 60)

Tracking public perceptions of the fairness of their treatment in stop and searches could lead to policy adjustment, including alterations to training and management of police handling of these interventions.

One officer involved in the case study reported subsequently described the tracking and adjustment approach to police policy management as follows:

When you leave policy implementation to practitioner discretion, you effectively say: “You know best—over to you.” With training, or training plus [compliance enforcement], the sentiment is more, “We know best, so we show you, and then over to you.” With [the tracking and policy adjustment] approach, the fundamental difference is two-fold: Firstly, no one knows best: rather…we have an idea about where we want to go, and some ideas about what might work, but we are expecting to constantly refine and adapt through an iterative process. Furthermore, by constantly assessing for progress, we never actually say “over to you.”

That presents an interesting challenge to how we do leadership/management within the police, an inherently hierarchical organization: we do have an expectation that once a decision is made, people will follow it. When it becomes apparent that some/many/all are not following that decision, ever-greater attention is focused on achieving compliance. However, it is rare that the original policy decision should be reviewed and adapted. This focus on compliance continues until managers change focus (or are replaced), at which point a new cycle begins with a new objective and new efforts at achieving compliance…[The tracking and policy adjustment] style of leadership is based on acknowledging that the leader does not have an answer, but is instead seeking an answer. Finding that answer is an iterative, interactive process where the leader and practitioner constantly share ownership of the problem. (Inspector Thomas Joyce, personal communication, December 8, 2014)

This approach to policy development and implementation acknowledges that when negative results are detected, both the policy itself (set by leaders) and the implementation quality (exhibited by field officers) are potential candidates for adjustment, not just one or the other. Since leaders tend to blame poor implementation, and field practitioners tend to blame poor policy, tracking, and adjustment can help to parse out this issue and move both toward continued exploration and improvement.

In the case study mentioned subsequently, a number of findings related to tracking can be seen throughout the phases of implementing the deferred prosecution innovation:

Different situations call for different types/intensities of tracking, ranging from low cost, and simple tracking measures to expensive and rigorous ones. Often less-intensive approaches can identify problems. It is only once a policy passes lower intensity tracking and appears promising that more rigorous approaches are needed to look for unexplored issues and ensure positive outcomes are being achieved.

Honing an approach to be effective is an iterative process, involving repeated patterns of targeting, testing, training, tracking, tweaking, feedback, and adjustment based on tracking results. This may be cyclical, but more likely it is iterative in a more complex pattern—the pattern of this study could be better described as:

Targeting → Tracking → Major Tweaking → Tracking → Major Tweaking → Tracking → Minor Tweaking → Testing…with more tweaking and tracking in the future as the approach continues to be scaled up and adapted to fit the realties of implementation.

How research is applied to policy and practice matters—even good faith, well-researched efforts designed to be “evidence based,” or to draw on existing literature to apply evidence about “what works” may not be applied effectively the first time around. Rather than scrapping the whole concept or body of evidence, it can be valuable to dig deeper to find better ways to apply research, and keep tracking to hone in on a desired outcome.

Ruling out low-quality implementation leaves only policy-level issues. If practitioners display high-quality implementation but tracking uncovers negative impacts, this indicates that the policy, or the types of situations in which the policy should be applied, may need to be revisited.

Turning Point Victims Study: Tracking Across Three Phases

Turning Point is an innovation in police management of offenders that was introduced across the entire City of Birmingham, United Kingdom, in a randomized controlled trial. The innovation is a negotiated and conditional out-of-court alternative to prosecution, applying the life-course criminology of “turning points” (Sampson & Laub, 1993) to create a strategy for offender desistance policing (Sherman, 2011; Sherman & Neyroud, 2012). Offenders in relatively low-level offenses were screened for approval to prosecute, then randomly assigned either to be charged and sent to court or diverted into a Turning Point disposal. In Turning Point, offenders signed a contract agreeing to complete a set of conditions within 4 months in order to avoid court. These conditions were individualized for each offender depending on the factors contributing to the offending behavior, the needs of any victims, and the impact of the crime. Conditions could be rehabilitative, reparative, restorative, and/or punitive. If offenders completed their conditions, they were neither charged nor prosecuted and received no criminal record. If they failed to comply with their contract, or reoffended, they were to be prosecuted for the initial offense. The reoffending differences between the treatment and control groups have not yet been measured; however, various other Turning Point substudies as well as details of the implementation of the study are reported on elsewhere (Hobday, 2014; Neyroud & Slothower, 2013, 2012; Slothower 2014a, 2014b).

The case study presented here focuses on the policy for managing victims in the development of the experiment. This “substudy” of victim relations in Turning Point explores the policy for police communicating Turning Point decisions to victims. The case study illustrates how tracking led to the discovery of an unintended consequence of Turning Point, that is, dissatisfied victims. Then tracking facilitated the development of new approaches, further assessment of whether the new approaches were effective, and allowed feedback to aid officers in improving their communication with victims.

The process of developing the victim strategy took place in three main phases:

Phase 1: Officer discretion only

Phase 2: Training only

Phase 3: TTF

Phase 1: Officer Discretion Only

In the winter of 2011, the Turning Point study began its initial pilot phase. The explanation of Turning Point deferrals that police officers gave to victims at that time did not receive special attention. Turning Point explanations were handled in the same way any other disposal was handled. The officer in charge of investigating the case, generally a detective, was a different officer from the “offender manager” who negotiated with the offender to set the Turning Point conditions. The detective, rather than the offender manager, was responsible for following up with victims and notifying them of the initial outcome of their case within 48 hr of the decision.

As with all out-of-court disposals, victim’s contact was governed by the U.K. Victims Code. This code entitles victims to information about what to expect from the criminal justice system, as well as to be informed about the investigation, any arrests, and the decision to charge or not, or to give an out-of-court disposal. Like all out-of-court disposals, victims were to be consulted regarding the decision to put the case in Turning Point; however, their consent was not required.

The question about the impact of Turning Point on victims arose both from the research team as well as from police officers and management involved in the study. Little research with representative samples has ever explored victims’ perceptions of out-of-court disposals, and no experimental research that the research team was aware of had been done comparing victim perceptions of this type of diversion. Further, Turning Point was now diverting more serious offense categories than were usually diverted from court. Without prior experience with those offenses, virtually nothing was known about how victims would react to diversion rather than prosecution.

The initial experience in this project from officers who had spoken to victims was that some victims were not satisfied with diversion. However, little was known about this, as many of the officers who contacted these victims were spread across the city and were not involved in Turning Point in any other way.

The Cambridge University project team, with funding from the Monument Trust, established tracking of victims’ views as a high priority for the project. As the field research manager, the first author contacted 20 of the 30 victims with cases in the initial Turning Point pilot period to interview them over the phone or in person about their perceptions of the handling of their cases. As described in a document at the time, “The interview process was semi-structured and designed more to support the continued development of practice in the experiment rather than a formally constructed qualitative research process” (Neyroud & Slothower, 2012, p. 29).

It quickly became clear that there were unintended consequences of the offender-focused Turning Point study in terms of the impact on victims (Neyroud & Slothower, 2012). While some victims were happy with their cases being handled in Turning Point, the majority of victims were unhappy, and often quite angry (approximately two thirds, although the contact and interview protocol was not as rigorous as in later iterations of tracking).

The interviews uncovered a clear source of victims’ dissatisfaction that is, victims widely felt that Turning Point would not be effective in “stopping the offender from doing it again.” Dissatisfied victims generally expressed substantial anger at the police for that reason. Virtually all unhappy victims stated among their main concerns that they did not think that Turning Point would be taken seriously by offenders and that the offender would not understand the impact of their actions, and therefore, Turning Point would not stop them from reoffending. Satisfied victims, in contrast, tended to focus on the possibility of rehabilitating the offender, or stopping the problem from happening again. Overall, there was a strong perception among unhappy victims that the outcome was not legitimate, and court would be preferable. As one victim stated:

“Just give them a list of good punish about it, because if you don’t give them good punish about it, they think they get away with it. If they had been given a bit of bad record kind of, then they will understand, they will have their own medicine to think ‘ah, I attacked someone, how did that person feel?’”

Some victims also mentioned that diversion into Turning Point would make the victims themselves less willing to contact or cooperate with the police in the future (“Next time I will hit him back!”).

In terms of policy implementation, it also quickly became clear in the interviews that victims were receiving a wide range of messages from police officers, and these explanations were often not strategic. Based on the statements by victims, it appeared that detectives were often explaining Turning Point in ways that may have increased victims’ anger. The officers contacting victims were not generally trained or briefed. The officer in charge of the case could be one of thousands of potential officers. The officers responsible for diverting offenders into the study (who were well briefed on Turning Point) were also responsible for explaining Turning Point to the detective who would then explain the project to the victim. Still, the detectives appeared to be unequipped to fulfill the role in terms of the message to victims.

Some detectives told the victims that they were upset the case had been diverted, as the victims recounted:

Officer [name] tried to override the decision [to divert the case to Turning Point]. He was wonderful, and he said he was so sorry, the police had let me down…

Other officers left the victims with a lack of clarity about what was happening with their cases:

It was not really explained to me, they just phoned and told me that he’s gone to some community course or something, but nobody came to the house or explained properly…

It’s a new law thing they’re doing, that’s all I really know…

In other cases, officers tried to persuade victims that Turning Point was a good idea based on reasons that may have been compelling to the officers, but not to victims, such as saving money, as in this case:

[The Turning Point officer] tried saying to me yeah you know what they’re going to put him on a [expletive] course yeah, they’re going to put him on a little course to save the taxpayers money. That was the exact words. You think I want the taxpayers’ money to be saved? Think about it. I’m paying these police officers, I’m paying the tax yeah, just so they could say to me oh we’re trying to save your money. You’re not saving my money. My money’s gone out of my pocket already.

Other communication concerns were reported by victims. Three victims were not told about the project, and were upset to find out at a later date. Some victims were upset that they did not receive follow-up information about the progress of the offender after the initial contact explaining Turning Point. In two cases, victims who reported potential trauma symptoms were told they would receive follow-up support from the organization Victim Support, and they were upset they had not been enrolled in the services.

As this tracking progressed, a few adjustments were made. The offender manager officers who actually decided the Turning Point conditions (and were well briefed) were told to contact the victim themselves, in two sequential ways. (1) The offender managers’ approach to explaining the Turning Point Project was initially left to the professional judgment of the officers. (2) Toward the end of the pilot period, offender managers received a brief (approximately 1.5 hour) training session from the project team involving the research on victim satisfaction, and using examples from actual cases. Officers were also given a script to read to victims. No improvement was observed in victim interviews and the script made officers feel uncomfortable and unconvincing. In addition, victims still tended to receive the initial contact from the officers in charge, rather than the Turning Point officer. By the time the Turning Point officer spoke to victims, they were already upset by what they had been told.

Phase 2: Training

Following the negative results of the victim surveys in the first pilot period, the project board and research team agreed that a new policy was needed for victim communication. The academic field research manager (first author) and the police sergeant, who was responsible for the police force’s restorative justice training, agreed to review the research on victim satisfaction with criminal justice; the goal was to put an evidence-based approach into practice. The team recognized that this would have wider implications than just for Turning Point. Little research had explored victim perceptions of out-of-court disposals in the past, and both police and academic research partners suspected that mixed messaging and dissatisfied victims might be a problem for many out-of-court disposals issued by the police.

To generate a policy for the duration of the study in regard to victims, the research manager (first author) and restorative justice training sergeant met with two leading researchers on restorative justice and victim perceptions of criminal justice outcomes. They discussed a range of possibilities before agreeing to a plan. Many of the options were not feasible in a police setting—they required too much training of officers, were too unwieldy, or they could not be fit into the tight time frame of the experiment (wherein an offender was randomly assigned on the spot, and a Turning Point plan was set within 48 hr for the offender).

Developing the Phase 2 approach to exploring victim perceptions of legitimacy of Turning Point was an exercise in throwing the book of evidence-based practices at the problem. The team focused on three widely supported research findings and attempted to craft an approach drawing on these principles:

Process factors (whether victims feel the police respect them, care about them, and are treating them fairly) are more important than outcomes in achieving victim satisfaction and perceptions of legitimacy of police and criminal justice responses to their victimization (for a full account of this theoretical underpinning, see Neyroud & Slothower, 2013; Slothower, 2014; see also Mazerolle et al., 2013; Shapland, Sorsby, & Robinson, 2011; Strang, 2002; Tyler, 1990; Tyler & Huo, 2002).

Victims are highly satisfied by restorative justice conferences, more so than traditional criminal justice processes (Shapland et al., 2011; Strang, 2002); and

Victims widely desire a voice in the process of their case handling (Strang, 2002; Shapland et al., 1985; Shapland et al., 2011).

The main assumption driving the design of a new policy was that Turning Point, as an out-of-court disposal designed to reduce offender recidivism, was inherently not victim oriented. The planning team concluded that Turning Point could not satisfy victims on its own merits because it was focused on needs of offenders, not needs of victims. To satisfy victims, the team concluded, the project would have to use an approach that would completely change victim’s experience of Turning Point.

The new policy was designed by (1) gaining victim input into the creation of Turning Point conditions, (2) offering to hold face-to-face restorative justice conferences for every victim who wanted one (with offender-absent conferences if the offender did not agree), and (3) use of a script to control officer messaging. The process would be as follows: Victims would be contacted prior to the first Turning Point meeting by the offender manager who was going to set the offender’s conditions. Victims would be asked if there were any conditions they wanted included in the plan. Offender managers, who would all be trained in restorative justice, would begin the process of offering and preparing victims for restorative justice. They would explain that if the offender did not agree to take part in the conference, an offender-absent conference would take place. They would also explain Turning Point to victims using a script to ensure the messaging was consistent, accurate, and focused on the reason the police were testing Turning Point (to prevent future offending). Offender managers would then hold the Turning Point meeting with the offender, setting conditions that included engaging in a restorative justice conference if requested and complying with the outcome agreement created by the victim and offender at the conference. If the offender agreed, the offender manager would then run the conference, and if not, the offender manager would run a victim-absent conference.

However, the plan had to take into account a big obstacle. While a range of leaders on the police force were pushing for offender managers to regularly facilitate restorative justice conferences with their offenders, these offender managers were not yet trained and practiced in restorative justice. The representative of the partner agency with experience in training restorative justice facilitators was present at the planning meeting, and offered to deploy staff to train the offender managers immediately to enable the plan to work.

Until this could be accomplished, a temporary plan had to be put in place. In this plan, both detectives and offender managers were told not to contact the victim. Instead, existing restorative justice facilitators were to be immediately notified when a case with a victim was entered into the study. These facilitators were supposed to contact the victim prior to the initial meeting between the offender and the offender manager when the Turning Point conditions were set. The facilitator would then be present at the initial meeting with the offender manager. In effect, the same approach was to be implemented as described in the overall Phase 2 plan, except that the facilitator would take the role of the offender manager in regard to contacts with the victim and implementing restorative justice, until the offender managers could be trained to take over the role.

Formal tracking of the Phase 2 policy took place in the form of victim interviews after Phase 3 was already initiated. But at the time of Phase 2 implementation, formal tracking was unnecessary. It soon became apparent through informal tracking (simply asking officers what was taking place) that the approach conceived by the working group was not logistically viable, and quickly fell apart. Almost every element of the plan was seen as undeliverable.

The main plan—to have offender managers quickly trained in restorative justice and situated to prepare and offer restorative justice conferences to victims within 48 hr after the diversion decision was made—was unsuccessful. There was neither implementation of the necessary training nor the capacity to contact victims in the time frame required. Due to changing circumstances, the partner agency was no longer able to provide the restorative justice training in the time frame and scale discussed. The restorative justice training that was eventually received by the offender managers was generally agreed to be of lower quality than intended, and was not implemented at all by most of the officers. Officers declined to use the script, as they stated it felt too stiff and awkward.

The interim plan was also unsuccessful. It proved logistically impossible to get the case allocated to a restorative justice facilitator and for the facilitator to pick up the case, speak to the victim, and then to attend the meeting with the offender manager in the time frame necessary. Instead of the policy, the practice implemented was that offender managers began to contact the victims themselves, and relied on their experience of Turning Point and brief training they had received in talking to victims about Turning Point several months prior.

Systematic tracking that took place later confirmed substantial dissatisfaction among victims dealt with by offender managers rather than detectives. The research team undertook a survey, with high attention to the quality of the sampling and survey methodology. Victims reported being 54% satisfied with the handling of their case in Turning Point in this sample, and dissatisfied victims reported being unhappy and angry for reasons similar to those offered when detectives had contacted victims.

Phase 3—TTF

This phase began when the policy development team examined the assumption that Turning Point was inherently not a victim-centered approach. The team knew that the literature showed victims often stated that the most important goal was stopping the offender from committing a crime again. They also knew that how victims were spoken to could matter more than what was said in shaping their perceptions of police legitimacy. Therefore, it would seem to follow that as long as victims are made to feel that the police respect them, care about them, and are doing something in the victim’s interest by trying to stop the offender from doing it again; victims are likely to be satisfied. There was no strong research suggesting that a case being handled out of court instead of in court should change that, and in fact the scant existing research seemed to suggest the opposite (see Slothower, 2014b, for review). Based on previous research, victims should be satisfied if the above-mentioned conditions are met regardless of whether the case is diverted or not, if the outcome is communicated in an effective way.

Based on this new perspective on the research, the team developed a new policy. A team of specialist officers was trained to have the conversation with victims. They were given the specific structure of the conversation (rather than a script), with freedom to adapt as necessary. They were also required to provide an account of their conversation in an online survey tool—what they tried, and what they thought worked well or did not work well—for monitoring by the Sergeant leading the team. They received feedback from the Sergeant based on tracking when needed, particularly during the early phases as they began this approach.

Before the official study began, officers spent about a month trying out cases, providing detailed accounts of their attempts, and then receiving feedback from the overseeing Sergeant. The guidelines were tweaked based on this small pilot phase to adapt to the necessary feedback. An area that received special attention was the conversations where victims reported negative feelings about the interaction, but the practitioners reported high compliance with the recommended policy. These situations suggested the policy protocol itself may not be right in some ways and provided fodder for small adjustments to the protocol.

This was an example of how negative results combined with high-quality implementation and compliance combined to suggest a need to tweak policy. This pilot phase also gave researchers the confidence that the approach seemed logistically feasible, and seemed to be going over well with victims, and thus was worth the resources required for more rigorous tracking and testing.

The tracking of this phase was twofold: first, with regular tracking by the officer managing the implementation; and second, with a survey of victims in this period well after the communication of their case decision was complete. This second type of tracking also took the role of testing (Sherman, 2013), as a second sample of victims—those with cases randomly assigned to court—was also surveyed.

Phase 3 tracking uncovered much improvement in victim satisfaction, with 73% of victims saying they were satisfied with the handling of their cases. The results are further reported elsewhere (Slothower, 2014b). The important point for this article is that this tracking uncovered substantial improvement in satisfaction. It also suggested areas that seemed to be working well and were worthy of continuation, as well as what needed to be improved further in order to solidify and scale up the approach in the future.

Discussion

The case study mentioned previously highlights the two different uses of tracking: improving compliance with policy and improving policy itself. Both these uses were implied in the cyclical process of evidence-based policing first proposed almost two decades ago (Sherman, 1998), but never before described in ethnographic detail. In Birmingham’s Turning Point, both uses of tracking played a critical role. Tracking told managers that the policy they had crafted was not getting the kind of delivery they intended. But tracking also told them they would be better off reshaping the policy rather than micromanaging the people in the wrong role for the task. Tracking was key link in providing continuing professional development to officers through TTF.

In Phase 1, tracking took the form of a semistructured interview of victims, and found an unintended consequence of Turning Point (substantial dissatisfaction among victims), as well as clues as to how to improve policy and implementation.

In Phase 2, tracking first took the form of an informal feedback request from officers leading to an informal inventory of cases, finding little and weak implementation of policy. A later retrospective tracking approach (survey of victims) confirmed that indeed there was still substantial dissatisfaction among victims in Phase 2. Rather than trying to force improved compliance, it quickly became clear that the policy was not well structured and needed to be redeveloped based on the feedback.

In Phase 3, tracking took the form of a rigorous, systematic survey of victims, and found mostly satisfied victims. It also found areas for continued improvement, setting the foundation for future iterations of the testing tweaking feedback cycle.

There were clear differences between the Turning Point phases. Neither officer discretion, nor training without tracking, nor tracking without feedback, were ever sufficient to secure high-quality results. But where an individual-level process of TTF was provided, the results improved. This favorable qualitative evidence on TTF also seems to be consistent with the findings from the wider literature on implementation.

There were a number of different types of tracking that took place, and different situations called for different types of tracking. Tracking began with less formal approaches, which were sufficient to uncover problems. Tracking became more rigorous as policy improved. The most resource-intensive tracking was used when an approach was developed that seemed to be viable based on lower cost methods of tracking. The first set of interviews was exploratory and was not systematic. The sampling frame was rigorous, with all victims in Turning Point cases in the pilot period. But there was not careful attention to ensuring that the interviews were conducted in the same order with the same tone of voice, or that there was an equal and substantial effort to reach every victim in the sample. What the informal surveys did well was uncovering problems and providing a basis for designing policy improvements.

The second phase was tracked simply by asking for feedback from officers and then through an informal inventory of cases. More rigorous tracking was not necessary because informal tracking made the problems apparent. The third phase was briefly tracked informally as officers tested it out for a month prior to the study start date and then launched into rigorous tracking through a systematic survey when the approach seemed to be viable through more informal tracking. In instances where officer implementation quality appeared to be high, but victim survey respondents were dissatisfied, the policy was revisited and improved accordingly.

The research team is currently exploring how to scale up the approach to tracking and implementation developed in this study. A simple online tool to enable basic guidance, qualitative tracking, and feedback is under development and piloting, drawing on what seemed to be effective in Turning Point, and is reported on elsewhere (see Neyroud & Slothower, 2014, for the basic principles, and the specifics are forthcoming). In the future, new technologies such as body-worn video cameras will continue to expand the opportunities for continued tracking and feedback to officers.

The pattern of neither discretion nor mere training being sufficient also reflects a pattern found across the three areas of officer decision making and policy implementation involved in Turning Point. These tracked areas were the decision by police Custody Sergeants about whether or not to divert a case into the study (Hobday, 2014), the setting of conditions for the individualized Turning Point contracts by police Offender Managers (Slothower, 2014a), and the communication with the victims about what was going to happen with their cases (Slothower, 2014b).

In each of these three areas, careful tracking uncovered implementation problems. In all three cases, these problems were (after other attempts at solutions) eventually addressed in part by using TTF. The case study suggests that in more complex officer decision making, IT-based decision-support tools were helpful in improving implementation, in combination with TTF.

In the initial phases, tracking of custody officers’ decisions of whether or not to include a case in the study uncovered a relatively low but still untenable rate of inappropriate decisions. This discovery was made possible by a decision-tracking IT system for online screening, intake, and random assignment, operated by the Cambridge Gateway at the Jerry Lee Centre of Experimental Criminology at the University of Cambridge (Ariel, Sherman, & Vila, 2012). The evidence from this analysis was used to substantially upgrade the online tool itself. The tool was then scaled up to guide decision making and allow for tracking and feedback to officers.

A similar process was also used with the negotiations with offenders selected for the program. In the initial phases, the first author’s tracking of offender managers setting of conditions uncovered a high rate of vague and unenforceable conditions. To address this, an online decision-support tool was created to support officers’ selection of conditions, and to allow for up-to-date tracking and feedback to officers.

In the initial phases, tracking uncovered substantial victim dissatisfaction. This was addressed by structured guidance in communication with victims, as well as an IT tracking tool. The complete randomized sample for the Turning Point Study was collected in Phase 3, ensuring consistency of process in the final sample.

Common themes emerged from all three of these substudies:

The tracking tools tracked reasons for decisions, not numbers of decisions;

The tools attempted to capture thought processes (reasoning) not just reason;

The internal (police actions) tracking tools were combined with external tracking tools (victims or offender outcomes) to triangulate quality of implementation;

Tracking tools allowed tracking to be aggregated and disaggregated to look at individuals, teams, and the whole police force;

Tracking tools allowed feedback to be given to individuals or groups, to learn from their own and others’ errors.

Conclusion

The Turning Point victim case study illustrates the key role of tracking in developing and implementing impactful policy. As has emerged in the larger implementation literature, discretion alone produced inconsistent and often problematic decisions. Training alone (the “train-and-hope” approach) or even training plus materials (such as a paper script) were not enough to change behavior on a large scale. Instead, a combination of TTF was required to achieve high-quality implementation. This conclusion is consistent with that from other Turning Point substudies, with additional decision-support technology geared toward bounding decision making playing a role as well. More focus on developing and testing innovative TTF could support continuing professional development. Where appropriate, decision-support IT, such as the tool being developed based on these Turning Point substudies, could become the subject of future randomized trials. In general, evidence-based policing needs just as much evidence on its own implementation as it requires about policing itself.

Footnotes

Acknowledgment

The authors gratefully acknowledge the support of the Monument Trust of the Sainsbury Foundations and Mark Woodruff for the Trust’s financial support of the research and authorship of this article. They also acknowledge the strong intellectual and financial support of the West Midlands Police, its Chief Constable Chris Sims, Assistant Chief Constable Marcus Beale, and Chief Superintendent Alex Murray. The authors also thank Jerry Lee of the Jerry Lee Foundation for creating the research center that designed and implemented the Turning Point Project in partnership with West Midlands Police.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

Notes

References

Ager

O’May

(2001). Issues in the definition and implementation of “best practice” for staff delivery of interventions for challenging behaviour. Journal of Intellectual & Developmental Disability, 26, 243–256.

Andrews

D. A.

Dowden

(2005). Managing correctional treatment for reduced recidivism: A meta analytic review of program integrity. Legal and Criminological Psychology, 10, 173–187.

Ariel

Farrar

Sutherland

(2014). The effect of police body-worn cameras on use-of-force and citizens’ complaints against the police: A randomized controlled trial. Journal of Quantitative Criminology. Published online: November 19, 2014.

Ariel

Neyroud

Slothower

(2014). Cambridge randomizer performance in a large scale, long-term experiment: Maintaining quality, reducing research costs. Presented at American Society of Criminology Conference, San Francisco, CA, November.

Ariel

Sherman

Vila

(2012). Random assignment without tears: How to stop worrying and love the Cambridge randomizer. Journal of Experimental Criminology, 8, 193–208.

Barnoski

(2002). Washington State’s implementation of Functional Family Therapy for juvenile offenders: Preliminary results (Document No. 02-08-1201). Olympia: Washington State Institute for Public Policy. Retrieved from http://www.wsipp.wa.gov/

Barnoski

(2004). Outcome evaluation of Washington State’s research-based programs for juvenile offenders (Document No. 04-01-1201). Olympia: Washington State Institute for Public Policy. Retrieved from http://www.wsipp.wa.gov/

Bronitt

Stenning

(2011). Understanding discretion in modern policing. Criminal Law Journal, 35, 319–332.

Davis

(1995). Changing physician performance: A systematic review of the effect of continuing medical education strategies. Journal of the American Medical Association, 274, 700–705.

10.

Dodge

K. A.

(2001). The science of youth violence prevention: Progressing from developmental epidemiology to efficacy to effectiveness to public policy. American Journal of Preventive Medicine, 20, 63–70.

11.

Fagan

Hanson

Hawkins

Arthur

(2008). Implementing effective community-based prevention programs in the community youth development study. Youth Violence and Juvenile Justice, 6, 256–278.

12.

Fixsen

Naoon

Blase

Friedman

Wallace

(2005). Implementation research: A synthesis of the literature. Tampa: University of South Florida, Louis de la Parte Florida Mental Health Institute, the National Implementation Research Network.

13.

Goggin

Gendreau

(2006). The implementation and maintenance of quality services in offender rehabilitation programmes. In Hollin

C. R.

Palmer

E. J.

(Eds.), Offending behaviour programmes: Development, application, and controversies (pp. 209–246). Chichester, England: Wiley.

14.

Goldstein

(1979). Improving policing: A problem oriented approach. Crime and Delinquency, 25, 236–258.

15.

Gottfredson

(2002). Quality of school-based prevention programs: Results from a national survey. Journal of Research in Crime and Delinquency, 39, 3–35.

16.

Grol

Grimshaw

(2003). From best evidence to best practice: Effective implementation of change in patients’ care. Lancet, 362, 1225–1230.

17.

Hallfors

Godette

(2002). Will the “principles of effectiveness” improve prevention practice? Early findings from a diffusion study. Health Education Research, 17, 461–470.

18.

Hasisi

Weisburd

(2011). Going beyond ascribed identities: The importance of procedural justice in airport security screening in Israel. Law and Society Review, 45, 867–892.

19.

Hobday

(2014). Targeting reasons for rejecting random assignment in an RCT (M.St. Dissertation). Cambridge, England: Institute of Criminology, University of Cambridge.

20.

Joyce

Showers

(2002). Student achievement through staff development (3rd ed.). Alexandria, VA: Association for Supervision and Curriculum Development.

21.

Karoly

L. A.

Greenwood

P. W.

Everingham

S. S.

Houbé

Kilburn

M. R.

Rydell

C. P.

(1998). Investing in our children. Santa Monica, CA: RAND.

22.

Kawamoto

Houliahan

Balas

Lobach

(2005). Improving clinical practice using clinical decision support systems: A systematic review of trials to identify features critical to success. BMJ, 330, 765. Epub 2005 Mar 14.

23.

Knitzer

Cooper

(2006). Beyond integration: Challenges for children’s mental health. Health Affairs, 25, 670–679.

24.

Koper

C. S.

Mayo-Wilson

(2006). Police crackdowns on illegal gun carrying: A systematic review of their impact on gun crime. Journal of Experimental Criminology, 2, 227–261.

25.

Landenberger

Lipsey

(2005). The positive effects of cognitive-behavioral programs for offenders: A meta-analysis of factors associated with effective treatment. Journal of Experimental Criminology, 1, 451–476.

26.

Latimer

(2001). A meta-analytic examination of youth delinquency, family treatment, and recidivism. Canadian Journal of Criminology, 43, 237–253.

27.

Lipsey

(1999). Can intervention rehabilitate serious delinquents? The Annals of the American Academy of Political and Social Science, 564, 142–166.

28.

Lipsey

(2009). The primary factors that characterize effective interventions with juvenile offenders: A meta-analytic overview. Victims and Offenders, 4, 124–147.

29.

Lipsey

Howell

Kelly

Chapman

Carver

(2010). Improving the effectiveness of juvenile justice programs: A new perspective on evidence-based practice. Washington, DC: Center for Juvenile Justice Reform, Georgetown University.

30.

Lipsey

Wilson

(1998). Effective intervention for serious juvenile offenders: A synthesis of research. In Loeber

Farrington

D. P.

(Eds.), Serious and violent juvenile offenders: Risk factors and successful interventions (Chap. 13, pp. 313–345). Thousand Oaks, CA: Sage.

31.

Lösel

Schmucker

(2005). The effectiveness of treatment for sexual offenders: A comprehensive meta-analysis. Journal of Experimental Criminology, 1, 117–146.

32.

Lowenkamp

L. E.

Smith

(2006). Does correctional program integrity really matter? The impact of adhering to the principles of effective intervention 2006. Criminology and Public Policy, 5, 201–220.

33.

Maloney

D. M.

Phillips

E. L.

Fixsen

D. L.

Wolf

M. M.

(1975). Training techniques for staff in group homes for juvenile offenders. Journal of Criminal Justice and Behavior, 2, 195–216.

34.

Mastrofski

(2004). Controlling street-level police discretion. Annals of the American Academy of Political and Social Science, 593, 100–118.

35.

Mazerolle

Bennett

Davis

Sargeant

Manning

(2013). Legitimacy in policing: A systematic review. Campbell Systematic Reviews, 9. http://www.campbellcollaboration.org/lib/project/141/

36.

McGuire

(2001). What works in correctional intervention? Evidence and practical implications. In Bernfeld

Farrington

Leschied

(Eds.), Offender rehabilitation in practice: Implementing and evaluating effective programs (pp. 25–43). London, England: Wiley.

37.

Meadows

Kinsella

Ellingworth

Wong

Senior

(2014). Mapping Restorative Provision in England & Wales: A National Report. Sheffield, UK: Hallam Centre for Community Justice at Sheffield Hallam University. Retrieved from http://creating-change.org.uk/resources/mapping-restorative-provision-in-england-wales

38.

Moore

(2002). Recognizing value in policing: The challenge of measuring police performance. Washington, DC: PERF.

39.

National Research Council. (2004). Fairness and effectiveness in policing: The evidence . Washington, DC: The National Academies Press.

40.

Nesovic

(2003). Psychometric evaluation of the correctional program assessment inventory (PhD Dissertation). Ottawa, Canada: Carlton University, Department of Psychology.

41.

Neyroud

(2008). Past, present and future performance: Lessons and prospects for the measurement of police performance. Policing, 2, 340–349.

42.

Neyroud

Slothower

(2013). Operation turning point: Second interim report. Interim report to West Midlands Police. Cambridge, England: Institute of Criminology, University of Cambridge.

43.

Neyroud

Slothower

(2012). Operation turning point: Interim report. Interim report to West Midlands Police. Cambridge, England: Institute of Criminology, University of Cambridge.

44.

Real

Poole

M. S.

(2005). Innovation implementation: Conceptualization and measurement in organizational research. Research in Organizational Change and Development, 15, 63–134.

45.

Rogers

(2002). White paper—The power of realization. Retrieved from http://www.ddiworld.com/research/publications.asp

46.

Sampson

R. J.

Laub

J. H.

(1993). Crime in the making: Pathways and turning points through life. Cambridge, MA: Harvard University Press.

47.

Schoenwald

S. K.

(2008). Toward evidence-based transport of evidence-based treatments: MST as an example. Journal of Child and Adolescent Substance Abuse, 17, 69–91.

48.

Shapland

Sorsby

Robinson

(2011). Restorative justice in practice: Evaluating what works for victims and offenders. New York, NY: Routledge.

49.

Shapland

Willmore

Duff

(1985). Victims in the criminal justice system. Aldershot, England: Gower.

50.

Sherman

(1998). Evidence-based policing. Washington, DC: Police Foundation.

51.

Sherman

(2011). Al Capone, the sword of Damocles, and the police–corrections budget ratio. Criminology & Public Policy, 10, 195–206.

52.

Sherman

(2013). The rise of evidence-based policing: Targeting, testing, and tracking. Crime and Justice, 42, 377–451.

53.

Sherman

Neyroud

(2012). Offender desistance policing and the sword of Damocles. London, England: Civitas.

54.

Sherman

Strang

(2014). The morality of evidence: The second annual restorative justice lecture. Presented at American Society of Criminology Conference, San Francisco, CA, November.

55.

Sherman

Strang

Woods

(2003). Captains of restorative justice: Experience, legitimacy, and recidivism by type of offense. In Weitekamp

Elmar

Kerner

Hans-Juergen

(Eds.), Restorative justice in context: International practice and direction (pp. 229–248). Dorset, England: Willan.

56.

Sherman

L. W.

Rogan

D. P.

(1995). Effects of gun seizures on gun violence: “Hot spots” patrol in Kansas City. Justice Quarterly, 12, 673–693.

57.

Sherman

L. W.

Williams

Ariel

Strang

L. R.

Wain

Slothower

Norton

(2014). An integrated theory of hot spots patrol strategy implementing prevention by scaling up and feeding back. Journal of Contemporary Criminal Justice, 30, 95–122.

58.

Slothower

(2014a). Strengthening police professionalism with decision support: Bounded discretion in out-of-court disposals. Policing, 8, 353–367.

59.

Slothower

(2014b). Victim perceptions of legitimacy of diversion to a conditional out-of-court disposal: A randomised controlled trial (M. A. Dissertation). Washington, DC: University of Maryland.

60.

Slothower

Joyce

(2014). Police restorative justice conferencing implementation across different management regimes. Manuscript in preparation. College Park: Department of Criminology and Criminal Justice, University of Maryland.

61.

Stokes

Baer

(1977). An implicit technology of generalization. Journal of Applied Behavior Analysis, 10, 349–367.

62.

St Pierre

T. L.

Osgood

D. W.

Mincemoyer

C. C.

Kaltreider

D. L.

Kauh

T. J.

(2005). Results of an independent evaluation of Project Alert delivered in schools by cooperative extension. Prevention Science, 6, 305–317.

63.

Strang

(2002). Repair or revenge: Victims and restorative justice. Claredon Studies in Criminology. New York, NY: Oxford University Press.

64.

Strang

Sherman

L. W.

Mayo-Wilson

Woods

Ariel

(2013). Restorative justice conferencing: Effects of face-to-face meetings of offenders and victims. A Systematic Review. Campbell Systematic Reviews, 12.

65.

Tyler

(1990). Why people obey the law. Princeton, NJ: Princeton University Press.

66.

Tyler

T. R.

(2006a). Legitimacy and legitimation. Annual Review of Psychology, 57, 375–400.

67.

Tyler

Tom R.

(2006b). What Do They Expect? New Findings Confirm the Precepts of Procedural Fairness. California Courts Review (p. 23). San Francisco, CA: Judicial Council of California.

68.

Tyler

Huo

Y. J.

(2002). Trust in the law: Encouraging public cooperation with the police and courts. New York, NY: Russell-Sage.

69.

Weisburd

Eck

J. E.

Hinkle

J. C.

Telep

C. W.

(2008). Effects of problem-oriented policing on crime and disorder. Campbell Collaboration Library of Systematic Reviews. Retrieved from http://www.campbellcollaboration.org/library.php

70.

Weisz

Sandler

Durlak

Anton

(2005). Promoting and protecting youth mental health through evidence-based prevention and treatment. American Psychologist, 60, 628–648.

71.

Welsh

B. C.

Sullivan

C. J.

Olds

D. L.

(2010). When early crime prevention goes to scale: A new look at the evidence. Prevention Science, 11, 115–125.

72.

Woods

(2009). Unpacking the impact of restorative justice in the RISE experiments: Facilitators, offenders, and conference non-delivery. Unpublished dissertation, University of Pennsylvania, Philadelphia, PA.