The Effectiveness of Policies and Programs That Attempt to Reduce Firearm Violence

Abstract

In response to rising rates of firearms violence that peaked in the mid-1990s, a wide range of policy interventions have been developed in an attempt to reduce violent crimes committed with firearms. Although some of these approaches appear to be effective at reducing gun violence, methodological variations make comparing effects across program evaluations difficult. Accordingly, in this article, the authors use meta-analytic techniques to determine what works in reducing gun violence. The results indicate that comprehensive community-based law enforcement initiatives have performed the best at reducing gun violence.

Keywords

firearm violence interventions meta-analysis

Violent crimes in the United States involving firearms reached epidemic proportions in the early 1990s (Wintemute, 1999). The crime rate increase was driven in large part by a spike in the gun homicide rates of urban youth that began in the mid-1980s (Fox, 1996). Since then, firearm violence in the United States has been a top priority for lawmakers, law enforcement agencies, and communities. Because of this growing public concern, during the 1990s, public policy responses specifically aimed at reducing gun violence began to surface.

These laws, programs, and interventions have ranged from medical campaigns to comprehensive community-oriented law enforcement strategies at local, state, and federal levels of government. Many of the more promising programs have been targeted at multiple levels of government (Sheppard, Rowe, Grant, & Jacobs, 2000). For example, in the first term of George W. Bush’s presidency, the federal government spent more than $1 billion in its own gun violence prevention strategy: Project Safe Neighborhoods (Herraiz, 2004). This approach, like many of the more costly anticrime policy interventions, encourages partnerships between local, state, and federal law enforcement agencies to establish multifaceted gun violence intervention strategies.

Despite their high costs, little is known about the effectiveness of these policy efforts. Systematic evaluations of programs are often overlooked by policy makers in favor of success accounts based on anecdotal evidence (Pratt, 2008). In addition, current evaluations of programs that attempt to reduce gun violence have indicated varying levels of success when they have been assessed empirically. Some programs have been shown to have a significant impact on gun violence, yet no research to date has attempted to quantitatively compare the relative effectiveness of the various gun violence reduction programs that have been put into place. To address this issue, we conducted a meta-analysis using standardized effect size estimates from studies that have evaluated gun violence interventions to determine the relative effectiveness of the various policy initiatives aimed at reducing firearm violence.

Variation in Gun Violence Interventions

Policy makers have approached the issue of gun violence in a variety of ways. These policy initiatives range from passing laws to create stiffer penalties for those convicted of gun crimes and to purchasing guns from citizens to reduce the availability of guns on the street. Accordingly, gun violence interventions can be categorized into four major areas: information, training, and storage campaigns; gun buy-back programs; gun laws; and law enforcement campaigns.

Information, Training, and Storage Campaigns

Emerging primarily out of the medical profession, these approaches view firearm violence as a public health risk. These programs have focused on providing counseling about the dangers of owning a firearm and giving information on the safe storage of these weapons (Grossman et al., 2000). Medical campaigns are based on the belief that informing gun owners about the health risks of owning a firearm will make them more likely to either remove the firearm from the home or to store it in a safe manner. Although some of this research addresses information given out by medical professionals (Emde, 2002; Grossman et al., 2000; Oatis, Buderer, Cummings, & Fleitz, 1999), other research has looked at whether traditional types of firearms training programs decrease gun ownership (Hemenway, Solnick, & Azrael, 1995). In the end, empirical research has found little support for the effectiveness of these programs in terms of reducing gun ownership and use.

Gun Buy-Back Programs

During the early 1990s, gun buy-backs, in which money or gift certificates of value are given in exchange for firearms, became a popular method to reduce the number of guns on the street. Based on the crime-control assumption of reducing gun availability (and thus gun crimes), these programs have been evaluated in different U.S. cities (Callahan, Rivara, & Koepsell, 1996; Rosenfeld, 1995) as well as nationally in Australia (Reuter & Mouzos, 2003). As such, no empirical research to date has shown significant changes in gun-related crimes due to these programs (Callahan et al., 1996; Reuter & Mouzos, 2003; Rosenfeld, 1995). Rosenfeld (1995) has nevertheless argued that these programs may be valuable to the extent that they enhance social cohesion, community bonds, or increase alternative forms of informal social control that may be capable of reducing gun violence.

Gun Laws

Laws have been enacted to reduce gun violence in many different ways, the most popular of which has been to increase the severity of legal sanctions for firearms-related crimes. In general, these laws either establish mandatory sentences or sentence enhancements (in some cases, both) in an effort to deter potential offenders from using a firearm when committing a crime (McPheters, Mann, & Schlagenhauf, 1984). The empirical research addressing the effectiveness of these laws is, at best, mixed (cf. Kleck & Patterson, 1993; Marvell & Moody, 1995; McDowall, Loftin, & Wiersema, 1992; McPheters et al., 1984).

Alternatively, there has been some research indicating that misdemeanants and felons who purchase handguns are at a greater risk of using such weapons in illegal activity (Wintemute, Drake, Beaumont, Wright, & Parham, 1998; Wintemute, Wright, Drake, & Beaumont, 2001; Wright, Wintemute, & Rivara, 1999). Under this assumption, other laws have attempted to restrict the ability of persons to purchase handguns through background checks and waiting periods at the time of sale. Again, the research shows contrasting results with regard to their effectiveness (Kleck & Patterson, 1993; McDowall, Loftin, & Wiersema, 1995; Ruddell & Mays, 2005).

Another way lawmakers have sought to reduce firearm crime is by passing laws that ban the sale and possession of particular types of firearms. To that end, empirical research indicates that the 1994 Federal Assault Weapons Ban had no significant effect on homicide rates, but this research was only able to examine short-term effects of the law (Koper & Roth, 2001). Research evaluating the impact of other types of bans has been mixed as well (cf. Britt, Kleck, & Bordura, 1996; Webster, Vernick, & Hepburn, 2002).

Finally, policy makers have attempted to reduce firearms violence by mandating that citizens store their firearms unloaded and locked when children may have access to them (Lott & Whitley, 1999). These laws are based on the premise that unloaded and locked firearms decrease the chance that children will use them in an unsafe manner and that such practices may also help to prevent theft. Still, critics of these laws argue that they reduce the ability of gun owners to effectively use their firearms in self-defense. Other critics have argued that these laws may actually result in increased crime rates by reducing the deterrent effect that an armed citizenry has on criminals—a concern that has some (albeit limited) empirical support (see Lott & Whitley, 1999).

Law Enforcement Campaigns

Policing

Policing strategies have focused on identifying problem areas or “hot spots.” Directed police patrols involve saturating hot spots with patrol officers based on the theory that increasing police presence will reduce firearm-related crime. In the seminal study in Kansas City, officers increased stops and searches of citizens in hot spot beats compared to control beats (Sherman & Rogan, 1995). The results indicated that the number of guns seized in the treatment beat increased and the number of gun crimes decreased. Replications and extensions of hot spots–directed patrol have shown support for these programs’ ability to reduce gun crime (Cohen & Ludwig, 2003; McGarrell, Chermak, Weiss, & Wilson, 2001).

Gun courts

The courts have also targeted gun violence. Some programs have attempted to speed up the processing of offenders who used a gun in the commission of a crime, and other jurisdictions have created specialized court systems for gun offenders. The Rhode Island Superior Court, for example, set up a gun court that hears these cases exclusively so that offenders will be brought to trial in the shortest possible time (Rodgers & Dimitri, 2004). Since the enactment of the gun court, the time it takes for cases to get to disposition has been reduced by more than 50%.

Another example of these types of judicial efforts is Detroit’s Handgun Intervention Program, which requires a 4-hour class on the negative consequences of gun use as a condition of pretrial release (Roth, 1998). Although Roth found that offenders’ attitudes toward gun use changed as a result of the program, it had little effect on their behavior. Finally, a more comprehensive program for youth offenders can be found in the Jefferson County (Alabama) Gun Court Program. Here, first-time gun offenders went to a boot camp, were required to complete drug and alcohol treatment, were put on intensive supervision, and their parents were required to attend counseling sessions. An evaluation of this program indicated that it significantly reduced criminal behavior relative to a matched pair of gun offenders (Cowin & Sloan, 2001).

Probation

Altering probation practices is another law enforcement strategy that has targeted gun offenders to reduce gun violence. Philadelphia’s Youth Violence Reduction Partnership is one such program that was intended to increase both accountability and social support to youth gun offenders on probation (McClanahan, 2004). Enhancing supervision is accomplished by increasing the level of contact that probation, police, and social workers have with probationers. This includes street patrols and home, school, and job visits. Probation officers also have the power to tighten or loosen the conditions of probation as they see fit (e.g., curfews and area restrictions). Line workers and probation officers are also responsible for providing positive supports, such as education, job searches, drug treatment programming, counseling, and organized recreation. An evaluation of the Youth Violence Reduction Partnership has shown it to be effective at reducing homicide in the areas that it operates in (McClanahan, 2004).

Federal prosecution

Recently, one of the more prevalent attempts to reduce gun violence has been the federal prosecution of gun offenders. Project Safe Neighborhoods targets gun violence by providing grants to cities to encourage the effective prosecution of gun offenders. Federal prosecutors work with state and local jurisdictions and attempt to impose harsher sentences for gun crimes. Project Safe Neighborhoods was modeled after Richmond, Virginia’s Project Exile, which used federal prosecution to deny bail and get longer sentences for those arrested for gun crimes (American Prosecutors Research Institute, 2002). Since it used federal prosecution, Project Exile focused on collaboration between federal and local authorities. It also contained a media campaign that sought to increase the deterrent effect of the crackdown.

Although Project Exile has reported increased convictions, increased length of sentences, shorter case processing times, and the increased denial of bail to gun offenders, its success in actually reducing gun crime has been the focus of controversy (Fahey et al., 1999; Johnson, Heineman, Smith, Walko-Frankovic, & Willard, 2003; Raphael & Ludwig, 2003; Rosenfeld, Fornango, & Baumer, 2005). The U.S. Attorney’s Office for the Eastern District of Virginia released a report claiming credit for a 40% reduction in homicide between 1997 and 1998 (Fahey et al., 1999). Follow-up analyses by Raphael and Ludwig (2003), however, found that the decrease in gun homicides was primarily due to pre-existing downward trends (also see Johnson et al., 2003; cf. Rosenfeld et al., 2005).

Community interventions

Scholars and practitioners have also begun to emphasize the importance of community-oriented strategies in combating firearms violence (Sheppard et al., 2000). These strategies focus on developing partnerships and seek to coordinate federal and state resources with local agencies to attack gun violence from all sides. To that end, police, probation, prosecution, social workers, and other community groups all take part in these holistic strategies to reduce gun violence. One such strategy is the Partnerships to Reduce Juvenile Gun Violence, created by the Office of Juvenile Justice and Delinquency Prevention. These programs focus on creating community partnerships that combat gun violence throughout the juvenile justice process. Not only do these programs target and increase supervision of high-risk juvenile offenders, but once these youths are arrested and prosecuted, these programs involve intense treatment efforts that continue through probation (see Sheppard et al., 2000).

As an example of this approach, Operation Eiger is a product of Baton Rouge’s Partnerships for the Prevention of Juvenile Gun Violence. Operation Eiger has three major areas of focus: suppression, intervention, and prevention of gun violence (Sheppard et al., 2000). Suppression focuses on using law enforcement strategies and citizen reports to seize illegal firearms, target gun traffickers, and prosecute gun crimes. Intervention focuses on risk assessment and treatment for youths on probation for gun offenses, where they are provided behavioral, substance abuse, and mental health counseling, along with an array of other types of counseling for family members. Finally, prevention efforts are provided to at-risk youth, such as informational school-based programs, mentoring programs, culture-specific and gender-specific treatment, and life skills training. An evaluation of Operation Eiger showed that youth involved were significantly less likely to commit a new criminal offense. Time-series analysis also indicated that neighborhood-level crime trends for gun assaults, robberies, and homicides significantly declined after the intervention (Sheppard et al., 2000).

Another strategy that has been used as a model for creating widescale community programs is the “Pulling Levers” strategy of Boston’s Operation Ceasefire program (see Braga, Kennedy, Waring, & Piehl, 2001). This program laid out a multidimensional approach to reduce gun violence among identified youth gang members. Its primary approach was to get community workers, churches, police, probation, and parole officers to reach out to gang members and deter them from gun-related violence. They did so by offering alternatives to violence while conveying the message that gun violence would provoke an immediate and intense response from law enforcement agencies. Police and probation officers cracked down in communities where gangs operated by focusing on a wide variety of criminal justice actions, ranging from federal prosecution, delivering strict terms of probation, and aggressively targeting low-level street crimes. Braga et al. (2001) found that this intervention significantly reduced homicides in Boston by 72% when controlling for seasonal trends and other relevant demographic factors. Recent analyses of 10 additional programs based on this model found them to be effective in reducing gun crime (Roehl et al., 2005).

Current Focus

The wide variation of policies and programs that have been developed to reduce gun violence appears to be matched by similar variation in the research results concerning their effectiveness. Indeed, some studies indicate that certain approaches are promising, others indicate that some initiatives do more harm than good, while others indicate null results. The purpose of this study, therefore, is to take a step back and examine this body of research in a systematic way in the form of a meta-analysis.

We recognize that the technique of meta-analysis is not without its critics (e.g., compare Logan & Gaes, 1993; Pratt, 2002; Rosenthal, 1979); yet this approach is becoming more popular in the criminal justice and criminology literature as scholars have noted its usefulness for establishing general patterns across a large number of empirical studies—especially when the results of such works are inconsistent (e.g., see Hsieh & Pugh, 1993; Lipsey, 1992; Lipsey & Derzon, 1998; Mitchell, Wilson, & MacKensie, 2005; Pearson & Lipton, 1999; Pratt & Cullen, 2000, 2005; Pratt, Cullen, Blevins, Daigle, & Madensen, 2006; Pratt, Cullen, Blevins, Daigle, & Unnever, 2002; Tittle, Villemez, & Smith, 1978). The central objective of this study is to use the meta-analytic technique to establish which types of policies and programs work in terms of reducing gun violence.

Method

Sample

The sample was generated by conducting a search of several electronic databases (NCJRS, ProQuest, First Search, Criminal Justice Abstracts, and Lexus Nexus) using different combinations of the search words: gun, firearm, handgun, violence, and crime. Also, consistent with Mullen (1989), an “ancestry approach,” which uses the reference lists from narrative reviews, was undertaken to identify studies that were missed in the electronic search. Using this approach, 47 studies were identified as evaluating a program or strategy that attempted to reduce gun violence.

Of this total, certain studies were excluded from the present analysis for a number of reasons. First, a small portion of the studies was undertaken at the individual level, which often measured outcomes in terms of recidivism. While important, these measures of effect size are quite different from the macro-level studies (which were most common) that measured crime rates. To avoid comparing effect sizes from fundamentally different units of analysis, this analysis includes only macro-level studies.¹ Second, studies were excluded if they either did not report the effect of the intervention on some type of violent or gun crime or if they measured the effect the program had on a non–crime-related measure of gun use (e.g., responsible storage of guns). Another portion of studies was excluded because they lacked methodological rigor and reductions in crime were reported based on anecdotal evidence only. Finally, some studies were excluded because an effect size estimate could not be calculated. In most cases, when a methodologically rigorous study failed to report sufficient statistical information, efforts were made to contact the authors to obtain the needed statistics. In some cases, the necessary information was provided and the study was included in the sample (McClanahan, 2004; Rosenfeld, 1995; Sheppard et al., 2000). In all, 29 studies were included in the analysis (see appendix). These studies produced a total of 172 estimates of the effect of some sort of policy or program on firearms violence.

Effect Size Estimate

The effect size estimate used here is the standardized correlation coefficient r. The coefficient r is particularly useful in meta-analysis because of the ease of interpretation and because of the ability to convert other test statistics into an r (Wolf, 1986). Since the distribution of r is skewed for all values other than zero, each r is converted into a z(r) score that has a distribution approaching normality (see Blalock, 1960).²

Independence of effect size estimates

As noted above, there are more effect size estimates (k = 172) than studies (n = 29). This is because most studies contained more than one effect size estimate. For example, most studies analyzed whether each program or strategy reduced multiple types of gun crime (e.g., Braga et al., 2001, reported Operation Ceasefire’s effect on the number of gun assaults, calls for shots fired, and youth homicide victims). There are two reasons for including multiple effect size estimates from these studies. First, selecting one effect size would limit our ability to assess how methodological variations in studies affected the effect size estimates (e.g., does the effect size of a program vary by methodological rigor of the program?). Second, in the absence of some defensible objective rule to govern which effect size estimate to include, choosing one effect size out of many may introduce, either wittingly or unwittingly, a researcher bias in favor of (or against) a particular empirical outcome (see Pratt & Cullen, 2000, p. 941).

Nevertheless, including multiple effect size estimates from the same study may potentially introduce inferential bias into the meta-analysis. In particular, because more than one effect size estimate is drawn from the same data set, the effect size estimates are not independent from each other and therefore the variance estimates produced from such studies may be biased downward. If so, it would be more likely that the mean effect size estimates produced from such studies will reveal a statistically significant overall effect size. Methods have been developed to address this issue (Raudenbush & Bryk, 2002) yet most often, using such techniques requires making assumptions that cannot be met (e.g., that Level 1 variances are known, which is not the case with the present body of literature). As an alternative, we calculated a pooled mean effect size per study (see appendix), which did not differ significantly from those reported in our main analyses. Thus, consistent with other meta-analytic studies in criminal justice and criminology that have addressed this issue empirically (Pratt & Cullen, 2000, 2005; Pratt et al., 2006; Pratt, McGloin, & Fearn, 2006), the lack of statistical independence does not appear to have significantly biased our study’s results.

Effect size predictor domains

The sample of studies produced effect size estimates that could be grouped into one of three broad predictor domains: gun laws, gun buy-back programs, and law enforcement strategies. Heterogeneity tests³ were conducted on the effect size estimates within each of these categories, and it was found that the gun law and law enforcement strategy predictor domains were significantly heterogeneous. To address this, these domains were disaggregated by differences in types of program and methodology. Types of gun laws include enhanced prison terms for gun crimes, waiting periods and background checks for gun purchases, safe storage laws, and weapons bans. Types of law enforcement strategies include policing strategies, probation strategies, prosecutorial strategies, and comprehensive community interventions.

To disaggregate our results according to the methodological rigor of the studies producing them, each effect size was rated for methodological quality on a scale similar to those used elsewhere (see Mitchell et al., 2005; Pearson & Lipton, 1999). The technique places effect sizes on a four-item scale ranging from poor quality to high quality. Effect sizes that were rated poor quality were those that used only before and after mean comparisons or some sort of bivariate correlation. Those rated limited quality used techniques to attempt to control for some intervening factors. Effect sizes rated fair quality used longitudinal techniques (e.g., Auto-regressive, integrated moving average [ARIMA]) to ensure that pre-existing trends were controlled. Finally, those effects that used either matched pairs or longitudinal techniques with a comparison group were rated high quality.

Analytic Strategy

The analysis proceeded in three stages. First, we calculated mean effect size estimates for the total sample (n = 168) across all studies. Second, after this general picture, we calculated separate mean effect sizes within the predictor domains of gun control laws, gun buy-backs, and law enforcement strategies to determine which types of strategies tend to be more effective than others for reducing gun violence. Finally, given the variation in gun control laws and law enforcement strategies, we broke these categories down even further to examine which types of laws and enforcement strategies appear to be most promising for reducing firearms violence.

Results

Mean effect sizes by predictor domain

Table 1 presents overall mean effect size estimates from all programs and the three major predictor domains. The overall mean effect size for all studies included in the analysis is −.144 (p < .05). This reveals that the studies included in this analysis, on average, indicated a weak to moderate impact of these policies and programs on gun violence. Furthermore, the fail safe N indicates that it would take a total of 81 null effect size estimates to reverse these findings.⁴ Still, when looking at effect size estimates by domain, it is apparent that the mean effect size for the entire sample is generated from different interventions that have different impacts on crime.

Table 1.

Effect Size Estimates by Domain

Type of Intervention	k	r	z(r)	Fail Safe N
All interventions	172	−.144*	−.153*	81
Gun buy-backs	16	−.010	−.011	—
Gun laws	87	−.089*	−.092*	18
Law enforcement	69	−.231*	−.245*	99.8

mean effect size statistically significant at p < .05.

The nonsignificant mean effect size coefficient for gun buy-back programs indicates that these interventions have performed poorly in reducing gun crime. On the other hand, gun laws have a negative and significant mean effect size. Still, the r of −.089 reveals that, on average, gun laws have had relatively weak effects on crime. The mean effect size found for law enforcement interventions indicates that these programs have also been found to significantly reduce gun crime. Furthermore, the r of −.231 clearly shows that programs in this domain have been most effective in reducing gun crime.

Mean Effect Sizes by Specific Type of Strategy

Some of the results reported in Table 1 may be potentially misleading because there is substantial variation in the types of gun laws and law enforcement strategies assessed and the methodological approaches used to evaluate them. Tables 2 and 3 disaggregate these categories by specifying the type of intervention and the level of methodological rigor used in the evaluation.

Table 2.

Effect Size Estimates by Type of Law

Type of Law	k	r	z(r)	Fail Safe N
Enhanced prison terms	35	−.089*	−.092*	17
High quality	15	−.081*	−.083*	2
Fair quality	10	.069	.074	—
Limited quality	10	−.151*	−.157*	6
Poor quality	0	—	—	—
Waiting period/background check	28	−.078*	−.081*	18
High quality	17	−.004	−.004	—
Fair quality	3	−.174	−.177	—
Limited quality	8	−.200*	−.207*	9
Poor quality	—	—	—	—
Weapons bans	19	−.194*	−.202*	20
High quality	12	−.207*	−.218*	15
Fair quality	0	—	—	—
Limited quality	1	−.038^a	−.038^a	—
Poor quality	6	−.193*	−.198*	6
Safe storage laws	5	.029	.030	—
High quality	5	.029	.030	—
Fair quality	0	—	—	—
Limited quality	0	—	—	—
Poor quality	0	—	—	—

no t test because there is only one r reported and thus no mean.

mean effect size statistically significant at p < .05.

Table 3.

Effect Size Estimates by Type of Law Enforcement Intervention

Type of Law Enforcement Intervention	k	r	z(r)	Fail Safe N
Policing strategy	13	−.233*	−.251*	21
High quality	11	−.234*	−.253*	18
Fair quality	0	—	—	—
Limited quality	0	—	—	—
Poor quality	2	−.229	−.241	3
Probation strategy	4	−.325*	−.340*	10
High quality	4	−.325*	−.340*	10
Fair quality	0	—	—	—
Limited quality	0	—	—	—
Poor quality	0	—	—
Prosecutorial strategy	8	−.082	−.082	—
High quality	2	−.157	−.159	—
Fair quality	6	−.057	−.057	—
Limited quality	0	—	—	—
Poor quality	0	—	—	—
Community programs	44	−.271*	−.289*	70
High quality	30	−.238*	−.251*	30
Fair quality	10	−.304*	−.330*	25
Limited quality	0	—	—	—
Poor quality	4	−.439*	−.472*	16

mean effect size statistically significant at p < .05.

Mean effects by type of gun law

Table 2 presents the mean effect size estimates for different types of laws by methodological quality rating. In particular, enhanced prison terms had a weak mean effect size (r = −.089), but these effects differ by methodology, where high quality studies show a significant yet weak mean effect size (r = −.081). The fail safe N reveals that only two more null effect sizes would need to be added to render this mean effect nonsignificant. Fair quality effects are not significant in reducing crime. Of interest, limited quality designs show the strongest mean effect size (−.151). This indicates that studies that are methodologically the weakest are most likely to reveal a significant crime-reducing effect of enhanced prison terms on crime.

Differences within the effects of waiting periods and background checks also indicate the need for caution in trusting findings from research of limited quality. A number of high-quality studies have found null effects (−.004). Still, including research of limited and fair quality (whose respective mean effects are −.200 and −.174) into the overall mean effect size gives the impression that these laws have been shown to significantly (yet weakly) reduce crime (r = −.078).

Weapons bans have an overall mean effect size of −.194. This indicates that these laws have been moderately effective in reducing crime. Furthermore, both high-quality and low-quality studies have similar mean effect sizes. On the other hand, the effects of safe storage laws show they have been ineffective at reducing gun crime. The five effect sizes examined were all of high quality and found that, if anything, safe storage laws work to increase crime.

Mean effects by type of law enforcement strategy

Although gun buy-backs and gun laws have been shown to have, at best, marginal effects on gun crime, law enforcement strategies provide much more promise. Overall, prosecutorial strategies show the least promise with a nonsignificant mean effect size. Of interest, policing strategies and community programs have moderate effects (r = −.233 and −.271, respectively). Finally, probation strategies have the strongest mean effect size (−.325).

Some caution should be taken, however, when examining the relative strength of the probation strategies’ mean effect size. The small number of effect sizes (k = 4), while of high quality, still produces a relatively small fail safe N (N_fs = 10), and these effect sizes were all obtained from one intervention. As more interventions are evaluated, the mean effect size could very easily change. Still, these results show that probation strategies have definite promise for reducing gun violence.

On the other hand, the mean effect size of community interventions is likely to remain quite stable. The 44 effect sizes combine with the strength of the mean effect size to produce a fail safe N of 70. In other words, community interventions have been shown to consistently produce some of the strongest effects across multiple studies and methodological approaches. Nevertheless, the effects of these interventions are also shown to be biased when failing to take into account the methodological quality of the studies being assessed. Although the mean effect size estimates from low-quality studies are quite large (r = –.439), effect sizes from high-quality studies are substantially smaller (r = −.238). Still, the overall mean effect size for community programs of −.271 is only marginally inflated by large effect sizes from poor quality studies.

Disaggregating the effect sizes from law enforcement interventions by methodological quality again highlights the need for caution when examining mean effect sizes that fail to account for methodological rigor. Similar mean effect sizes are found in both high and poor methodological quality studies of policing strategies (r = −.234 and −.229, respectively). Still, prosecutorial strategies show that the weak overall effect is driven by effect sizes from fair quality studies. High-quality prosecutorial effects, while too few to produce statistical significance, show moderate effects (r = −.157); yet a larger number of weak effects (r = −.057) from fair and limited quality estimates drives the overall mean effect size down (r = −.082).

Discussion

The results presented here indicate that there are certain gun violence reduction interventions that do not work, some that do work, and some that work better than others. The purpose of this meta-analysis was to assess this body of literature in a way that would reveal patterns across the various policies and interventions to aid in the development of approaches that do in fact work to reduce gun violence. To that end, the results of our study lead to three major conclusions.

First, when it comes to determining the empirical success or failure of any given gun violence reduction strategy, methodological quality matters. Disaggregating mean effect size estimates by methodological rigor revealed clear inconsistencies in results across studies. Even more troubling, in several cases, combining effect size estimates from studies with low methodological rigor with effect size estimates produced from more rigorous studies inflated overall estimates. For example, effect sizes from studies of limited quality showed modest strength for laws that instituted enhanced prison terms for gun crimes, while the effect sizes from the most rigorous studies showed that these laws had, at best, weak effects. This pattern was fairly consistent across our sample of studies, with many effect sizes from studies of limited and low rigor showing stronger effects than the effect sizes drawn from studies of high methodological quality. This highlights the importance of controlling for methodological rigor when conducting meta-analyses as well as using rigorous methodological designs when evaluating gun violence interventions.

Second, a number of politically popular programs show little or no promise for reducing gun violence. For example, gun buy-backs did not demonstrate empirical relationship with gun violence. These programs, at best, hope to affect gun crime indirectly by decreasing the availability of guns and thus reducing gun crimes. Little evidence has been shown to support the assumption that they are able to decrease the number of guns available to criminals, much less gun crime (Rosenfeld, 1995).

Also showing little promise were several popular types of gun laws. Effect sizes drawn from methodologically rigorous studies evaluating waiting periods and background checks were not statistically significant. Also, although it appears that enhanced prison terms show a significant and negative impact on gun violence, the mean effect size from the high-quality studies showed this relationship to be weak. This finding is certainly not anomalous in the criminological literature, where fear-based policies rooted in the deterrence theory framework that hope to increase the costs of crime have been shown to have little empirical support (Pratt & Cullen, 2005). In fact, the only laws that were shown to have marginal effects were bans on the sale of firearms. These laws most likely are effective to the extent that they target a measure of opportunity (the availability of guns), a concept with considerable support in the community victimization literature (see, e.g., Sampson & Wooldredge, 1987).

Finally, certain types of policies and programs do show considerable promise for reducing gun violence. Specifically, law enforcement programs are clearly more effective than gun laws. Furthermore, certain law enforcement programs and strategies are better at reducing firearms violence than others. For example, prosecutorial strategies were found to have, at best, marginal impacts on gun crime, whereas directed patrol policing strategies were shown to have a moderate impact on firearms violence. Also, probation-based strategies were shown to be quite successful at reducing gun violence; yet the small amount of existing research along these lines limits any firm conclusions in this area.

The programs that were found to be the most consistently effective were those that were also the most comprehensive. Our analysis of multidimensional, community-based approaches showed that these interventions noticeably outperformed other more limited interventions. This should come as no surprise because these programs capitalize on the strengths of multiple law enforcement strategies, such as directed patrol, federal prosecution, and specialized probation. Furthermore, the majority of these programs also included a community-level component that targeted well-established community risk factors, such as community organization and mobilization (see, e.g., Pratt & Cullen, 2005; Sampson, Morenoff, & Gannon-Rowley, 2002).

Also of interest, the effective community interventions examined here move beyond a purely punitive approach and seek to provide support to both the community and offenders (see Cullen, 1994). For example, Sheppard et al.’s (2000) evaluation of Operation Eiger provided intensive treatment to both offenders and their families—an approach that has plenty of support in the correctional intervention literature (Lipsey, 1992; Pratt, 2002). Alternatively, punitive interventions such as enhanced prison terms and prosecutorial strategies were shown to be much less effective. Furthermore, although some of the punitive strategies were shown here to have marginal impacts on gun violence, previous literature has pointed out that they tend to have adverse secondary impacts. For example, even though enhanced prison terms significantly decreased violent crime, Mauer (2006) discussed how they have also served to increase prison populations and undercut the perceived legitimacy of the criminal justice system.

This is not to say that providing support to gun offenders is the sole means to reduce gun violence. In fact, the most effective programs combined both punitive and supportive strategies to effectively reduce gun violence. As McClanahan’s (2004) evaluation shows, there is clear promise for programs that attempt to increase both accountability and social support to the program’s participants.

In conclusion, it has become more politically fashionable in recent years to create evidence-based crime control policies (MacKenzie, 2000; Pratt, in press; Sherman et al., 1997). The area of gun violence—where issues of emotion and political capital may still trump empiricism—is nevertheless amenable to an evidence-based approach. To be sure, a large body of empirical evidence concerning the problem of gun violence has been produced. The assessment of it here provides clear guidance concerning which approaches are most likely to result in enhanced public safety—an outcome that should be attractive to policy makers regardless of their ideological persuasion.

Footnotes

Appendix

Pooled Effect Size Estimates by Study

Study	r	s	k
Braga, Kennedy, Waring, & Piehl (2001)	−.412	.126	9
Britt, Kleck, & Bordura (1996)	−.149	.149	8
Cohen & Ludwig (2003)	−.416	.201	4
Jung & Jason (1988)	.240	.235	4
Kleck & Patterson (1993)	−.026	.074	9
Koper & Roth (2001)	−.037	—	1
Kwon, Scott, Safranski, & Bae (1997)	−.242	.068	2
Lester & Murrell (1986)	−.005	.106	2
Loftin, Heumann, & McDowall (1983)	−.101	.097	9
Loftin & McDowall (1984)	−.105	.177	3
Lott & Whitley (1999)	.029	.070	5
Ludwig & Cook (2000)	.002	.034	16
Marvell & Moody (1995)	.037	.068	3
McClanahan (2004)	−.325	.092	4
McDowall, Loftin, & Wiersema (1992)	−.044	.253	2
McDowall, Loftin, & Wiersema (1995)	−.157	.089	4
McGarrell & Chermak (2004)	−.218	.225	5
McGarrel, Chermak, Weiss, & Wilson (2001)	−.065	.133	4
McPheters, Mann, & Schlagenhauf (1984)	−.337	.115	4
O’Carroll et al. (1991)	.132	.069	2
Raphael & Ludwig (2003)	−.057	.041	6
Roehl et al. (2005)	−.478	.059	4
Rosenfeld (1995)	−.010	.094	16
Rosenfeld, Fornango, & Baumer (2005)	−.142	.059	6
Ruddell & Mays (2005)	−.319	.058	5
Sheppard, Rowe, Grant, & Jacobs (2000)	−.296	.201	6
Sherman & Rogan (1995)	−.281	.199	3
Tita et al. (2003)	−.176	.146	18
Webster, Vernick, & Hepburn (2002)	−.304	.183	8

An earlier version of this article was presented at the 2005 Annual Meeting of the Academy of Criminal Justice Sciences in Chicago.

The authors declared no potential conflicts of interests with respect to the authorship and/or publication of this article.

The authors received no financial support for the research and/or authorship of this article.

Notes

References

American Prosecutors Research Institute. (2002). Combating gun violence: Promising practices for America’s prosecutors. Alexandria, VA: Author.

Blalock

H. M.

(1960). Social statistics. New York: McGraw-Hill.

Braga

A. A.

Kennedy

D. M.

Waring

E. J.

Piehl

A. M.

(2001). Problem-oriented policing, deterrence, and youth violence: An evaluation of Boston’s Operation Ceasefire. Journal of Research in Crime and Delinquency, 38, 195-225.

Britt

C. L.

Kleck

Bordura

D. J.

(1996). A reassessment of the D. C. Gun Law: Some cautionary notes on the use of interrupted time series designs for policy impact assessment. Law & Society Review, 30, 361-380.

Callahan

C. M.

Rivara

F. P.

Koepsell

T. D.

(1996). Money for guns: Evaluation of the Seattle gun buy-back program. Public Health Reports, 109, 472-477.

Cohen

Ludwig

(2003). Policing crime guns. In Ludwig

Cook

P. J.

(Eds.), Evaluating gun policy: Effects on crime and violence (pp. 217-239). Washington, DC: Brookings Institution.

Cowin

J. A.

Sloan

J. J.

(2001). Final report: Jefferson County gun court program: Process and impact. Unpublished manuscript, COSMOS, Inc.

Cullen

F. T.

(1994). Social support as an organizing concept for criminology: Presidential address to the Academy of Criminal Justice Sciences. Justice Quarterly, 11, 527-559.

Emde

(2002). Cops and docs: A gun injury prevention program. International Journal of Trauma Nursing, 8, 54-56.

10.

Fahey

H. F.

Comey

J. B.

Miller

S. W.

Schiller

Heaps

McKeel

(1999). Project Exile: Richmond. Richmond: United States Attorney’s Office for the Eastern District of Virginia. Retrieved June 14, 2006, from http://www.vahv.org/Exile/Richmond/PE-R005.html

11.

Fox

J. A.

(1996). Trends in juvenile violence: A report to the United States Attorney General on current and future rates of juvenile offending. Washington, DC: Bureau of Justice Statistics.

12.

Grossman

D. C.

Cummings

Koepsell

T. D.

Marshall

D’Ambrosio

Thompson

R. S.

Mack

(2000). Firearm safety counseling in primary care pediatrics: A randomized, controlled trial. Pediatrics, 106, 22-26.

13.

Hemenway

Solnick

S. J.

Azrael

D. R.

(1995). Firearm training and storage. Journal of the American Medical Association, 273, 46-50.

14.

Herraiz

(2004). Project Safe Neighborhoods: America’s network against gun violence. Washington, DC: Bureau of Justice Assistance.

15.

Hsieh

Pugh

M. D.

(1993). Poverty, income inequality and violent crime: A meta-analysis of recent aggregate data studies. Criminal Justice Review, 18, 182-203.

16.

Johnson

Heineman

Smith

Walko-Frankovic

Willard

T. B.

(2003). Evaluation of the Virginia Exile Program, Final Report. Richmond: Virginia Department of Criminal Justice Services, Criminal Justice Research Center.

17.

Jung

R. S.

Jason

L. A.

(1988). Firearm violence and the effects of gun control legislation. American Journal of Community Psychology, 16, 515-524.

18.

Kleck

Patterson

E. B.

(1993). The impact of gun control rates and gun ownership levels on violence. Journal of Quantitative Criminology, 9, 249-287.

19.

Koper

C. S.

Roth

J. A.

(2001). The impact of the 1994 Federal Assault Weapons Ban on gun violence outcomes: An assessment of multiple outcome measures and some lessons for policy evaluation. Journal of Quantitative Criminology, 18, 239-266.

20.

Kwon

I. G.

Scott

Safranski

S. R.

Bae

(1997). The effectiveness of gun control laws: A multivariate statistical analysis. The American Journal of Economics and Sociology, 56, 41-50.

21.

Lester

Murrell

M. E.

(1986). The influence of gun control laws on personal violence. Journal of Community Psychology, 14, 315-318.

22.

Lipsey

M. W.

(1992). Juvenile delinquency treatment: A meta-analytic inquiry into the variability of effects. In Cook

T. D.

Cooper

Cordray

D. S.

Hartmann

Hedges

L. V.

Light

R. J.

. (Eds.), Meta-analysis for explanation (pp. 83-127). New York: Russell Sage.

23.

Lipsey

M. W.

Derzon

J. H.

(1998). Predictors of serious or violent delinquency in adolescence and early adulthood: A synthesis of longitudinal research. In Loeber

Farrington

D. P.

(Eds.), Serious and violent Juvenile Offenders: Risk factors and successful interventions (pp. 86-106). Thousand Oaks, CA: Sage.

24.

Loftin

Heumann

McDowall

(1983). Mandatory sentencing and firearms violence: Evaluating an alternative to gun control. Law & Society Review, 17, 287-318.

25.

Loftin

McDowall

(1984). The deterrent effects of the Florida felony firearm law. The Journal of Criminal Law and Criminology, 75, 250-259.

26.

Logan

C. H.

Gaes

G. G.

(1993). Meta-analysis and the rehabilitation of punishment. Justice Quarterly, 10, 245-263.

27.

Lott

J. R.

Whitley

J. E.

(1999). Safe storage gun laws: Accidental deaths, suicides, and crime. Yale Law School, Law & Economics Working Paper No. 237. Retrieved October 17, 2006, from http://ssrn.com/abstract=228534

28.

Ludwig

Cook

P. J.

(2000). Homicide and suicide rates associated with the implementation of the Brady Handgun Violence Prevention Act. Journal of the American Medical Association, 284, 585-591.

29.

MacKenzie

D. L.

(2000). Evidence-based corrections: Identifying what works. Crime and Delinquency, 46, 457-471.

30.

Marvell

T. B.

Moody

C. E.

(1995). The impact of enhanced prison terms for felonies committed with guns. Criminology, 33, 247-260.

31.

Mauer

(2006). Race to incarcerate (2nd ed.). New York: New Press.

32.

McClanahan

W. S.

(2004). Alive at 25: Reducing youth violence through monitoring and support (Field report series). Philadelphia: Private Versus Public Ventures.

33.

McDowall

Loftin

Wiersema

(1992). A comparative study of the preventive effects of mandatory sentencing laws for gun crimes. Journal of Criminal Law and Criminology, 83, 378-390.

34.

McDowall

Loftin

Wiersema

(1995). Easing concealed firearms laws: Effects on homicides in three states. Journal of Criminal Law and Criminology, 8, 193-206.

35.

McGarrell

E. F.

Chermak

(2004). Strategic approaches to reducing firearms violence: Final report on the Indianapolis violence reduction partnership (Report No. 203976). Washington, DC: National Institute of Justice.

36.

McGarrell

E. F.

Chermak

Weiss

Wilson

J. M.

(2001). Reducing firearms violence through directed patrol. Criminology & Public Policy, 1, 119-148.

37.

McGarrell

E. F.

Chermak

Wilson

J. M.

Corsaro

(2006). Reducing homicide through a “pulling levers” strategy. Justice Quarterly, 23, 214-231.

38.

McPheters

L. R.

Mann

Schlagenhauf

(1984) Economic response to a crime deterrence program: Mandatory sentencing for robbery with a firearm. Economic Inquiry, 22, 550-570.

39.

Mitchell

Wilson

D. B.

MacKensie

D. L.

(2005). The effectiveness of incarceration-based drug treatment on criminal behavior: A Campbell collaboration systematic review. Retrieved November 20, 2007, from http://www.campbellcollaboration.org/doc-pdf/ Incarceration-BasedDrugTxSept06final.pdf

40.

Mullen

(1989). Advanced BASIC meta-analysis. Hillsdale, NJ: Lawrence Erlbaum.

41.

Oatis

P. J.

Fenn Buderer

N. M.

Cummings

Fleitz

(1999). Pediatric practice based evaluation of the Step to Prevent Firearm Injury program. Injury Prevention, 5, 48-52.

42.

O’Carroll

P. W.

Loftin

Waller

J. B.

McDowall

Bukoff

Scott

R. O.

Mercy

J. A.

Wiersema

(1991). Preventing homicide: An evaluation of the efficacy of a Detroit gun ordinance. American Journal of Public Health, 81, 576-581.

43.

Pearson

F. S.

Lipton

D. S.

(1999). A meta-analytic review of the effectiveness of corrections-based treatments for drug abuse. The Prison Journal, 79, 384-410.

44.

Pratt

T. C.

(2002). Meta-Analysis and its discontents: Treatment destruction techniques revisited. Journal of Offender Rehabilitation, 35, 23-40.

45.

Pratt

T. C.

(2008). Rational choice theory, crime control policy, and criminological relevance. Criminology and Public Policy, 7, 43-52.

46.

Pratt

T. C.

(in press). Addicted to incarceration: Corrections policy and the politics of misinformation in the United States. Thousand Oaks, CA: Sage.

47.

Pratt

T. C.

Cullen

F. T.

(2000). The empirical status of Gottfredson and Hirschi’s General Theory of Crime: A meta-analysis. Criminology, 38, 931-964.

48.

Pratt

T. C.

Cullen

F. T.

(2005). Assessing macro-level predictors and theories of crime: A meta-analysis. Crime and Justice, 32, 373-450.

49.

Pratt

T. C.

Cullen

F. T.

Blevins

K. R.

Daigle

Madensen

(2006). The empirical status of deterrence theory: A meta-analysis. In Cullen

F. T.

Wright

J. P.

Blevins

K. R.

(Eds.), Taking stock: The empirical status of criminological theory—Advances in criminological theory (Vol. 15, pp. 367-395). New Brunswick, NJ: Transaction Books.

50.

Pratt

T. C.

Cullen

F. T.

Blevins

K. R.

Daigle

Unnever

J. D.

(2002). The relationship between attention deficit hyperactivity disorder to crime and delinquency: A meta-analysis. International Journal of Police Science & Management, 4, 344-360.

51.

Pratt

T. C.

McGloin

K. M.

Fearn

N. E.

(2006). Maternal cigarette smoking during pregnancy and criminal/deviant behavior: A meta-analysis. International Journal of Offender Therapy and Comparative Criminology, 50(6), 672-690.

52.

Raphael

Ludwig

(2003). Prison sentence enhancements: The case of Project Exile. In Ludwig

Cook

P. J.

(Eds.), Evaluating gun policy: Effects on crime and violence (pp. 251-278). Washington, DC: Brookings Institution.

53.

Raudenbush

S. W.

Bryk

A. S.

(2002). Hierarchical linear models: Applications and data analysis methods (2nd ed.). Thousand Oaks, CA: Sage.

54.

Reuter

Mouzos

(2003). Australia: A massive buyback of low-risk guns. In Ludwig

Cook

P. J.

(Eds.), Evaluating gun policy: Effects on crime and violence. Washington, DC: Brookings Institution.

55.

Rodgers

J. F.

Dimitri

W. A.

(2004). Rhode Island Superior Court Gun Court Statistics: September 12, 1994-March 31, 2004. Unpublished manuscript, Rhode Island Superior Court, Providence.

56.

Roehl

Rosenbaum

D. P.

Costello

S. K.

Coldren

J. R.

Schuck

A. M.

Kunard

. (2005). Strategic approaches to Community Safety Initiative (SACSI) in 10 U.S. cities: The building blocks for Project Safe Neighborhoods (Final Report No. 212866). Washington, DC: National Institute of Justice.

57.

Rosenfeld

(1995). Gun buy-backs: Crime control or community mobilization? In Plotkin

M. R.

(Ed.), Under fire: Gun buy-backs, exchanges, and amnesty programs. Washington, DC: Police Executive Research Forum.

58.

Rosenfeld

Fornango

Baumer

(2005). Did Ceasefire, Compstat, and Exile reduce homicide? Criminology & Public Policy, 4, 419-449.

59.

Rosenthal

(1979). The ‘file drawer’ problem and tolerance for null results. Psychological Bulletin, 86, 638-641.

60.

Roth

(1998). The Detroit Handgun Intervention Program: A court-based program for youthful handgun offenders (Research preview). Washington, DC: National Institute of Justice.

61.

Ruddell

Mays

G. L.

(2005). State background checks and firearms homicides. Journal of Criminal Justice, 33, 127-136.

62.

Sampson

R. J.

Morenoff

J. D.

Gannon-Rowley

(2002). Assessing “neighborhood effects”: Social processes and new directions in research. Annual Review of Sociology, 28, 443-478.

63.

Sampson

R. J.

Wooldredge

(1987). Linking the micro- and macro-level dimensions of lifestyle-routine activity and opportunity models of predatory victimization. Journal of Quantitative Criminology, 3, 371-393.

64.

Sheppard

Rowe

Grant

Jacobs

(2000). Fighting juvenile gun violence (Juvenile Justice Bulletin). Washington, DC: Office of Juvenile Justice, Delinquency, and Programs.

65.

Sherman

L. W.

Gottredson

D. C.

MacKenzie

D. L.

Eck

Reuter

Bushway

S. D.

(1997). Preventing crime: What works, what doesn’t, what’s promising (NCJ report No. 171676). Washington, DC: National Institute of Justice.

66.

Sherman

L. W.

Rogan

D. P.

(1995). Effects of gun seizures on gun violence: “Hot Spots” patrol in Kansas City. Justice Quarterly, 12, 673-693.

67.

Tita

Riley

K. J.

Ridgeway

Grammich

Abrahamse

A. F.

Greenwood

P. F.

(2003). Reducing gun violence: Results from an intervention in east Los Angles. Santa Monica, CA: RAND.

68.

Tittle

C. R.

Villemez

W. J.

Smith

D. A.

(1978). The myth of social class and criminality: An assessment of the empirical evidence. American Sociological Review, 43, 643-656.

69.

Webster

D. W.

Vernick

J. S.

Hepburn

L. M.

(2002). Effects of Maryland’s law banning “Saturday night special” handguns on homicides. American Journal of Epidemiology, 155, 406-412.

70.

Wintemute

G. J.

(1999). The future of firearm violence prevention: Building on success. Journal of the American Medical Association, 281, 475-478.

71.

Wintemute

G. J.

Drake

Beaumont

J. J.

Wright

M. A.

Parham

C. A.

(1998). Prior misdemeanor convictions as a risk factor of later violent and firearm-related criminal activity among authorized purchasers of handguns. Journal of the American Medical Association, 280, 2083-2087.

72.

Wintemute

G. J.

Wright

M. A.

Drake

C. M.

Beaumont

J. J.

(2001). Subsequent criminal activity among violent misdemeanants who seek to purchase handguns: Risk factors and effectiveness of denying handgun purchase. Journal of the American Medical Association, 285, 1019-1026.

73.

Wolf

F. M.

(1986). Meta-analysis: Quantitative methods for research synthesis. Beverly Hills, CA: Sage.

74.

Wright

M. A.

Wintemute

G. J.

Rivara

F. P.

(1999). Effectiveness of denial of handgun purchase to persons believed to be at high risk for firearm violence. American Journal of Public Health, 89, 88-90.