Juvenile Curfew Effects on Criminal Behavior and Victimization: A Systematic Review

3.2.1 Type of curfew ordinance evaluated

Eligible studies must have tested the effect of an official state or local policy intended to restrict or otherwise penalize a juvenile's presence outside the home during certain times of day. The curfew policy must have been a general preventive measure directed at all youth within a certain age range. Thus, we excluded studies that examined curfews imposed as part of a specific sentence or probation condition for an individual youth. The curfew ordinance must have been passed as a measure to improve public safety through the specific targeting of juvenile crime and delinquency; military or riot curfews directed at all citizens were excluded. Also excluded were ordinances that solely placed restrictions on youth driving. Curfew policies that included exemptions for certain types of legitimate activity, such as employment, were eligible, as most curfews have such exemptions.

3.2.2 Types of research designs

All quantitative evaluation designs were eligible. We expected that most studies of juvenile curfews would have been conducted at the macro- or aggregate-level (i.e., comparing crime and delinquency rates in jurisdictions where policies are imposed to rates in the same jurisdiction prior to the policy, or to comparable jurisdictions that do not impose a curfew). Such designs are often called ecological studies (Hingson, Howland, Koepsell, & Cummings, 2001). This expectation stemmed from the nature of the intervention. Curfews are imposed categorically on all juveniles living within a specific geographic area. As such, randomized controlled trials at the individual-level are not possible. Although randomized controlled trials at the jurisdiction (or geographic) level were eligible, we anticipated that these were highly unlikely to exist given the legal nature of curfews. Jurisdictions legally cannot agree to a study where a random process determined whether they would pass or not pass a juvenile curfew ordinance.

The strongest study design we expected to find was an interrupted time-series, including designs with multiple series across different cities (i.e., panel studies). For our purpose, an interrupted time series involves multiple measurements of the outcome both before and after the curfew ordinance went into effect. Interrupted time-series designs are often considered one of the stronger quasi-experimental designs (e.g., Shadish, Cook, and Campbell, 2002). The primary threat to internal validity for this design is a historical artifact - that is, some other change coinciding with the imposition of the curfew that might account for the change in crime rates. A variant on this is a study with multiple interrupted time-series with one or more series serving as a comparison, such as a community that did not implement a curfew. These have clear advantages over a simple single time-series in that broad-scale historical effects can be statistically controlled for given the staggered timing of the start of the curfew laws. These designs also account for regression to the mean (see Kline, 2012).

Micro-level studies at the individual level were eligible. For example, a study might compare youth living in various jurisdictions, some with and some without a curfew. Similarly, non-equivalent comparison group designs at the macro-level that contrast crime rates for communities with and without curfews were eligible.

A simple pre-post analysis at the macro-level is similar to an interrupted time-series, but lacks the necessary multiple pre- and post-measurement time points (i.e., they don't have a series of observations for each time period). These studies are at high risk of bias given that there is no assessment of the underlying trend in the crime rate (i.e., whether crime was already increasing or decreasing before the start of a curfew). These designs were eligible, but were kept separate in all analyses and figures and did not weigh into our overall assessment of the effectiveness of juvenile curfews. We included them to fully document the statistical evidence on this topic. For a study to be included in this synthesis, it must have been possible to compute an index of the effect of the curfew on crime or victimization.

3.2.3 Types of participants

Juvenile curfews do not have participants in the typical sense. To be eligible, the curfew ordinance must have applied to youth within a specified age range within a given jurisdiction.

3.2.4 Types of outcome measures

Eligible studies had to have measured and reported data on at least one of the two primary outcomes of interest: juvenile crime or juvenile victimization (where a youth is the victim of a crime). Outcome data may have been based on official records (arrests, charges, convictions, etc.) or self-reported measures. These outcomes may have reflected crime only during the curfew hours or crime at any time of day. For example, a study may have reported the total juvenile arrest rate by month with this rate reflecting both curfew and non-curfew hours combined. The logic of accepting the latter as a valid primary outcome is that curfews are intended to reduce juvenile offending overall, and not just displace crimes from the curfew to non-curfew hours. Similarly, victimization data might reflect victimization during curfew hours or victimization at any time of day.

The secondary outcomes examined in this review are better conceptualized as control outcomes. That is, outcomes that should be unaffected by a curfew ordinance. These include adult crime rates and juvenile crime rates during non-curfew hours. No other outcome measures were coded as part of this review.

3.2.5 Types of settings

Eligible studies could have been conducted in any country and published in any form. Given the linguistic limitations of the authors, only English language publications were eligible. We worked with our international contacts within the Campbell Collaboration to learn which countries were likely to have used and evaluated juvenile curfew ordinances so that we could target our searches appropriately. As discussed in the Background section of this review (Section 1), only the United States and Iceland have or currently make use of the type of curfews eligible for this review. We were unable to identify any Icelandic studies. Thus, the English language restriction is unlikely to have resulted in the exclusion of any study (i.e., we are not aware of any other eligible study reported in a language other than English).

We restricted this review to studies using data collected after 1959. The generalizability of any study conducted prior to 1960 to the present is questionable given the markedly different current social and legal context.

3.4.1 Data extraction and management

In addition to extracting effect size data as described in the next section, we coded descriptive information about the location of the research, the nature of the curfew, and the methodological characteristics of the study (see ‘Coding Manual’ in Section 14). All descriptive study information was double-coded and the first author resolved any differences. The first author coded effect sizes and all of the data and computations relevant to each effect size are presented in Section 9. Double-coding of effect sizes was not feasible given the complexity of these computations. However, a second coder verified the accuracy of the data used in these computations. At the time of coding, the second coder was a doctoral candidate with over three years of experience in working with systematic reviews and data extraction.

3.4.2 Measure of Treatment Effect

The most common research design was a macro-level examination of crime or victimization counts over time. Typical effect sizes, such as the standardized mean difference and the odds ratio, cannot be computed from the data generated by these designs. However, it is possible to use percent change in the crime or victimization count as the effect size.

A simple comparison of the relative pre-and post-crime count (or rate) associated with a curfew may be confounded with an underlying upward or downward trend in crime. For example, most of the studies included in this review were conducted in the United States during the late 1990s or early 2000s. This time period coincides with what is often referred to as the “great crime drop,” during which almost all categories of crime decreased across the country. Because of this, it was desirable to have a measure of the effect of a curfew that accounted for this underlying trend.

Five basic design types were used across the eligible studies: (1) an interrupted time-series with 26 or more time points (months or years), some before and some after the curfew ordinance went into effect; (2) a short interrupted time-series with 4 to 6 time points (e.g., years), with only 2 or 3 time points before and after the start of the curfew ordinance; (3) a simple pre-post comparison of counts (or rates) for one year before and one year after the adoption of a curfew; (4) regression analysis of crime rates for multiple cities or counties across multiple years; and (5) an individual level logistic regression analysis of juveniles geo-coded into locations with and without a curfew. Each of these designs requires a different method of effect estimation. The first step was to estimate the difference in the logged rate or count, ideally adjusted for any time trend. These logged rate differences were then converted into a percent difference, as explained below.

For studies that used an interrupted time-series design, percent change was estimated using a negative binomial model on the crime or victimization counts over time. Although count data such as these can be estimated with a Poisson model, crime data are typically over-dispersed and, as such, more appropriately handled with a negative binomial model. Both Poisson and negative binomial regression models produce comparable coefficients, but the latter produces larger standard errors unless the data are not over-dispersed. Data were extracted from tables (if available) or figures, and inputted into the statistical software program ‘R‘. PlotDigitizer (see http://plotdigitizer.sourceforge.net) was used to extract data from figures. Data that reflected rates and not counts were converted into approximate counts using a constant population estimate before running the negative binomial model. This conversion does not affect the estimated effect, but does produce more accurate standard errors. The model estimated included a term for time (month or year) and a dummy code for whether the curfew policy was in effect (1=yes, 0=no). The coefficient for the curfew dummy reflects the difference in the log number of crimes for the post versus pre periods adjusting for any linear time trend.

The negative binomial model could not be used for short time-series (e.g., 4 to 6 time points) as there were insufficient data to estimate the over-dispersion parameter. However, these studies have sufficient data to estimate the curfew effect using a Poisson model. Given that the variance of the Poisson distribution equals the mean, a Poisson-based model can estimate the standard error of the coefficient even with only four data points, two prior to the curfew and two after the curfew. The standard errors from a Poisson model on these data are likely to be negatively biased (i.e., too small) given that these standard errors do not take into account any over-dispersion. Furthermore, the limited number of time points affects the precision of the estimate of the linear time trend. These potential biases with the estimates from these studies were considered when interpreting the results. As with the negative binomial, the coefficient for the curfew dummy variable reflected the difference in the log number of crimes for the post- versus pre-curfew period, adjusting for any linear time trend.

Studies that provided data on the rate or count of crimes or victimization for some period of time (usually one year) before and after the adoption of a juvenile curfew provided the weakest evidence on the effectiveness of the curfew. With such data it is not possible to estimate the underlying linear trend in the crime rate. Percent change, however, was computed as the difference in the two counts divided by the baseline value. We made use of the Poisson distribution for estimating the standard error. This made the assumption that these counts were Poisson distributed, as with the short time-series studies. The method used was adapted from Ng and Tang (2005). Assuming that the two time periods are equal (which they were in all cases), the standard error for the difference in the logged counts was computed as

s e = 1 x 1 + 1 x 2

The fourth design type used OLS regression to model the logged arrest rates for multiple cities over multiple years. Some or all of the selected cities experienced a change in curfew policy during the years examined. Because the dependent variable was logged, the regression coefficient for the curfew indicator variable is directly interpretable as the difference in the logged crime rates, post- versus pre-curfew, adjusted for the linear effect of year and any other covariates in the model. As such, the coefficient associated with the curfew was coded as the effect size. The standard error reported in the study for this coefficient was also used as the standard error used to compute the inverse variance weight. Note also that the percent change in rates is directly comparable to the percent change in counts, given that the former is simply the latter divided by a constant (the population).

Finally, a single study reported on a logistic regression model based on individual level data. This study produced an odds ratio as the estimate of the effect of a juvenile residing in a jurisdiction with a curfew, relative to a juvenile residing in a jurisdiction without a curfew. Converting an odds ratio into a percent change relative to a baserate requires knowing the baserate, which was not provided in this study. However, applying the same conversion that was used for the logged differences to the logged odds ratio produces a percent change in the odds relative to the baseline odds. This was done in this case to improve comparability across forest plots. The effect sizes from this study were not, however, combined with effect sizes from other studies.

Although different methods were used to compute the effect size across these different design types, the effects are comparable in that they all reflect percent change in crime counts or rates, with the exception of the logistic regression on individual data. The latter was treated separately in all analyses. Although the metric is the same across these different estimation methods, that is, percent change, the effect estimate do vary in their ability to isolate the curfew effect from other potential sources of change. As such, estimates that adjust for a linear time trend were handled separately from those that do not. The estimates based on multicity data over numerous years include covariates, such as population size. These covariates do not affect the metric of the effect size (it remains an index of percent change associated with the introduction of the curfew policy) but do remove potential sources of bias.

Meta-analyses were performed on the logged difference effect sizes generated by the above methods. However, the goal was to represent the effects as percent change. The standard equation for converting a logged difference in a count or rate into a percentage change is

% c h a n g e = 100 * ( e log g e d d i f f e r e n c e − 1 ) .

% c h a n g e = 100 * ( e − log g e d d i f f e r e n c e − 1 ) .

3.4.3 Moderator analysis

We had proposed conducting moderator analyses that examined the pattern of evidence for different types of curfew policies, if possible. However, the number of studies with meaningfully different policies was too small to allow for any such analyses.

3.4.4 Assessment of risk of bias

The assessment of risk of bias in the study designs eligible for this study was challenging (see Hingson et al., 2001; Hartling et al., 2004). The typical quality assessment tools used in Cochrane and Campbell systematic reviews are designed for non-ecological studies that are implemented at the individual level or other units that can be assigned randomly to conditions (e.g., small geographic regions such as crime hot spots). The potential sources of bias that we were most concerned about across these studies and tried to assess were:

Historical artifacts: anything that might be confounded with the intervention, such as other community or police initiatives directed at reducing crime.

3.4.5 Unit of analysis issues

Multiple reports or manuscripts based on the same study or data were treated as a single entity. We selected the most complete reference as the primary document for coding. Other documents were used only if they provided unique information of relevance to this review. Section 7.1 lists all references used during the coding process. No meta-analytic synthesis included two effects from the same study sample.

3.4.6 Assessment of reporting biases

The number of effect sizes for each outcome of interest was too low for formal assessment of publication selection bias such as through the use of the Duval and Tweedie (2000) trim and fill method. However, we did search for and found unpublished studies.

3.4.7 Data synthesis

Random-effects inverse-variance weighted meta-analysis was used to synthesize effect sizes across studies. Analyses were performed on the difference in the logged counts or rates. The method-of-moments estimator of the random effects variance component,τ², was used. Final results were converted into percent change, as discussed in Section 3.4.2. These analyses were performed using the metafor package in R, Version 1.9-5 (Viechtbauer, 2010). All R code used in the analyses and for producing the forest plots is presented in Section 12.

4.1.1 Results of the systematic search

The search strategy yielded 7,349 references. Removal of duplicates reduced this to 6,499. A pre-eligibility screening of titles and abstracts identified 100 references potentially relevant to this review, including review articles or other pieces with relevant background information, such as legal discussions. Two coders examined these 100 documents in detail for eligibility. Fifteen documents representing 12 unique studies satisfied our eligibility criteria. As is often the case, several studies that were initially coded as eligible were later recoded as ineligible upon closer inspection. These studies and others that were near misses on eligibility are listed in Section 7.2.

Of the 85 references coded an ineligible, 42 were excluded as simply not relevant. That is, the publication did not examine any aspect of a juvenile curfew ordinance. Thirty studies were categorized as background articles, including review articles, editorials, or descriptions of curfew policies. Ten studies examined some aspect of a curfew other than its impact on juvenile crime or victimization, such as the differences in the characteristics of curfew violators versus non-violators or parental attitudes towards curfews. Finally, two were excluded because of insufficient information to compute an effect size and one had data collected prior to 1959.

4.1.2 Description of included studies

Tables 1 and 2 provide descriptive information on the included studies. As shown in Table 1, most of these studies were published in journal articles (9 of 12). Two were published as technical reports and one was a Master's thesis. The publication dates ranged from 1999 through to 2012; 10 of the 12 studies being published between 1999 and 2003, inclusively. All of these studies were conducted in the United States. These studies were based on data collected between 1980 and 2004, inclusively, with only two studies (Kline 2012 and McDowall et al. 2000) using data prior to 1990.

Most of the studies examined a nocturnal curfew, as shown in Table 2, although two studies examined a daytime curfew during school hours. The age range affected by the curfew was typically 17 and under although a few only affected youth 16 and under or 15 and under. It is interesting to note that some curfew ordinances exclude children under the ages of 11 or 12. Presumably a curfew is unneeded for such young children given that police would intervene if they found an unaccompanied child less than 11 years of age out in public regardless of the existence of a curfew ordinance (at least within the United States).

Seven of the twelve studies used some form of interrupted time series design (see Table 2). These designs involved comparing the crime rate for some number of months or years prior to the curfew implementation to some number of months or years after the curfew implementation. These have been differentiated in Table 2 as either “ITS” or “Short ITS”. The latter are interrupted time-series designs with fewer than 10 total observational units (i.e., months or years). This distinction affected how effect sizes were estimated. The latter are also less able to effectively control for any time trend unrelated to the curfew.

It is worth noting that Cole (2003) used an ABAB interrupted time series design. This study used data from Washington, DC, where the curfew law was ruled unconstitutional after having been in effect for a period of 13 months. However, the ruling was later reversed and the curfew was reinstated a little over two years later. Taking advantage of this ‘turning on-and-off’ of the curfew, Cole compared the juvenile arrest rate across four time periods that alternated from a non-curfew period to a curfew period.

Two studies (Moscovitz et al., 2000; Rodabough & Young, 2002) collected short time-series type data, but collapsed the data such that there was only a single pre-curfew and post-curfew rate. Moscovitz and colleagues compared emergency medical services transportation of juveniles who were victims of an assault during a 3-month period in the year before a curfew went into effect with the same period after a curfew went into effect. Rodabough and Young compared juvenile arrest and victimization data for two one-year time periods: one year before the curfew was implemented and one year after the curfew was implemented. In Table 2, these two studies are categorized as pre-post designs.

Finally, three studies (Gius, 2011; Kline, 2012; McDowall et al., 2000) used a regression-based design. Gius (2011) used 1997 data from the National Longitudinal Survey of Youth (NLSY) that included the state and county of residence for each youth. This allowed for an examination of the effect of living in an area with a curfew based on self-reported criminal behavior of youth and, as such, reflects a cross-sectional analysis.

The Kline (2012) and McDowall et al. (2000) papers made use of city level data over several years (panel data). Essentially, these are multiple interrupted time-series analyzed in a single regression model. Kline (2012) analyzed a sample of 54 cities (with a 1990 population greater than 180,000 that began enforcing a curfew law during the years from 1980 through 2004. The regression models used logged arrest rates from the FBI's Uniform Crime Report (UCR). The regression model used an ‘event-study’ design to estimate the impact of curfew enactment. Kline (2012) claims that this model also accounts for any confounders related to a city enacting a curfew in response to an increase in arrests and any general time trends.

The McDowall et al. (2000) study used a similar design to Kline (2012). McDowall and colleagues included 11 years of FBI UCR arrest data for youths 17 and younger, and 19 years of homicide victimization data. Two sets of analyses were performed, one on 52 counties and the other on 12 city-counties. The former included counties where a change in the curfew law may have affected only a portion of the county. The latter included only the subset of cities or counties for which any change in a curfew law affected the entire city or county. We used only the latter as it is a more direct test of the effect of implementing a curfew, although the results for both sets of analyses lead to the same conclusion. As with the Kline (2012) study, McDowall et al. (2000) used the logged arrest rate as the dependent variable.

4.3.1 Effect on crime

A primary purpose of a curfew is to reduce the number of crimes committed by juveniles. The pattern of evidence across these studies suggests that curfews do not accomplish this goal. The strongest evidence comes from the effects adjusted for a time trend for the outcomes of juvenile crime during curfew hours and during all hours (see Figures 1 and 2). Two studies provided effect sizes for the effects of curfews on juvenile crime during curfew hours. Both studies found a small increase in crime with an overall mean percent change of 9.5% (95% confidence interval of -9.1% to 24.9%, p = 0.292). This mean effect is not statistically significant and has a fairly large confidence interval. Eight studies provided estimates of the percent change in juvenile crime during all hours (curfew and non-curfew). The overall mean was very close to zero (0.5% change in juvenile arrests with a 95% confidence interval of -9.8% to 8.1%, p = 0.915) with five studies reporting small increases in crime, one a near zero effect, and two a decrease in crime. One of these effects (Fivella, 2000) was substantial, representing a 46% decrease in juvenile arrests across both curfew and non-curfew hours. This is a clear outlier. Given that a relatively small proportion of juvenile crime occurs during curfew hours, this large effect is only possible if crime dropped substantially during both curfew and non-curfew hours. Removing this effect from the meta-analysis, however, produces roughly the same result, although with substantially less heterogeneity (see Table 5).

OPEN IN VIEWER

1

Juvenile crime during curfew hours – adjusted for time trend

OPEN IN VIEWER

2

Juvenile crime during all hours – adjusted for time trend

The two effects from Gius (2011) are based on individual level data and, as such, could not meaningfully be combined with effects from other studies. The logged odds ratios were converted to percent change in odds to facilitate comparison with the other results. No mean effect size was computed given that these two effects are from the same study. Counter-intuitively, the effect for self-reported arrests was lower than the effect for self-reported criminal behavior (whether resulting in an arrest or not) (see Figure 3). It is not clear why a curfew ordinance would produce a decrease in arrests and not in criminal behavior more generally.

OPEN IN VIEWER

3

Juvenile crime during all hours – individual level data

Figure 4 provides a single estimate from Rodabough and Young (2002) for juvenile arrests during curfew hours. This effect, while in the desired direction and statistically significant, is based on simple pre-post data and does not account for any underlying crime drop over time. Thus, it is at a high risk of bias.

OPEN IN VIEWER

4

Juvenile crime during curfew hours – not adjusted for time trend

Two studies (Kline, 2012; Fivella, 2000) provided estimates of the effects of juvenile curfews on adult crime during all hours (curfew and non-curfew). These effects are a type of control outcome. There is no theoretical reason to expect a juvenile curfew to have an effect on adult rates of crime. Both of these effects are negative (a decrease in crime) with an overall mean percent change of -9% (see Figure 5, 95% confidence interval of -25.3% to 5.3%, p = 0.230). While this effect is not statistically significant, it represents a larger effect than any of the mean effects for juvenile crime rates.

OPEN IN VIEWER

5

Adult crime during all hours – adjusted for time trend

Thus, any observed changes in juvenile crime of comparable magnitude might easily be the result of some process other than the curfew that is affecting all crime, whether committed by juveniles or adults.

4.3.2 Effects on victimization

Another goal of juvenile curfews is to reduce juvenile victimization. As with crime, the evidence does not support the effectiveness of curfews in accomplishing this. Figures 6 through 8 show these effects. Across these Figures there is no clear pattern of positive or negative change in victimization.

OPEN IN VIEWER

6

Juvenile victimization during all hours – adjusted for time trend

Figure 6 shows two effects and the mean effect for juvenile victimization during all hours adjusted for a time trend. One effect reflects a positive change and the other a small negative change. The mean effect is close to zero and not statistically significant (a 0.6% decrease in victimization with a 95% confidence interval of -21.7% to 16.9%, p = .953). These two studies provide the more credible estimates of curfew effects on victimization given that they are adjusted for any time trend.

The effects shown in Figure 7 are from the two studies that used a simple pre-post design. Three effects are from Moscovitz et al. (2000), one for juvenile victimization during curfew hours, another for during non-curfew hours, and one for all hours. The Rodabough and Young (2002) study provided a single effect of juvenile victimization during all hours. Because three of these four effects are from the same study, a mean effect was not computed (this is also true for the effects shown in Figure 8). The pattern of effects in Figure 7 is consistent with the more credible effects shown in Figure 6.

OPEN IN VIEWER

7

Juvenile victimization – not adjusted for time trend

OPEN IN VIEWER

8

Adult victimization – not adjusted for time trend

Finally, Figure 8 shows adult victimization effects. All three of these effects are from Moscovitz et al. (2000). We would not expect a juvenile curfew to have an effect on adult victimization. Thus, these effects help establish natural variation in victimization and are of comparable magnitude to the effects observed for juveniles, further supporting to the conclusion that curfews are ineffective at reducing victimization.