Abstract
2. Background
Justice practitioners have tremendous discretion on how to handle low-level juvenile offenders. Low-level juvenile offenders are those that commit offenses that are of moderate or low severity, e.g., small property crimes, disorderly persons violations. Police officers, district attorneys, juvenile court intake officers, juvenile and family court judges, and other officials can decide whether the juvenile should be “officially processed” by the juvenile justice system, diverted from the system to counseling or services, or released altogether. An
Given the juvenile justice system's dual goal of protecting public safety while rehabilitating juvenile offenders, it is not surprising that a strong argument for traditional processing can be made. For example, some officials believe that low-level offenses are a “gateway” to more serious offending, and should be dealt with intensively to prevent the juvenile from becoming a repeat offender. Some officials believe that official system processing and subsequent handling by the juvenile court will deter or “scare” low-level offenders from future misconduct. Some officials also believe that the primary role of the juvenile (or sometimes family) court is to rehabilitate the child, and therefore believe it necessary that even low-level offenders would benefit from treatment and services.
On the other hand, there are those who believe in a “minimalist” position: that low-level offenders should be handled in as non-intrusive a manner as possible. Researchers have warned of a possible “labeling” effect that may come from official processing of juveniles. For example, a petition results in an official label of the child as a delinquent, and significant others around the child will now begin to treat him or her differently. Such a juvenile may receive increased police scrutiny and end up getting rearrested more often than juveniles who are not under the same surveillance. The same actions that resulted in police turning a blind eye to misconduct may now result in an arrest. Labeling is theorized to have other potential impacts, including economic or educational losses, and marginalization by significant others such as family and friends. There are other theories, apart from labeling, that could explain why further processing in the juvenile system may increase crime. For example, such processing could further expose youth to more deviant peers, resulting in a criminogenic effect (e.g., Dishion, et al., 1999).
For low-level juvenile offenders, the question is whether it is better to process the child through juvenile court, or to handle the child through informal means? To find out whether a policy alternative “works”, we have to examine the scientific evidence on the question. What do prior assessments, or evaluations, of the outcome of this decision tell us? Does it support handling juvenile offenders formally or informally?
Fortunately, there have been randomized experiments in the juvenile courts that can be gathered together in a systematic fashion to provide rigorous evidence about the impact of this decision on subsequent offending by juveniles. Since the 1960s, a series of randomized experiments have been done in the juvenile courts to test the efficacy of programs that diverted juveniles from official processing into more informal strategies. These experiments for the most part tested diversion programs that included counseling or other services. The control or comparison condition in most of these experiments has been the “traditional system processing” or petition condition. By turning the experiment around, and treating traditional system processing as our “treatment” or “intervention” condition, and the diversion, release, and other conditions as the control, we now have the perfect opportunity to test for the impact of traditional court and further “penetration” on juvenile delinquents.
Despite the fact that there have been a fair number of randomized controlled studies that included traditional system processing as a condition, there has not been an attempt to systematically gather only this experimental evidence and analyze it to determine what the crime deterrent impact is for traditional system processing on low-level juvenile offenders. This Campbell Collaboration review is proposed to address this gap in knowledge.
Besides the policy importance of the proposed review, it also provides a further test of deterrence theory, thereby adding to our knowledge of criminological theory. In our earlier Campbell review (which was also published by the Cochrane Collaboration), we systematically analyzed the experimental evidence for “Scared Straight” and similar programs (Petrosino, Turpin-Petrosino, and Buehler, 2003a; 2003b). In such programs, juvenile delinquents or troubled kids are taken on tours of prisons or reformatories. In most of these programs, the kids are also involved in a confrontational interaction with adult inmates, who try to scare the youths by telling them about the horrors of prison. More recent iterations of this program are less confrontational and more educational, and adult inmates try to warn kids about the dangers of continuing on their current path of delinquency and anti-social behavior.
We found that Scared Straight type programs did not deter its participants from future delinquency, but actually backfired or had a toxic effect. In other words, children participating in the juvenile awareness program did worse than juveniles who did not. What is remarkable is that a meta-analysis of nearly 400 experimental or well-controlled quasi-experimental evaluations of preventative or treatment interventions for juvenile delinquency showed that nearly two-thirds (64%) were positive in direction (Lipsey, 1992). Our review showed that all seven experiments with sufficient outcome data were all negative in direction at first follow-up. The Scared Straight Campbell/Cochrane review underscored that a presumably beneficial intervention can go against conventional wisdom and best intentions and have a negative impact on the very juveniles and citizens that policymakers and practitioners desire to help. It also underscored that Scared Straight did not deter or scare kids at all; in fact, it seemed to encourage more offending in the experimental trials we uncovered.
We wondered why, contrary to popular perceptions, Scared Straight would actually result in juvenile offenders doing worse than kids who did not attend the program. After all, deterrence is a major goal of the criminal justice system. There is a great belief among officials and the general citizenry that the threat of punishment will keep people in-line and restrain them from criminal behavior. But was Scared Straight really a test of deterrence? In a sense, the program was an “implied threat” of punishment that could result should the youth continue in delinquent behavior. But maybe one reason why Scared Straight does not work and instead results in a harmful effect (according to the experiments) is that the kids exposed to the program did not take the warnings seriously. After all, the messengers in Scared Straight were not justice officials, but, for the most part, adult prisoners locked up behind bars for many years. Maybe the juveniles participating in the program recognized that there is a big difference between a warning delivered by inmates with no authority to arrest, prosecute and punish - and the actual delivery of a sanction or punishment by the criminal or juvenile justice system.
But what is the result when such an official threat is made? What happens when juveniles are officially processed by the juvenile justice system? Does such a sanction meet the goal of specific or special deterrence? Under specific or special deterrence, the offender who is punished will refrain from future criminal acts; this is distinguished from general deterrence theory, in which the threat of punishment restrains citizens (who are not yet punished) from breaking the law. If specific deterrence does operate with juvenile offenders, surely officially processing them through the juvenile justice system would reduce subsequent offending when compared to less harsh options such as diversion or counsel and release?
3. Objectives
For this project, we will be collecting studies that respond to the question: Does juvenile system processing reduce subsequent delinquency?
4. Methodology
Criteria for inclusion and exclusion of studies in the review
For this project, we will only include those studies that have the following characteristics:
Used random or quasi-random assignment. Because a well-implemented randomized experiment is the only design that produces statistically unbiased estimates of impact for program or policy, our review only includes evaluations that involves the random assignment of juvenile delinquent to traditional system processing or to a different condition such as “release,” “counsel and release,” “diversion,” or “diversion with services.” Studies that use ‘quasi-random’ methods for assignment, such as alternation (or assigning every other case to treatment), will also be included.
Randomly assigned juvenile delinquents (ages 17 and younger) who have not yet been “officially adjudicated” for their current offense to traditional system processing -- or to an alternative non-system condition. Traditional system processing includes any condition in which the juvenile offender is assigned to a condition that involves official processing by the juvenile justice system. Such conditions have been described in prior experiments as “juvenile system processing” (Dunford, et al., 1982), “traditional handling by the juvenile court” (Baron and Feeney, 1976), “traditional processing” (Severy and Whitaker, 1982), “regular petition and processing by the juvenile court” (Klein, 1986). The control conditions in studies gathered by this review will include, but are not limited to, such alternatives as diversion, counseling and release, and outright release. Because the system processing condition is usually the control group in the experiments, it is often not described further. Nonetheless, the category does provide a contrast between further official justice system contacts versus non-system alternatives. Note that juveniles may be included in an experiment although they have a prior record (and may have even been adjudicated for a prior offense). This review will include those experiments that randomly assign juveniles to traditional system or non-system conditions. We will code (discussed later) whether the juvenile had a prior record.
Included at least one outcome measure of criminal behavior. We will collect outcomes of crime, regardless of whether they are measured by official records, self-report, victim report, or other measures. The interest of policymakers, practitioners, and ordinary citizens is whether traditional system processing has a crime reduction effect. Other measures, such as impact on costs, attitudes or satisfaction levels will also be collected provided the study includes at least one measure of crime.
The study report is published or available through July 2008, without regard to language. We will also search for trials published up to and including July 2008, without regard for the start date of publication. However, we anticipate that most experiments in this area will have been published after 1960. In concert with Campbell principles, we will attempt to find English and non-English studies. Unfortunately, most randomized experiments in justice are carried out in the U.S., Great Britain, and Canada and reported in English.
Example of studies that would be included in our review
A large experiment that would meet the study inclusion criteria was conducted in Memphis (Severy and Whitaker, 1982). More than 2,000 Memphis-area youth were assigned to ‘penetrate’ the traditional system, to ‘services,’ or to a control group ‘without services.’ No one approach was found to be superior to the other. This experiment would be included because the “penetrate the traditional system” is juvenile system processing condition.
Because our review includes experiments in which juvenile system processing is compared to non-system alternative, it will include trials that randomize youthful offenders at every stage. For example, there may be experiments in which youths who came to the attention of police (police contact stage) were randomly assigned to be formally booked or to a release condition. Thus, although they did not appear in a juvenile court, they avoided further juvenile system processing.
Example of studies that would not be included in our review
One experimental study that would not be included in our review was conducted in California (Smith et al., 1979). In the experiment, over 500 juvenile offenders were randomly assigned to three treatment programs: counseling and release, in-house diversion counseling, and referral to community release. Recidivism rates of the groups were determined by checking arrest records at 6 and 12 months. The re-arrest rates of youths assigned to alternative treatment strategies did not differ significantly. We would not have included this study because it did not randomly assign youth to a juvenile system condition. All of the other program conditions, however, meet the relevant criteria.
Search strategy for identification of relevant studies
Our review builds upon earlier work by Weisburd, Sherman and Petrosino (1990) and Petrosino (Petrosino, 1995a, 1995b, 1997, 1998) that identified a large number of randomized experiments in criminal justice. For example, Petrosino (1997) conducted electronic searches of bibliographic databases (e.g., Criminal Justice Abstracts); did visual hand searching of 29 leading social science journals; made personal contact with reviewers and experimental researchers; published solicitations for reports in association newsletters; and chased down citations from existing reviews and experimental literature. Despite the narrow eligibility criteria, several hundred trials were identified; retrieval methods ended after the first 300 trials were obtained. In that collection alone, which only covers experiments published or available through 1993, there were more than ten experiments that would meet the criteria for this review.
To augment the ten trials in our existing data file, we will rely on two strategies (that have been most productive in prior projects) to identify relevant trials published between 1994–2008. These are:
Electronic searches of bibliographic databases. Researchers will use available online resources and databases at Bridgewater State College, WestEd, the University of Pennsylvania, Boston Public Library, Boston University Medical Library and the University of Massachusetts, Lowell. The databases that are currently available to us and will be searched are listed as Appendix A.
Existing reviews. There have been many prior reviews of offender treatment, delinquency prevention, experiments, and other relevant literature, particularly since 1993. We will search through the bibliographies of these reviews of research for references to potential experiments meeting our criteria. These reviews include but are not limited to the following: the University of Maryland Report to the Congress on Crime Prevention (Sherman et al 1997); the review of experiments in violent behavior by the Cochrane Collaboration's Schizophrenia Group (Cure et al 2005), the ongoing meta-analyses of Mark Lipsey (e.g., 1992) on juvenile delinquency treatment and prevention at the Center for Evaluation Research and Methodology at Vanderbilt and the recent review of experiments by Farrington and Welsh (2005).
As noted in the eligibility criteria, we are not exclusively seeking English reports. We will ask our colleagues from Spain, Germany and other nations for help in identifying any non-English studies. Colleagues or students at our host institutions (e.g., WestEd and Bridgewater State College each has employees bilingual in Spanish, French, Japanese, and Chinese) will translate abstracts or full-text documents in non-English to determine their eligibility for this review.
Keyword strategies for bibliographic databases
The databases in Appendix A can be somewhat idiosyncratic. We believe the best strategy is to conduct a broad search of the available databases that errs on the side of sensitivity rather than specificity. In other words, we would rather get many titles and abstracts to sift through rather than potentially miss relevant citations because our search terms were drawn narrowly.
We will use two different search strategies, depending on the focus of the bibliographic database. If the database is focused on criminal justice (such as Criminal Justice Abstracts or NCJRS), we will combine two types of keywords. First, we will use the following keywords (and their derivatives) to find outcome studies: “random,” “experiment,” “control,” “evaluate,” “trial,” “impact,” “effect,” and “outcome.” Second, we will combine those keywords with ones that focus the search on youth: “juvenile,” “youth,” “child,” “adolescent,” “student,” “delinquent,” “boy,” “girl,” “teen,” “young,” and “minor.” We know this strategy will produce a number of false positives, but our experience is that examining the abstracts is not time consuming and researchers can go through them quite quickly.
If the database is not focused on criminal justice (e.g., ERIC or Medline), the above strategy must then be supplemented by either a classification code or a third set of keywords that identifies criminological literature. In some databases, a classification code exists; for example, in Sociological Abstracts (Sociofile), one can limit the abstracts to those dealing with “criminology” or “penology”. But in many of others, there is no classification code. In those instances, we will use keywords (and their derivatives) related to crime and delinquency, for example: “crime,” “sanctions,” “law,” “legal,” “justice,” “adjudicate,” “sentence,” “convict,” “divert,” “court,” “police,” “corrections,” “anti-social,” “aggression,” “violent,” offense,” “illegal,” “truant,” “misconduct,” “judge,” and “magistrate.”
This keyword search strategy has worked very well in past reviews of this type. For example, Petrosino (1997) used this keyword search strategy to help identify 300 randomized controlled trials of “individually focused” crime reduction programs. Nonetheless, the appendices to our final review report will carefully document all keywords used for each database to permit replication.
Retrieving and Final Screening of Studies
Search methods will identify a large number of citations and abstracts. Many of these will be easily excluded as not being relevant to the proposed review. In some cases, however, they will identify potentially eligible studies. The full text documents of those potentially eligible studies will be retrieved and screened by two of the authors before the study can be formally included in the review. Fortunately, with the advent of the Internet, full-text electronic journal access, and Bridgewater State College's Interlibrary Loan capacity, we will be able to rapidly retrieve the reports to do a more thorough reading. When a full text report is received, we will scan it to ensure that it includes randomization of juvenile offenders and includes at least one outcome of criminality. If both Investigators do not agree on the inclusion of a particular study, it will be excluded and documented in the final report.
Extracting Information from Each Study
Informed by our prior research (Petrosino, 1997; Petrosino, Turpin-Petrosino and Buehler, 2003a), we have designed a preliminary instrument to guide us in recording information from each study (see Appendix B). Although the instrument contains several open-ended items, these will be collapsed when appropriate into a smaller number of categories to permit further analysis. For example, items such as “how randomization was performed” can be collapsed into three or four larger categories representing the most frequent responses (e.g., coin toss, computer generated) and an “other” response that captures all those responses that do not fit into the three or four most common methods of randomization in this set of studies.
The instrument has items in the following areas:
Researcher and Study Characteristics:
Study reports can be used to provide information about the publication and characteristics about the experiment. For example, we will extract data about the type of publication the study was reported in and the setting in which the trial was conducted.
Study Methods and Methodological Quality:
We will extract information about the randomization and other methodological aspects of the trials. It is especially critical that information about two key issues in the implementation of a randomized field experiment in social policy be extracted from each study report:
Whether the researchers reported that randomization was subverted by practitioners or was not fully implemented, resulting in less confidence that the groups did not remain fully balanced on all known and unknown factors.
Whether the researchers report a loss of participants from the initial randomly assigned sample at the end of the study. Such attrition, if it is significant, can undermine the ability of randomization to produce balanced groups, particularly if different types of people drop out from the intervention than dropped out from the other conditions.
Treatment and Control Conditions data:
These items will solicit detailed descriptions of the treatment and control condition, and the number of participants assigned to each. Although there is usually only one treatment group in our sample of studies (the juvenile system processing condition is usually represented just once in an experiment), these same studies often have several alternatives to the processing condition. Therefore, we will also detail our rationale for selecting the control group when there are other alternatives (release, diversion, diversion with counseling, etc.). Our standard principle is to select the least intrusive or least harsh condition, i.e., diversion over diversion with services. Our rationale is that we are looking for the control condition that presents the “strongest contrast” with the juvenile system processing condition. If special deterrence applies to these juveniles, then a contrast between juvenile court processing and release (the harshest versus least harshest disposition) would be the ideal comparison to test that theory. On the other hand, if labeling applies, the same comparison of juvenile court processing and release presents the best test of that theory.
Participants in the Trial data:
These items solicit detail about the type of participants in the trials, including information on race, gender, prior record, and current offense.
Outcome data:
For each eligible study (remember that eligible studies have at minimum one outcome measure of crime), we will extract information on reported outcomes including crime, costs, educational measures, and psychological results. Crime outcomes will be organized into four main indices that we will report:
Prevalence: What percentage of each group failed or succeeded?
Incidence: What was the average number of offenses or other incidents per group?
Severity: What was the average severity of offenses committed by each group? Or what percentage of persons in each group later committed crimes against the person?
Time to Event, Time to Failure or Latency: How long was return to crime or failure delayed for each group?
There will also be other outcomes reported that do not fit into the four crime outcome categories. For these data, we will include an “Other” category to capture them (e.g., continuous score on a self-report delinquency measure). We will also record any subgroup effects. Non-crime measures (such as education, psychological, attitudinal) will also be collected if reported, to shed light on potential positive or negative consequences for juvenile system processing. It will also help us identify clues as to why the juvenile system processing condition reduced crime or did not.
Description of methods used in the component studies
All of the studies in our review will have randomly assigned low-level juvenile offenders to alternative conditions. The number of conditions conventionally will be two, but in several instances, assignment will be made to three or more alternatives. In a few cases, it is possible that the authors will have used quasi-random procedures such as alternation to assign cases. Studies will vary considerably in size, but many will be underpowered to detect small to moderate treatment effects. This underscores the import of this review, particularly if quantitative meta-analysis can be conducted. All of the included studies will have measured the outcomes of assigned participants on subsequent offending, though the indices and types of data used will vary.
Handling multiple reports on the same experiment
Note that investigators may publish several articles on the same study. Our unit of analysis is the individual experiment and not the individual research article, and so it is reasonable to extract information from all documents to complete the coding instrument for one experiment. When reports on the same experiment contain conflicting information, we will employ a number of strategies, including contacting the original investigator(s) for resolution or determining how other meta-analyses or reviews of juvenile programs (e.g., Lipsey, 1992) coded the information.
Criteria for determination of independent findings
If there are different types of data used to report on prevalence at 0–3 months (e.g., police data, petitions), we will select the outcome that represents the earliest point of contact in the juvenile justice system (i.e., usually police contact).
If there are enough outcome data, we will conduct the analysis for each of the cells separately. Thus, each study will only be represented once in any quantitative analysis. As occurred in the Scared Straight Campbell review (Petrosino et al., 2003a), most studies had only a single outcome, and we ended up simply analyzing the impact of the program on “first reported post-treatment effect.”
Details of study coding categories
To ensure that we achieve good coding reliability, we will have two co-authors read and record information from all reports. This should not be difficult given that our anticipated yield is likely going to be less than 20 studies. We will assess coding reliability (i.e., interrater agreement) by using the percentage of agreement for each item, rather than reporting a global inter-rater reliability statistic. This will avoid inflating reliability measures with study characteristics that generally achieve perfect agreement (e.g. year of publication) with those that do not. Items with lower rates of agreement (less than 80%) will be investigated to determine the source for conflict. The Co-Investigators will meet to resolve disagreements and discussing coded items. We will drop those items from our database in which resolution could not be reached, as well as items that lack clear interpretation.
Statistical procedures and conventions
The data will be entered into the Cochrane Collaboration's specialized free review manager software (RevMan). The Investigators used this software for their earlier Scared Straight review. RevMan is packaged together with Meta-View, a statistical program for meta-analysis. We will use Meta-View to quantitatively combine results from the experiment.
For each study, we will describe whether there is any increase or decrease in outcome measures, and the direction of that effect. Given that the most common type of outcome is prevalence rates, usually expressed dichotomously such as failure or success (e.g., percentage of each group recidivating or being rearrested), we will use odds ratios as our effect size. A forest plot will be used to display the results from the odds ratios. Our first analysis, therefore, will be an overall display of the odds ratios—using the success/failure rates from the studies—in the form of a forest plot.
But, we also want to account for the full array of outcome data (incidence, severity and latency) that are reported in these juvenile justice trials. Unfortunately, these types of outcomes are usually reported in a number of ways. Sometimes original study authors report only the test statistic. Or they provide a significance level, but no other information. In such instances, Cohen's d is a very flexible measure that can be used. We will compute Cohen's d for all outcomes, using the transformation formulae provided in Lipsey and Wilson (2001). So, our second analysis will include Cohen's d for all outcome measures, organized by the earlier table of outcome data. This was a strategy we used in our Scared Straight review (see Petrosino et al. 2003b).
We will assume random effects models, which tends to be more conservative than the fixed effects approach, in weighting treatment effects across the randomized trials to compare outcomes. Because crime outcomes are often reported at various time intervals, we will analyze data distinctively for the following time intervals: 0–6 months, 7–12 months, 13–18 months, 19–24 months, and beyond 2 years. Given the paucity of follow-up data in juvenile justice trials, it is likely that we will report on the “first-effect” only.
Given that we do not expect to find a large number of experimental studies, moderator analyses are not anticipated. A small number of total studies could lead us to reject a potentially important moderator because of insufficient statistical power. Nonetheless, we will qualitatively examine whether the results vary depending on the type of control condition (in this review, the nature and quality of the non-system alternative, such as diversion or release) or the type of experimental condition (in this review, the nature and quality of the further processing condition, including when it occurs in the system process). It is also true that juveniles assigned to the diversion condition have, in fact, experienced a type of sanction and may also have more onerous conditions to comply with than juveniles assigned to the traditional system group. We will conduct analysis to determine if the type of control condition has any impact on the meta-analytic results.
Another important moderator is the length of follow-up period. Meta-analyses generally show, across different fields, that treatment effects decay over time. There is also the possibility, however, that some of the processes in juvenile system processing, such as labeling, may occur after some time period has passed. We will examine the influence of “follow-up length” by looking at how the effect size varies given incremental increases in follow-up length (0–3 months, 4–6 months, 7–9 months, 10–12 months, etc.)
Treatment of qualitative research
Our scan of the first 10 reports of randomized experiments evidenced no qualitative data to include in our review. However, we will use such data, if reported, to provide information on context. We will do this as we describe the findings in each study in narrative fashion first, before we describe our quantitative analyses and results.
5. Timeframe
6. Plans for updating the review
We plan to update this review in 36 months, in concert with C2 guidelines.
7. Acknowledgments
We would like to thank the Norwegian Knowledge Centre for the Health Sciences (the C2 Secretariat) for partial support of this project. Additional support was provided by an organization that wishes to remain unnamed.
8. Statement concerning conflict of interest
We do not have any conflicts of interest regarding juvenile system processing or diversion programs. None of the authors has any financial interest in the results of this review, regardless of how they turn out.
Footnotes
Appendix A
C2 Review: Juvenile Justice System Processing CODING INSTRUMENT
Sarah Guckenburg Carolyn Turpin-Petrosino Anthony Petrosino Other ______
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________