“Scared Straight” and Other Juvenile Awareness Programs for Preventing Juvenile Delinquency

1.1 REVIEW POLICY BRIEF - NOVEMBER 28, 2003

Does taking juveniles on tours of prison deter them from future crime and delinquency?

1.2 EMERGENT POLICY CONCERNS

On August 18, 2003, Illinois Governor Rod Blagojevich signed a bill into law that mandates the Chicago Public School system set up a program called “Choices” (United Press International 2003). The program would identify students at risk for committing future crime and set up a program to give them “tours of state prison” to discourage any future criminal conduct (Long and Chase, 2003). As the news article makes clear, policymakers had good reasons for passing the law. Parents of young children were desperate to find ways to deter their kids from a life of crime. With some youth (even at ages 11 and 12) getting involved early in gangs, there was mounting pressure on policymakers to intervene early in their lives to dissuade them from potentially more serious behavior. The Governor himself is quoted as saying that the law is intended to “give some kids a chance to see what happens if they don't follow the rules, follow the law, and what's ahead for them if they don't do that” (Long and Chase, 2003, p. 1).

This is only the surface of the discussion that was briskly reported in the Chicago Tribune article. There was certainly some opposition to it, and some of this criticism reflected upon prior research about similar programs. Indeed, one opponent said that the prison tours were an attempt to resurrect “Scared Straight” type programs, which had been found to be ineffective in curbing delinquency (Long and Chase, 2003, p. 1). Is this true? Or is Illinois government right on target by introducing this law?

Answering such a question requires an examination of the scientific evidence. In this Campbell update, we report the results of a systematic review of the nine randomized experiments of Scared Straight and other prison tour programs (also referred to as juvenile awareness or prison awareness programs). Of course, prior research is no guarantee that interventions will work (or not work) in a future setting. But a reader might ask herself the following question upon reading the results of this systematic review: would I want a doctor to prescribe a treatment for my child that has the same track record of research results?

4.1 CRITERIA FOR CONSIDERING STUDIES FOR THIS REVIEW

4.2 SEARCH STRATEGY FOR IDENTIFICATION OF STUDIES

In order to minimize potential for publication bias (the possibility journals are more likely to publish findings that reject the null hypothesis and find programs to be more effective than unpublished literature generally does), we conducted a search strategy designed to identify published and unpublished studies. We also conducted a comprehensive search strategy to minimize potential for discipline bias, i.e. evaluations reported in criminological journals or indexed in field-specific abstracting data bases might differ from those reported in psychological, sociological, social service, public health or educational sources.

First, randomized experiments were identified from a larger review of field trials in crime reduction conducted by the first author (Petrosino 1997). Petrosino had used the following methods to find more than 300 randomized experiments: (1) hand search (i.e. visually inspecting the entire contents) of 29 leading criminology or social science journals; (2) checking the citations reported in the ‘Registry of Randomized Experiments in Criminal Sanctions’ (Weisburd 1990); (3) detailed electronic searches of Criminal Justice Abstracts, Sociological Abstracts and Social Development and Planning Abstracts (Sociofile), Education Resource Information Clearinghouse (ERIC) and Psychological Abstracts (PsycINFO); (4) searches by information specialists of 18 bibliographic databases, including the National Criminal Justice Reference Service (NCJRS); (5) an extensive mail campaign with over 200 researchers and 100 research centers; (6) published solicitations in association newsletters; (7) tracking of references in over 50 relevant systematic reviews and literature syntheses; and (8) tracking of references in relevant bibliographies, books, articles, and other documents. More detail about these search methods can be found in Petrosino (Petrosino 1995, Petrosino 1997). The citations found in Petrosino (Petrosino 1997) covered literature with a publication date between January 1,1945 and December 31,1993. Seven randomized trials meeting the eligibility criteria were identified from this sample.

Second, we augmented this work with searches designed to uncover experiments missed by Petrosino (Petrosino 1997) and to cover more recent literature (1994–2001). These methods included: (1) broad searches of the Campbell Collaboration Social, Psychological, Educational & Criminological Trials Register (C2-SPECTR) developed by the U.K. Cochrane Centre and now supervised by the University of Pennsylvania Graduate School of Education (Petrosino et al 2000); (2) check of citations from more recent systematic or traditional reviews to provide coverage of more recent studies (e.g. Sherman et al 1997; Lipsey and Wilson 1998); (3) citation checking of documents relevant to ‘Scared Straight’ and like programs (e.g. Finckenauer and Gavin 1999); (4) email correspondence with investigators; and (5) broad searches of the Cochrane Controlled Trials Register [CENTRAL] in the Cochrane Library (Issue 1, 2002). By broad searches, we mean that we tried to first identify studies relevant to crime or delinquency and then visually scanned the citations or abstracts to see if any were relevant to this intervention. For example, we used words like ‘crime,’ ‘justice,’ ‘law,’ ‘offender,’ ‘delinquency,’ and so on to identify a large potential pool of studies and then went through these to determine if any were relevant to this review.

Third, we decided to conduct a more specific search of 14 available electronic databases relevant to the topic area. Many of these include published and unpublished literature (e.g. dissertations or government reports). Searches were done online using available Harvard University resources or other databases freely searchable via the Internet. Several trips were made to the University of Massachusetts, Lowell to use Criminal Justice Abstracts and other Silver Platter data bases not accessible at Harvard University or via the Internet. The bibliographic data bases and the years searched were:

Criminal Justice Abstracts, 1968- September 2001

In addition, the Cochrane Developmental, Psychosocial and Learning Disorders Group (the editorial group that handled the inaugural review and is responsible for the simultaneous Cochrane review) conducted searches of its specialized trial register. This trial register is broad, covers a wide range of bibliographic databases, and is international in scope. Details on searches used by the CDPLDG to build and maintain its register are provided in the Cochrane Library (2003, Issue 4).

We anticipated that the amount of literature on ‘Scared Straight’ would be of moderate size - and that our best course of action would be to identify all citations relevant to the program and screen them for potential leads to eligible studies. This removed the need to include keywords for identifying randomized trials (e.g. ‘random assignment’) in our searches. After several trial runs, we found that nearly all documents used phrases like ‘Scared Straight’ or ‘juvenile awareness’ in the title or abstract of the citation. Therefore, the following searches were run in each relevant database to identify relevant citations:

‘scared straight’

5.1 SELECTION OF TRIALS

The search methods (excluding the Internet searches, which generated thousands of websites) generated over 500 citations, most of which had abstracts. Anthony Petrosino (AP) initially screened these citations, determining that 30 were evaluation reports. AP and Carolyn Turpin-Petrosino (CTP) independently examined these citations and agreed that 11 were potential randomized trials. All reports were obtained. Upon inspection of the full text reports, we excluded two studies. One study was excluded because it did not include any post-program measure of offending. This was an evaluation of the ‘Project Aware’ program conducted in a Wisconsin prison (Dean 1982). Attempts to contact the author or retrieve these data from any other reports by the Wisconsin Department of Corrections have been unsuccessful. A second study of ‘Stay Straight,’ conducted in Hawaii, was also excluded due to the absence of random assignment (Chesney-Lind 1981). After the two exclusions, we were left with nine randomized trials. We did not find any ongoing trials.

5.2 ASSESSMENT OF METHODOLOGICAL QUALITY

There are many factors in which to grade the quality of studies. Complicating any assessment of methods is that review teams, by and large, must rely on written reports by investigators. In some cases, methodology sections may be briskly written (sometimes due to journal space requirements) and key features of design and analysis may be deleted or considerably condensed. We determined that four were most critical to criminological experiments and practical to extract from the experimental reports. These were: 1)

Randomization integrity

Did the investigators report that randomization of participants to experimental or control conditions experience serious violation or subversion of random assignment procedures?

5.3 DATA MANAGEMENT AND EXTRACTION

AP extracted data from each of the nine main study reports using a specially designed instrument. The data collection instrument was adapted from Petrosino's earlier study (Petrosino 1997); some items are listed in the ‘Table of Included Studies.’ In cases in which outcome information was missing from the original reports, we made attempts via email and regular mail correspondence to retrieve the data for the analysis from the original investigators. Investigators were helpful but unable to locate additional data. In two cases we retrieved unpublished Masters’ theses from University Libraries to see if they contained this information (Cook 1990; Locke 1984). They did not.

5.4 DATA SYNTHESIS

We ran statistical analyses using Cochrane Collaboration's MetaView statistical software, a component of Review Manager Version 4.1 (RevMan). We expressed dichotomous outcome measures of crime as Odds Ratios (OR) and continuous measures of crime as weighted mean differences (WMD). We reported the 95% Confidence Intervals (CI) for both. Both fixed and random effects models were assumed in weighting treatment effects across the randomized trials. We planned to examine these effects at follow-up intervals of 0–6 months, 7–12 months, 13–18 months, 19–24 months, and beyond 2 years. As we explain later, it was only possible to conduct meta-analysis for “first effect” only.

These were repeated, and additional analyses run, using Meta Analyst software created by Dr. Joseph Lau of the New England Cochrane Center. One of us (John Buehler) also created meta-analytic formulae in Excel to double-check three of the analyses. Results were identical.

6.1 NARRATIVE FINDINGS FROM THE REPORTS

Whether relying on the actual data reported or measures of statistical significance, the nine trials do not yield evidence for the effectiveness of ‘Scared Straight’ and other juvenile awareness programs on subsequent delinquency.’

6.2 SHOULD WE BELIEVE THESE STUDIES? ASSESSMENT OF METHODOLOGICAL QUALITY

We examined each study to determine the quality of methods on the four criteria aforementioned. We found three studies with reported methodological problems that should be taken into account, with two having implications for our statistical analysis: 1)

Randomization Integrity

One study reported problems with randomization, and they were dramatic (Finckenauer 1982). Only eight of the eleven participating agencies that referred troubled or delinquent boys to the program correctly assigned their cases. He did conduct additional analyses in an attempt to compensate for violation of randomization; the program still had harmful effects. We conducted sensitivity analyses, i.e. dropped this study from the meta-analysis to determine its impact on the results.

6.3 META-ANALYSIS

For each study, we extracted all of the relevant crime outcome data. Our protocol included an organization of analyses by examining official reports (from government administrative records) distinct from self-reported criminality (obtained from investigator-administered survey questionnaires). Given that we expected a diverse number of measures of crime to be reported, the protocol called for us to organize it into four indexes that would be most relevant to policy and practice: These included prevalence rates (i.e. what percentage of each group re-offended or did not?), average incidence rates (i.e. what was the average number of offenses or other incidents per individual in each group?), offense severity rates (i.e. what was the average severity of offenses per individual in each group?), and latency (i.e. how long was the average return to crime or failure delayed per individual in each group?). Unfortunately, as Table 2 shows, the full array of data showed that many of these indicators were missing.

Given the limitation of the data, we conduct one meta-analysis. We report the crime outcomes for official measures at the first-effect or first follow-up interval (and usually the only) period reported. Each analysis focused on proportion data (i.e. the proportion of each group re-offending), as the outcomes reporting means or averages is sparse and often does not include the standard deviations. Thus, because the data rely on dichotomous outcomes, both analyses report Odds Ratios (OR) for each study and their 95% Confidence Intervals (CI). Because there is some disagreement in the literature about this, we assume both random and fixed effects models for treatment effects across the studies.

The analysis of the data in comparison table 1 from the seven studies reporting reoffending rates shows that intervention increases the crime or delinquency outcomes at the first follow-up period. Assuming either a fixed effect or random effects model does not change its overall negative impact. Using a fixed effect model, the OR is 1.68 (CI 1.20–2.36); the mean OR assuming a random effects model is not much different at 1.72 (CI 1.132.62). These are both statistically significant. The intervention increases the odds of offending by between 1.6 and 1.7 to 1.

OPEN IN VIEWER

Figures 1 and 2

Immediate post-treatment effects for re-offending rates, official measures

Random Effects Model Results

n: Number of participants re-offending; N: Number assigned to Group; OR: Odds Ratio; CI: Confidence Intervals; Weight: Amount of weight given to study in analysis

OPEN IN VIEWER

2

Sensitivity analysis, excluding Finckenauer study

We excluded the Finckenauer study, because of its randomization problems, from the analyses reported in Figure 1 and 2. Given the little difference in OR whether assuming a fixed effect or random effects model, we conducted a meta-analysis assuming a random effects model. As the Finckenauer study reported the largest negative effects for the program, it is not surprising that the OR decreases. It is, however, still negative in direction at 1.47, and statistically significant (CI 1.03–2.11).

OPEN IN VIEWER

3

Sensitivity analysis, excluding Yarborough study

OPEN IN VIEWER

4

Sensitivity analysis, excluding Finckenauer and Yarborough studies

We excluded the Yarborough study because of its deletion of no-shows, indicating a potential for large attrition from the initial study sample. We again assumed a random effects model. The deletion of this study does not alter the overall negative impact of these programs, as the OR is 1.96. This is statistically significant (CI 1.25, 3.08).

studies

We excluded both the Finckenauer and Yarborough studies to see how this affected the overall meta-analysis. As the Figure shows, even with two studies removed for sensitivity analysis, the overall effect of the intervention in the five remaining studies shows a “criminogenic” effect, i.e., favors the control group not receiving Scared Straight.

7.1 THE ‘WHY’ QUESTION

One question that continues to arise about these findings is ‘why’ ‘Scared Straight’ and similar programs seem to lead to more crime rather than less in its participants. What is the critical mechanism? Understanding why something works or fails is of great interest to evaluators, program designers, and criminological theorists. Evaluators for the Oklahoma ‘Speak Outs’ program wondered about the criminogenic effect of these programs when they asked:

If one argued that a two hour visit cannot perform the miracle of deterring socially unacceptable behavior (see Cook & Spirrison, 1992), it can also be argued that it was extremely simplistic to assert that a two hour visit can perform the miracle of causing socially unacceptable behavior (Holley and Brewster 1996).

Although there were many good post-hoc theories about why these programs had negative effects, the evaluations were not structured to provide the kind of mediating variables or ‘causal models’ necessary for an empirical response to this question in a systematic review (Petrosino 2000).

8.1 IMPLICATIONS FOR PRACTICE

We note the following irony: despite the gloomy findings reported here and elsewhere, ‘Scared Straight’ and its derivatives continue in use, although a randomized trial has not been reported since 1992. As Finckenauer and Gavin (Finckenauer 1999) noted, when the negative results from the California SQUIRES study came out, the response was to end the evaluation - not the programmed. Today the SQUIRES program continues, evaluated by the testimonials of prisoners and participants alike. Despite evidence, beliefs in the program's efficacy continue. Middleton and his colleagues report on the extension of this strategy in one UK town to scare ordinary schoolchildren by using former correctional officers to set up a prison-type atmosphere in the public school system (Middleton 2001). In 1982, Finckenauer called this the ‘Panacea Phenomenon,’ describing how policy-makers, practitioners, media reporters and others sometimes latch onto quick, short-term and inexpensive cures to solve difficult social problems (Finckenauer 1982). Others claim that the program by itself is of little value but could be instrumental if embedded in an overall multi-component package of interventions delivered to youths. We believe that our review places the onus on every jurisdiction to show how their current or proposed program is different than the ones studied here. Given that, they should then put in place rigorous evaluation to ensure that no harm is caused by the intervention.

Some literature indicates the program can have a positive effect on the inmate providers and that argument is sometimes used to legitimize use of the programmed. These arguments are undoubtedly used under the assumption that the program does no harm. In light of these findings, assertions that ‘Scared Straight’ and similar programs ought to be used because it achieves other things raises ethical questions about hurting children in order to accomplish other important, but latent, goals.

Personal repentance and redemption is a noble goal and correctional facilities are not the only institutions that encourage this. Perhaps administrators can encourage motivated offenders in penal institutions to engage in benevolent activities that do not pose harm to juveniles (or the communities they offend in). There are many charitable activities that inmates do take part in (i.e. making toys for hospitalized children), and these should be encouraged, not only for the ‘good’ they bring, but also for the part that contributing to such a task may have in offender rehabilitation.

Another interesting opportunity for inmates is to serve as counselors and tutors to each other. For example, Franklin (2000) described a program at a Washington state correctional facility that used more educated inmates as literacy tutors for incarcerated offenders who could not read. The ability to contribute in such a way is cited as one of the positive factors of the program, along with its low cost.

The authors have received communications from different prison facilities that are using a juvenile awareness program. One argument used to sustain such programs is that the research reported here does not apply to their particular program. Our recommendation is that correctional research units, either at the facility or at a regional or national government level, collaborate with program staff to conduct a rigorous evaluation. If such units do not exist or cannot conduct their own study, we suggest they collaborate with a local university, college, or research firm that could undertake this work to ensure that the program is working as planned and not unintentionally causing more harm than good.

Correctional administrators sometimes ask whether our results are relevant to their particular program. For example, inmates running the program may go outside the prison to speak at schools about their life experiences. Our review only looked at “kids visit prisons” programs, and as far as we know, no review has examined juvenile awareness interventions that involve offenders leaving prison grounds to speak to children at school. To date, we have not found a single controlled study testing it.

Since versions of this review began circulating on the Internet, the first author receives periodic correspondence from a concerned citizen about how to get a juvenile in trouble with the law into a Scared Straight program. They are obviously not reading the full report but are just trying to find contact information about the program. We cannot in good conscience recommend this program. We have no data on the type of kid or constellation of personality characteristics that could possibly be helped by going through Scared Straight or a similar program. Our response to these well-meaning citizens is to refer them to national, regional or local centers that specialize in youth prevention services.

8.2 IMPLICATIONS FOR RESEARCH

In concert with Campbell and Cochrane guidelines, we plan to update this review again within 24 months to incorporate any new studies or respond to cogent criticisms. Given that we found only nine studies (and only seven used in the meta-analysis), we were cautious not to propose the use of moderating variables in subsequent analyses. We initially believed that one program factor that may have salience, however, is the degree of harshness in the inmate presentations. It may be that the more brutal and vulgar the presentation, the more that it causes a type of ‘backfire’ effect, producing in the juveniles the very behavior it seeks to deter. When looking at this a bit more closely, we discovered that one trial involving a tour of a reformatory with no presentation reported one of the largest negative effects (Michigan Department of Correction 1967). Until more experiments of juvenile awareness programs are reported - with adequate description of this variable - we will not pursue this particular point of inquiry.

This review has led us to consider two others, contingent on future funding. “Shock value” type interventions are tried across many fields. For example, high school students are sometimes shown horrific footage of car accidents in order to deter them from drinking and driving. In industrial arts classes, students are shown films of what occurs when safety glasses are not worn; this is often graphic and is designed to increase compliance with such regulations. There are many other examples across fields. But is there any evidence that any of these “shock value” interventions work? Or do they produce disappointing, or even toxic results, as we have reported here?

It may be true that Scared Straight and like programs do not work because they only convey a threat that juveniles do not think will be carried out. What about the evidence for deterrence when an actual official punishment is delivered? There have been a wide range of randomized trials that test for the effects of official processing in juvenile courts with some other intervention (such as diverting the kid from such processing). Is there evidence that the delivery of a threat - official system processing - deters future criminal behavior?

The type of broad search we undertook could allow for a more in-depth examination of the effects of using different evaluative designs to test criminological interventions. We now have a good collection of empirical studies on ‘Scared Straight’, and these include non-randomized studies. We have foregone our own plans to compare results across the different designs, but others may wish to take this up in the future.

9.1 WHAT'S NEW IN THIS UPDATE

In this update of a Campbell Inaugural Review, the authors:

Included a ‘C2 Research Review Policy Brief

9.2 CITATION/USES OF THIS RESEARCH

9.3 ACKNOWLEDGEMENTS

This work was principally supported by $5,000 (US) from a grant from the Smith-Richardson Foundation to the Graduate School of Education at the University of Pennsylvania (Robert Boruch, Principal Investigator). It received partial support from a Mellon Foundation grant to the Center for Evaluation, Initiatives for Children Program at the American Academy of Arts & Sciences (Frederick Mosteller, Principal Investigator) and a grant from the U.K. Home Office to Cambridge University Institute of Criminology (David Farrington, Principal Investigator). The latter two sources supported Anthony Petrosino's time during work and use of his office and computer at the American Academy of Arts & Sciences.

The update was supported in part by a Smith-Richardson Foundation grant to the Jerry Lee Center of Criminology at the University of Pennsylvania (Lawrence Sherman, Principal Investigator).

The Criminal Justice Collection at Rutgers University's Center for Law and Justice and the Gutman Library at the Harvard Graduate School of Education facilitated interlibrary loan requests. We thank Phyllis Schultze and Carla Lillvik for their expertise, patience and assistance.

We appreciate the great work of the Cochrane Developmental, Psychosocial, and Learning Disorders Group. This includes the guidance and editorial comments of Dr. Jane Dennis, Review Group Coordinator and Professor Geraldine Macdonald, Coordinating Editor. It also includes editorial comments from Dr. Julian Higgins, Dr. Stuart Logan and other editorial members of the Developmental, Psychosocial and Learning Disorders Group. Additional searches, among other helpful tasks, were provided done by Jo Abbott and Celia Almeida.

This paper benefited greatly from comments and criticisms by Professor Robert Boruch, Sir Iain Chalmers, Dr. Phoebe Cottingham, Professor Lyn Feder, and Professor Joan McCord also helped.

Professor Boruch served as the Principal Advisor on the Campbell-version of this review. We also appreciate the criticisms by three external peer reviewers during that Campbell editorial process.

The update was greatly improved by the editorial criticism of Professor Hiroshi Tsutomi.

9.4 POTENTIAL CONFLICT OF INTEREST

Two of the authors published an article in the journal Crime & Delinquency that indicated a harmful effect for these programs, based on preliminary analyses (Petrosino 2000b). We believe the potential for bias toward ‘replicating’ the earlier Crime & Delinquency findings here is reduced by the explicit and transparent review methods used.

9.5 SOURCES OF SUPPORT

9.6 PUBLISHED [OR FORTHCOMING] PAPERS BASED ON THIS RESEARCH

Petrosino Anthony, Carolyn Turpin-Petrosino, James O. Finckenauer. 2000. Well-meaning programs can have harmful effects! Lessons from experiments in Scared Straight and other like programs. Crime & Delinquency 46: 354–379.

Petrosino A, Turpin-Petrosino C, Buehler J. “Scared Straight” and other juvenile awareness programs for preventing juvenile delinquency (Cochrane Review). In: The Cochrane Library, Issue 4, 2003. Chichester, UK: John Wiley & Sons, Ltd.

Petrosino, Anthony, Carolyn Turpin-Petrosino and John Buehler. 2003. The effects of Scared Straight and other juvenile awareness programs on juvenile delinquency: A systematic review of the randomized experimental evidence. Annals of the American Academy of Political and Social Science (September):

Petrosino, Anthony, Carolyn Turpin-Petrosino and John Buehler (forthcoming). “Scared Straight and Juvenile Awareness Programs.” In Brandon Welsh and David Farrington (eds.) What Works? Wadsworth Publishing.

Synopsis: January 2003, Diane Mueller, Evidence-based Nursing