Validating the Level of Service Inventory-Revised

Abstract

This study addresses the validity of the Level of Service Inventory-Revised (LSI-R) as a predictor of recidivism for women. We analyzed all female parolees that were released in New Jersey in 2006 (n = 450) as well as a randomly selected group of male parolees. Results indicate that the LSI-R was a valid predictor of rearrests, reconvictions, and technical parole violations, but that the instrument was problematic in accurately predicting true positives both across genders as well as outcome types. It is likely that this lack of specificity is due to local-level policies regarding the use of the instrument.

Keywords

corrections parole reentry recidivism Level of Service Inventory-Revised females gender

It is a well known maxim that the United States has recently experienced an unparalleled growth in its prison populations and that this growth has resulted in strains on community corrections agencies that are charged with supervising reintegrating offenders upon their release. According to the Bureau of Justice Statistics’ most recent data, there were over 1.6 million prisoners housed in state and federal correctional facilities as of June 2009 (West, 2010) and over 5 million offenders were under some form of community supervision at the end of 2008 (Glaze & Bonczar, 2009). In 2008, the Pew Center on the States communicated this fascination with mass incarceration in a very humanistic way, finding that approximately 1 out of every 100 American adults was serving time behind bars (Pew Center on the States, 2008). A follow-up report found that when considering those under the supervision of community corrections including probation and parole, 1 out of every 31 American adults is under some form of criminal justice supervision (Pew Center on the States, 2009).

An attendant result of the prison boom has been a renewed scholarly interest in the study of corrections, community corrections, and rehabilitation. Scholars engaged in exploring this area of research have aided corrections agencies in developing new tools and innovative strategies in an attempt to increase positive outcomes for the formerly incarcerated. One of the key findings from this area of study has been that the risk of recidivism can be mitigated by corrections agencies through guiding rehabilitative efforts according to actuarially assessed risk (Lowenkamp & Latessa, 2004; Lowenkamp, Latessa, & Holsinger, 2006). Overall, it has been found that better outcomes can be achieved by gearing correctional programs toward offenders who have higher assessed risk in conjunction with their criminogenic needs (Dowden & Andrews, 1999; Lowenkamp & Latessa, 2005).

The current study investigates the predictive validity of a risk assessment instrument, the Level of Service Inventory-Revised (LSI-R), on a cohort of inmates released from prison in New Jersey in 2006. In an effort to add to the debate about the predictive validity of this instrument for female offenders, all females that were released to parole in this year who had an LSI-R (n = 450) were evaluated in addition to an equivalently sized randomly selected sample of male parolees. The instrument’s predictive abilities are evaluated for three measures of recidivism: (a) an arrest for a new crime; (b) a conviction stemming from a new arrest; and (c) a technical violation of parole. Unlike other studies of the LSI-R, in addition to traditional validity statistics (e.g., area under the curve and correlation matrixes), this study presents results from Receiver Operating Characteristic (ROC) curve coordinates. These analyses communicate the rate of true versus false positive identification that the instrument achieved at a given score cutoff.

The Level of Service Inventory-Revised

The LSI-R is one of community corrections’ most used and empirically tested assessment instruments (Lowenkamp, Lovins, & Latessa, 2009; Vose, Cullen, & Smith, 2008). It has received significant empirical support for its ability to reliably predict recidivism across many jurisdictions for a variety of criminal justice populations, including inmates and offenders in community and residential settings (Gendreau, Little, Goggin, 1996; Holsinger, Lowenkamp, & Latessa, 2003). The LSI-R has been found to predict reincarcerations for drug-involved offenders, producing a significant moderate correlation (r = 0.25) (Kelly & Welsh, 2008). In a study of federal probationers, the instrument was found to be a valid and reliable tool in predicting future incarceration, even when age, sex, and ethnicity were controlled for in multivariate analyses (Flores, Lowenkamp, Smith, & Latessa, 2006). Further, the instrument sufficiently predicted future criminal activity in two separate studies of probationers, parolees, and inmates in the Iowa Department of Corrections (Lowenkamp & Bechtel, 2007; Vose, 2008). In a study of 22,533 offenders in Washington State, the LSI-R was found to predict rearrests and reconvictions moderately well during a 2-year follow-up period (Barnoski & Aos, 2003) and in New Jersey, the tool significantly increased prediction above that of chance for reconvictions among a community corrections sample (Schlager, 2005). Other validation studies have also been conducted in a number of U.S. states including Ohio, North Dakota, and Idaho with samples of probationers, parolees, and inmates with reports indicating similar results (Lowenkamp & Latessa, 2001; Lowenkamp & Latessa, 2002).

However, research has also been critical of the LSI-R in its ability to predict recidivism among certain criminal justice populations such as halfway house residents as well as subpopulations of the larger criminal justice system including racial minorities and women. A study of inmates in the Pennsylvania Department of Corrections found that only eight of the 54 items that make up the LSI-R were associated with various types of recidivism (e.g., arrests, detentions, absconding, and returns to prison) and made recommendations that the tool should only be used to identify supervision and service needs after a discretionary release decision had already been made (Austin et al., 2003). Similarly, a study of a sample of Colorado halfway house participants found a weak correlation between the LSI and rearrests during a 2-year follow-up period (r = .14). Within this study regression analyses demonstrated relatively small odds ratios (OR = 1.03) indicating that unit increases in LSI scores only marginally increased the odds of halfway house failure as well as rearrests (Dowdy, Lacy, & Unnithan, 2002).

Weak correlations between LSI-R scores and recidivism have also been found with minority populations. In a study of African American and Hispanic males, rearrest and reconviction outcomes were only weakly correlated with total LSI-R score (r = 0.06 and r = 0.09 respectively, Schlager & Simourd, 2007). Similarly weak correlations (r = 0.11) have been found for the prediction of rearrest with Native American offenders (Holsinger, Lowenkamp, & Latessa, 2006). In a study that included a large sample of racial minorities (n = 842), the LSI-R was found to be inconsistent in predicting rearrests across different racial groups. African Americans were significantly more likely to be overclassified as recidivists by the tool when compared to White and Hispanic offenders (Fass, Heilbrun, Dematteo, & Fretz, 2008). In the study of offenders in Washington State mentioned previously, the LSI-R was found to predict rearrests more accurately for men than for women (Barnoski & Aos, 2003). Support for the use of the LSI-R with community corrections populations and certain subpopulations has been mixed and continuing research can expand knowledge about the applicability of the LSI-R to these particular populations.

Gender and Risk Assessment

Risk assessment assumes that similar factors that support criminal behavior operate across gender groups and that parole is best conceptualized as a gender-neutral process in which risk factors for recidivism are the same for both men and women (Herrschaft, Veysey, Tubman-Carbone, & Christian, 2009). The deficit-based model of traditional parole also assumes a position of an external change agent and that offenders must be “fixed” through appropriate interventions (Andrews, 2000). However, a growing body of research is beginning to challenge this assumption, reinterpreting reentry as an individual process, motivated by personal characteristics and circumstances, where differences influencing pathways out of criminal behavior differ for men and women (Maruna, 2001; Salisbury & Van Voorhis, 2009). Yet, despite empirical and theoretical evidence that demonstrates that reentry is an individual process, and that men and women follow different pathways into and out of criminal behavior, parole agencies continue to use gender-neutral risk assessment tools.

Some research has found that traditional tools, such as the LSI-R, have a tendency to overclassify women as high risk, leading to potentially unnecessary intensive supervision and restrictive programming (Chesney-Lind, 1989; Fowler, 1993). Further, some studies suggest that these tools do not accurately identify the needs of female offenders because they are not informed by theories of female criminality (Chesney-Lind, 1989; Lowenkamp et al., 2001; Van Voorhis, Salisbury, Wright, & Bauman, 2008; Van Voorhis, Wright, Salisbury, & Bauman, 2010). Some researchers advocate that there are probably needs that are common for both male and female offenders that can be identified by the LSI-R, but that the LSI-R and other assessment tools cannot assess the level of relative importance and priority of these common needs as correlates for and pathways into criminal behavior (Hollin & Palmer, 2006). Understanding the context of offending behavior is important, since research has shown that the LSI-R has predictive accuracy for women who do not follow gendered pathways into crime, but, on the contrary, traditional assessment instruments such as the LSI-R have been found to be inaccurate for women whose offending is influenced by a gendered context (Holtfreter & Cupp, 2007; Reisig, Holtfreter, & Morash, 2006; Salisbury, Van Voorhis, & Spiropoulous, 2009). In fact, more recent research has shown that the strength of the predictive validity of traditional assessment tools, such as the LSI-R, can be magnified for both men and women when gender-responsive factors are added to the assessment tool (Hsu, Caputi, & Byrne, 2011; Van Voorhis, et al., 2010).

Earlier studies that test the predictive validity of the LSI-R have been conducted with samples of only male offenders (Bonta & Motiuk, 1985; Loza & Simourd, 1994), but some validation research has included samples of female offenders. In a study of parolees and probationers, similar correlations between composite LSI-R scores and reconvictions were obtained for both men (r = 0.14) and women (r = 0.11) in the sample (Vose et al., 2009). For serious, violent offenders, the LSI-R performed well in predicting convictions for new offenses (AUC = 0.77) for women during a 1-year follow-up period (Manchak, Skeem, Douglas, & Siranosian, 2009). In a meta-analysis of LSI-R validation studies that included a sub-sample of 14,737 women, effect sizes for men and women indicated similar predictive value for both genders (Smith et al., 2009). These studies, and others, have largely indicated that the LSI-R is equally predictive of recidivism for both male and female offenders (Flores, Lowenkamp, Smith, & Latessa, 2006; Folsom & Atkinson, 2007). However, these studies are constrained by several limitations and the results must be approached with some caution.

Many of the previous studies of the LSI-R’s predictive power for females have simply controlled for gender through multivariate modeling rather than testing the validity of the LSI-R on an exclusively female sample and comparing results to a similarly sized sample of male offenders (Belknap, 2007). Gendreau et al.’s (1996) meta-analytic work, which serves as one of this literature’s most cited studies, recognized that their research was limited by only controlling for gender and that results were not largely generalizable to the prediction of recidivism for women. In many of the studies included in the Smith et al., (2009) gender-specific meta-analysis, the primary focus was a general validation of the LSI-R on a large sample of offenders rather than a particular evaluation of the degree to which the instrument differentially predicted outcomes between genders (Flores, Lowenkamp, Smith, & Latessa, 2006; Lowenkamp & Latessa, 2001, 2002).

Studies that have included female-specific samples have used data on relatively small groups and have not included comparisons to a similarly situated sample of male offenders. For example, a study of 100 female offenders in Canada that were serving 2 or more years in custody found the LSI-R to predict new convictions (Folsom & Atkinson, 2007), but did not provide a comparison of predictive validity of the instrument to a sample of male offenders who were serving similar amounts of custodial time. These approaches provide limited generalizability to the female offender population. One of the only studies that has specifically examined the differences between male and female offender samples has focused solely on differences in raw scores and subscores and not on the overall differences of the predictive validity of the instrument between gender groups (Holsinger, Lowenkamp, & Latessa, 2003).

Data and Method

Prerelease and recidivism data for this study were collected from the New Jersey State Parole Board (SPB). The Board provided the researchers with a database that highlighted all inmates that were released from DOC custody in 2006. This database contained a total of 12,555 released inmates. Identifying information was not attached to cases in order to ensure the confidentiality of the research subjects. Release types included unconditional (n = 4,592) and parole releases (n = 7,963). Of the 7,963 inmates released to parole, 7,528 were discretionary releases and the remaining 435 were statutorily mandated to be supervised by the SPB. The researchers only considered discretionary parole releases for analysis within this study. The researchers were able to attach recidivism information to 6,306 of these discretionary parole releases. Of these remaining cases, 5,728 had a valid prerelease LSI-R score.

The 1,222 cases that the researchers were not able to attach recidivism data to are likely due to mismatches of unique case identifiers between the SPB’s release database and the data abstract from which the agency pulls recidivism information. This data abstract is ultimately maintained by the NJ State Police. The 578 cases without LSI-R data are likely due to agency-level complications/oversight inherent with manual data entry. In order to provide insight into the validity of the LSI-R to predict recidivism of reintegrating women, the researchers selected all females from the final group of discretionary parolees who were released in 2006 who were able to have recidivism information attached to their case and who had an LSI-R score (n = 450). An identically sized sample of men was randomly selected from this remaining pool of 5,278 offenders for a total sample size of 900.

This study uses 3 years of postrelease recidivism data. Recidivism is defined in three different ways: (a) an arrest for a new crime; (b) a conviction that stemmed from a new arrest; or (c) a technical parole violation. We present descriptive prerelease and recidivism statistics for each gender and for each risk group according to prerelease scores on the LSI-R. Two risk category paradigms are considered: (a) risk categories presented by Andrews (1982) during the LSI-R’s original validation on a Canadian sample and (b) risk categories presented by Schlager (2005) from a NJ specific normalization study. The validity of the LSI-R was explored through the use of correlation and ROC curve analysis. Composite LSI-R scores and postrelease recidivism data were entered into a correlation matrix separately for the entire sample, for men only, and for women only. ROC curves were similarly constructed. The ROC analysis makes use of the area under the curve (AUC) statistic to communicate the LSI-R’s ability to predict postrelease failure. This statistic communicates the performance of a diagnostic test, with a larger area corresponding to better performance (Westin, 2001). Unlike previous LSI-R validity studies, ROC curve coordinate results are presented. Curve coordinates communicate the rate of true positive versus false positive identification at a given score cutoff.

Results

Descriptive statistics for male and female parolees can be viewed in Table 1. Females were significantly older than males on their 2006 release date as well as on the date on which they were first arrested or convicted. Both males and females were mostly represented as being Black and single; however, the female sample of parolees significantly differed from their male counterparts according to ethnic makeup. More females were White (34.9%) when compared to males (17.0%), and more males were recorded as being Hispanic (16.1%) when compared to females (9.8%).

Table 1.

Descriptive Statistics by Gender.

	t / χ²	Male	Female
Age at release	***	33.84 (9.43)	37.35 (8.90)
Age at first arrest	***	21.83 (8.24)	26.64 (8.55)
Age at first conviction	**	22.45 (8.45)	26.32 (8.69)
Ethnicity (%)	***
Black		66.7	55.0
White		17.0	34.9
Hispanic		16.1	9.8
Asian		0.2	0.2
Marital status (%)
Single		79.0	76.8
Widowed		0.5	2.9
Married		10.4	9.9
Divorced		5.8	5.9
Separated		4.3	4.5
LSI-R score		22.72	22.93
Andrews’ LSI-R groups (%)
Low (0-13)		7.8	6.0
Low/Moderate (14-23)		47.1	46.4
Moderate (24-33)		42.2	44.7
Moderate/High (34-40)		2.9	2.9
High (41+)		0	0
Prior NJ normalized LSI-R groups (%)
Low (0-16)		16.2	13.1
Moderate (17-23)		38.7	39.3
Medium (24-30)		34.2	38.9
High (31+)		10.9	8.7
Number of instant offenses (%)
One		40.2	47.3
Two		26.4	28.7
Three		13.1	15.8
Four or more		20.2	8.2
Instant offense type (%)	***
Property		14.7	19.3
Drug		52.9	62.7
Sex (nonviolent)		0.2	0
Violent		27.1	13.8
Other		5.1	4.2
Number of in-prison programs participated		0.80 (1.47)	0.80 (1.29)
Number of in-prison programs completed		0.28 (0.74)	0.26 (0.73)
Number of in-prison disciplinary infractions		0.45 (0.87)	0.40 (0.79)
Sentence (years)	***	5.19 (3.73)	4.02 (2.38)
Time served (days)	***	806.09 (929.49)	513.11 (894.67)
Rearrested (%)	***	65.6	50.4
Rearrest type (%)	*
Property		13.9	18.9
Drug		45.1	50.7
Sex (nonviolent)		0.7	0
Violent		24.4	13.2
Other		15.9	17.2
Arrests prior to release	**	7.98 (5.64)	8.37 (8.18)
Arrests after release		1.69 (1.95)	1.45 (2.27)
Reconvicted (%)	**	50.2	41.1
Reconviction type (%)	**
Property		11.9	21.6
Drug		54.4	55.7
Sex (nonviolent)		0	0
Violent		15.5	7.0
Other		18.1	15.7
Convictions prior to release		6.47 (4.62)	6.48 (5.48)
Convictions after release		1.05 (1.49)	0.99 (2.27)
Technical parole violation after release (%)		35.1	30.9
Time imprisoned for postrelease technical parole violations (days)	***	102.02 (206.18)	79.83 (175.68)
N		450	450

Note: Standard deviations of means are presented in parentheses.

p ≤ .05. **p ≤ .01. ***p ≤ .001

After their 2006 release, a significantly higher proportion of males were both rearrested (65.6%) and reconvicted (50.2%) during the follow-up time when compared to females (50.4% and 41.1%, respectively). A significantly higher proportion of male recidivists were rearrested (24.4%) and reconvicted (15.5%) for violent crimes when compared to females (13.2% and 7.0%, respectively), but rearrests and reconvictions for drug crimes were the modal categories for both groups of recidivating parolees. While males and females did not differ according to the proportion that experienced a postrelease technical parole violation, males that did experience a violation spent a significantly greater amount of time in an incarcerated setting as a result of a violation.

Table 2 presents differences in recidivism outcomes between gender groups according to appropriate risk cutoffs. Cutoffs include both Andrews’ (1982) risk score cutoffs as well as the NJ normalized groups. No parolees in either of the gender groups were considered at “high risk” of recidivating according to Andrews’ (1982) cutoffs. Both males and females evidenced statistically significant differences in the proportion that were rearrested, reconvicted, or violated their parole according to both the Andrews (1982) and the NJ normalized LSI-R risk group cutoffs. Generally, as the assessed risk classification increased from lower risk bands to higher risk bands, the proportion of individuals that experienced a postrelease failure increased. However, in several of these analyses, the proportion of those who were classified as the “highest risk” that failed did not exceed the proportion of failures in the risk category immediately below the highest risk classification. For example, 76.3% of males within the Andrews (1982) moderate risk category and 76.0% of males within the NJ medium-risk category were rearrested compared to 69.2% of males in the Andrews moderate/high category and 75.5% of males in the NJ high-risk category. Neither male nor female recidivists evidenced significant proportional differences regarding the type of rearrest or reconviction that was experienced according to the LSI-R risk category.

Table 2.

Recidivism by Gender and LSI-R Groups.

	Male						Female
Andrews’ LSI-R groups	1	2	3	4	5	χ²	1	2	3	4	5	χ²
Rearrested (%)	40.0	59.9	76.3	69.2	—	***	14.8	46.4	58.7	61.5	—	***
Rearrest type (%)
Property	14.3	9.4	17.9	11.1	—		50.0	18.6	17.8	25.0	—
Drug	71.4	39.4	47.6	44.4	—		50.0	53.6	49.2	37.5	—
Sex (nonviolent)	0	0	1.4	0	—		0	0	0	0	—
Violent	7.1	33.1	17.9	33.3	—		0	9.3	16.1	25.0	—
Other	7.1	18.1	15.2	11.1	—		0	18.6	16.9	12.5	—
Reconvicted (%)	28.6	44.3	60.5	53.8	—	***	7.4	37.8	48.8	46.2	—	***
Reconviction type (%)
Property	10.0	8.5	14.8	14.3	—		50.0	21.5	21.4	16.7	—
Drug	70.0	52.1	53.0	85.7	—		0	58.2	57.1	16.7	—
Sex (nonviolent)	0	0	0	0	—		0	0	0	0	—
Violent	10.0	17.0	15.7	0	—		50.0	5.1	7.1	16.7	—
Other	10.0	22.3	16.5	0	—		0	15.2	14.3	50.0	—
Technical parole violation (%)	14.3	33.5	38.4	69.2		**	3.7	25.8	39.3	38.5	—	***
N	35	212	190	13	0		27	209	201	13	0
NJ normalized LSI-R groups	1	2	3	4	—	χ²	1	2	3	4	—	χ²
Rearrested (%)	37.0	65.5	76.0	75.5	—	***	27.1	48.0	54.3	79.5	—	***
Rearrest type (%)
Property	14.8	8.8	19.7	10.8	—		18.8	20.0	15.8	25.8	—
Drug	44.4	42.1	47.9	45.9	—		56.3	52.9	50.5	41.9	—
Sex (nonviolent)	0	0	1.7	0	—		0	0	0	0	—
Violent	29.6	30.7	14.5	32.4	—		12.5	8.2	16.8	16.1	—
Other	11.1	18.4	16.2	10.8	—		12.5	18.8	16.8	16.1	—
Reconvicted (%)	27.4	48.3	59.1	63.3	—	***	20.3	39.0	43.4	71.8	—	***
Reconviction type (%)
Property	15.0	7.1	18.7	3.2	—		25.0	21.7	18.4	28.6	—
Drug	45.0	56.0	50.5	67.7	—		58.3	56.5	59.2	42.9	—
Sex (nonviolent)	0	0	0	0	—		0	0	0	0	—
Violent	15.0	16.7	12.1	22.6	—	**	8.3	5.8	7.9	7.1	—	***
Other	25.0	20.2	18.7	6.5	—		8.3	15.9	14.5	21.4	—
Technical parole violation (%)	19.2	35.6	37.7	49.0	—		11.9	27.1	39.4	38.5	—
N	73	174	154	49	—		59	177	175	39	—

Note: Andrews’ LSI-R groups: 1 = Low (0-13). 2 = Low/Moderate (14-23). 3 = Moderate (24-33). 4 = Moderate/High (34-40). 5 = High (41+).

NJ normalized LSI-R groups: 1 = Low (0-16). 2 = Moderate (17-23). 3 = Medium (24-30). 4 = High (31+).

p ≤ .05. **p ≤ .01. ***p ≤ .001

Analyses of the LSI-R’s predictive abilities were accomplished through Pearson correlation and ROC analysis. Results from the correlation analysis can be viewed in Table 3. The composite LSI-R score exemplified statistically significant positive correlations with all recidivism events across all groups. However, the strength of the correlation was weak across all groups and all outcome types. In addition, r values did not exhibit marked variation either across the different groups or across the different measures of recidivism that were considered.

Table 3.

Correlations Between Recidivism and Composite LSI-R Score.

	Rearrest	Reconviction	Technical parole violation
All (n = 900)	.230**	.210**	.200**
Male (n = 450)	.240**	.212**	.203**
Female (n = 450)	.230**	.214**	.199**

Note: *p ≤ .05. **p ≤ .01. ***p ≤ .001.

Results from the ROC analyses are presented in two separate tables. Table 4 highlights results from AUC analyses and Table 5 presents results from curve coordinate analyses. The latter of these analyses highlights the relative tradeoffs between the true positives (sensitivity) and false positives (1-specificity) that were predicted at each LSI-R score cutoff when the instrument was used to predict a postrelease rearrest for a new crime for males, females, and combined gender groups.

Table 4.

Predicting Recidivism With the LSI-R: Receiver Operating Characteristic Area Under the Curve Results.

	Rearrest	Reconviction	Technical parole violation
All (n = 900)	.630 (.592–.667)***	.618 (.582–.655)***	.620 (.581–.658)***
Male (n = 450)	.646 (.591–.701)***	.624 (.572–.675)***	.619 (.565–.673)***
Female (n = 450)	.622 (.571–.674)***	.615 (.564–.667)***	.622 (.568–.675)***

Note: Asymptotic 95% confidence intervals are presented in parentheses

p ≤ .05. **p ≤ .01. ***p ≤ .00.

Table 5.

Predicting Rearrest With the LSI-R: ROC Curve Coordinates.

	All (N = 900)		Male (n = 450)		Female (n = 450)
LSI-R score cutoff	True positive rate	False positive rate	True positive rate	False positive rate	True positive rate	False positive rate
4.00	1.000	1.000	1.000	1.000	—	—
5.00	—	—	—	—	1.000	1.000
5.50	1.000	.997	1.00	.994	—	—
6.50	.998	.995	.997	.994	1.000	.996
7.50	.998	.989	.997	.987	1.000	.991
8.50	.996	.984	.993	.987	1.000	.982
9.50	.994	.974	.990	.981	1.000	.969
10.50	.992	.960	.986	.955	1.000	.964
11.50	.987	.931	.976	.923	1.000	.937
12.50	.977	.910	.966	.897	.991	.919
13.50	.966	.884	.953	.865	.982	.897
14.50	.952	.852	.939	.826	.969	.870
15.50	.933	.817	.922	.774	.947	.848
16.50	.918	.765	.908	.703	.930	.807
17.50	.885	.714	.875	.665	.899	.749
18.50	.852	.661	.834	.613	.877	.695
19.50	.807	.614	.790	.568	.828	.646
20.50	.751	.540	.736	.490	.771	.574
21.50	.684	.495	.685	.452	.683	.525
22.50	.609	.418	.597	.406	.626	.426
23.50	.536	.362	.522	.316	.555	.395
24.50	.452	.312	.447	.265	.458	.345
25.50	.400	.267	.390	.245	.414	.283
26.50	.337	.217	.339	.200	.335	.229
27.50	.262	.172	.268	.161	.256	.179
28.50	.195	.124	.190	.116	.203	.130
29.50	.157	.090	.153	.084	.163	.094
30.50	.130	.053	.125	.077	.137	.036
31.50	.088	.045	.081	.058	.097	.036
32.50	.065	.032	.064	.026	.066	.036
33.50	.033	.024	.031	.026	.035	.022
34.50	.011	.013	.014	.019	.009	.009
35.50	.008	.008	.010	.013	.004	.004
36.50	.002	.005	.000	.013	.004	.000
37.50	.000	.003	.000	.006	—	—
38.00	—	—	—	—	.000	.000
39.00	.000	.000	.000	.000	—	—

The AUC results presented in Table 4 show that the LSI-R made valid predictions that a reintegrating parolee would be rearrested, reconvicted, and/or have their parole technically violated. The LSI-R predicted recidivism outcomes significantly above chance both for the entire sample as well as for the separate gender groups. The AUC figure represents the chance that a randomly selected recidivist would have a higher LSI-R score than a randomly selected non-recidivist. For example, for females, the AUC for rearrest communicates that a randomly selected female parolee who was rearrested would have a 62.2% chance (AUC = .622) of having a higher LSI-R score than a randomly selected female parolee who was not rearrested. The AUC values range from .615 for the reconviction dependent variable amongst female parolees to .646 for the male parolees when predicting rearrest outcomes. Considering an AUC of .500 to be equivalent to chance (e.g., a coin flip), this range of AUC values communicates that the LSI-R increased predictive accuracy between a low of 11.5% above chance to a high of a 14.6% above chance. These values represent a low discriminating ability for the LSI-R to distinguish between recidivists and non-recidivists both for males and females as well as for combined groups.

Table 5 shows the LSI-R’s relative rates of benefits (true positives) versus costs (false positives) at each score cutoff when the instrument is used to predict a post-release rearrest for the entire sample, for males, and for females. The coordinates contained in Table 5 show that the LSI-R demonstrated substantial trade-offs between correctly identifying an individual that would be rearrested and identifying an individual that would be rearrested and being incorrect. These tradeoffs were evident at all of the LSI-R score cutoffs. For example, at the score of 23, which represents the upper cutoff point for low/moderate risk in the Andrews scale as well as the moderate risk level in the NJ normalized scale, 60.9% of those within the total sample that scored at or above a 23 that were rearrested would be correctly identified as recidivists, however, 41.8% of those at or above this cutoff who were not recidivists would be incorrectly identified.

Discussion

This study addresses the validity of the LSI-R as a predictive instrument for parolee criminal recidivism and noncriminal technical violations. All female parolees with LSI-R scores that were released in NJ in 2006 were assessed to provide insight into the instrument’s predictive ability for female recidivism. Generally, within both gender groups, higher risk LSI-R classification translated into higher proportions of each group experiencing a rearrest, reconviction, or technical parole violation. However, in the Andrews coding scheme, males that were classified as being the penultimate risk group experienced higher rates of rearrest and reconviction after release when compared to males in the highest risk group. This finding was also true for reconvictions and technical parole violations for females. Within the NJ normalized cutoffs, a higher proportion of males that were classified as being medium risk were rearrested when compared to those who were classified as high risk and a higher proportion of medium-risk females experienced a technical parole violation when compared to females who were classified as high risk. However, the absolute differences in proportions were not as pronounced as within the original Andrews cutoffs. The type of rearrest or reconviction that the individual experienced did not significantly differ across risk classifications for either gender according to either cutoff scheme.

Both of the measures that were used to assess the predictive ability of the LSI-R indicated that the instrument makes valid predictions, but both the Pearson correlation and the AUC analyses resulted in small overall effect sizes. Correlations indicated that the relationship between the composite LSI-R score and rates of rearrest, reconviction, and technical parole violations were positive indicating that as the score increases so does the likelihood that the individual will fail, but the relationship was very weak (ranging from .199-.240). Further, the largest AUC value that was obtained within this study (i.e., male rearrests) indicated that the LSI-R allowed for an increase in predictive accuracy of only 14.6% above that of a coin flip.

These concerns with the overall strength of the relationship between LSI-R scores and recidivism are further magnified upon analysis of true versus false positive rates at particular score cutoffs. Analyses of specific coordinates of the ROC curves indicated substantial tradeoffs between the benefits (i.e., identifying true positives) of the LSI-R respective to the costs of its administration (i.e., identifying false positives). Particular to the discussion of whether the LSI-R is an appropriate predictor of recidivism for women, these data are indicative that the instrument’s predictive ability was roughly equivalent for males and females across several outcome types. However, the ability of the instrument to differentiate between those who would or would not recidivate at various cut scores was found wanting. Taken as a whole, these results indicate that while the LSI-R was able to significantly improve the ability of an individual employed in the pursuit of predicting recidivism to make good decisions, the improvement was not very large above that of chance alone. Further, if solely relying on the LSI-R to make predictions, oftentimes he/she would be incorrect. But, these findings may be more indicative of local policies and practices that underlie the administration and use of the LSI-R than the overall utility of the instrument.

In this jurisdiction, parole board members are statutorily required to use an actuarial risk assessment instrument to inform their release decisions. When used to inform release decisions, the LSI-R is conducted by a privately contracted entity in the prison prior to an individual’s parole hearing. The results of the instrument are made available to board members in the inmate’s file. Prior research conducted in this context has found that these releasing authorities, while using the LSI-R, are skilled at identifying those who are and are not likely to fail while on parole (Ostermann, 2010) and that they are skilled at consuming risk-based information when targeting candidates for services as they transition into the community (Ostermann, 2009).

After the inmate is released, however, findings from the LSI-R are not made readily available to parole officers who are charged with managing the risks and needs of the individual parolee. In the field, the LSI-R retains an identical form as it did while the parolee was incarcerated: a piece of paper in a file. While parolee files are readily accessible in an officer’s district parole office, the results of the LSI-R are likely lost among hundreds of other pages of materials that are in these files, including presentence investigations, information on in-prison programming and disciplinary infractions, and victim input. Further, parole work in this setting has largely shifted to computerized data management. Parole officers track cases and record field notes through an information system; however, only the LSI-R’s composite score is retained in this centralized database. While the composite score communicates the parolee’s overall risk level, it does not speak to the substance behind how the risk level was accomplished or to the particular criminogenic needs that the parolee exhibits.

Problems with obtaining current information about a parolee’s relative risk are also significant in this jurisdiction. While officers are typically required to have a parolee’s risk assessment score updated once every 6 months, of the 15,918 offenders currently under the SPB’s supervision, 6,546 have LSI-R scores that are older than 6 months and 1,455 do not have any LSI-R score recorded in the agency’s centralized database. This large number of parolees without updated risk assessment information is likely a function of scores being completed by contracted community programs such as day reporting centers and halfway back programs (Ostermann, 2009) rather than the officers that are actually supervising the case.

The lack of ready access to complete and up-to-date LSI-R information paired with the lack of participation in the assessment process by parole officers sheds interesting light onto the overall findings of this study. While results indicated that the LSI-R was a valid predictor of recidivism, the predictive accuracy of the instrument was found to be lacking across multiple outcomes, irrespective of gender. While these findings were surprising, results such as these can be expected in accordance with the way the instrument is actually used in this setting. Board members use the instrument to place potential parolees into “likely to succeed” and “not likely to succeed” groups and to guide transitional service decisions, but the instrument is not used in its intended manner after the inmate is released to the Division of Parole’s supervision. In stark contrast to findings from the evidenced-based practices literature, results of the LSI-R are seemingly not used by officers to gear offenders towards rehabilitative resources. As evidenced by the findings of this research, this has resulted in significant trade-offs between identifying true and false positives.

Policy implications from this research suggest that paroling authorities in this jurisdiction should make concerted efforts toward improving their use of findings from actuarial risk assessment instruments in the field. Making parole officers actively involved in the assessment process and having those charged with supervising parolees in the community demonstrate how they have incorporated findings from assessment instruments into their case plans should be the paramount of these efforts. The officers that are managing the risks of those who are released from prison should be responsible for creating risk profiles and appropriate case plans for parolees rather than contracted third parties. This policy implication is especially salient given the costs associated with conducting actuarial risk assessment both in the community and behind the prison wall.

While the SPB spends over US$1 million annually to have prisoners’ mental health statuses evaluated by a contracted third-party in order to inform release decisions, the agency spends tens of millions of dollars per year to have parolees served within community programs. The LSI-R is a critical component of both the prerelease assessment as well as the community programming regimen that is built by relevant staff. The results of this research show a high false positive rate at high risk score cutoffs. If officers attempted to use results of the LSI-R to guide decision making in the field, this high rate of misclassification could result in having offenders receive programming that is not aligned with their actuarially based risks. This approach would both squander taxpayer dollars and, as prior research has shown, may result in iatrogenic treatment effects (Lowenkamp & Latessa, 2005).

Finally, the paroling authority in this jurisdiction should consider using a more appropriate field-centric risk assessment instrument for parole officers while retaining the LSI-R for board members making release decisions. Parole officers supervising parolees in the field would likely be better served if they were to use an instrument such as the Level of Service/Case Management Inventory (LS/CMI) rather than the LSI-R. The LS/CMI is a more appropriate tool for the management of individual cases during supervision because it serves as both a full-fledged case management tool in addition to a scientifically rigorous risk and needs assessment instrument. Compared to the LSI-R, this tool better reflects the dynamic status indicators and current conditions affecting the parolee (Andrews, Bonta, & Wormith, 2004).

In these economically harsh times, it is important for correctional agencies that are faced with limited resources to manage the populations that they are charged with supervising as effectively as possible. Paroling authorities are in a unique position to both aid in managing high-cost corrections systems through providing relief to burgeoning prison populations through the release function as well as attempting to improve overall outcomes through its community supervision function. While results from actuarial risk assessment instruments play an important role in both of these functions, it is the latter of these that must be greater concentrated upon through reform in policy and practice within the current setting. If substantial efforts are made to increase the utility of risk assessment in this jurisdiction, it is likely that the accuracy of the instrument’s ability to identify true positives will be increased and false positive identification will decrease. Consequently, overall positive outcomes for the reintegrating population will likely follow suit.

Footnotes

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

Author Biographies

Michael Ostermann is a research professor at the Rutgers University School of Criminal Justice where he serves as the director of the Evidence-Based Institute for Justice Policy Research. His recent work has been featured in Criminology and Public Policy, Criminal Justice and Behavior, Justice Research and Policy, The Journal of Offender Rehabilitation, Crime & Delinquency, and Justice Quarterly.

Bryn A. Herrschaft is the research data coordinator for HealthCore, a health services research organization, and a doctoral student at Rutgers University. Some of her most recent contributions can be found in The Journal of Offender Rehabilitation and the book, How Offenders Transform their Lives, published by Willan.

References

Andrews

(1982). The level of supervision inventory (lsi): The first follow-up. Ottawa, Ontario, Canada: Carleton University.

Andrews

D. A.

(2000). The Principles of effective correctional programs. In Motiuk

L. L.

Serin

R. C.

(Eds.), Compendium 2000 on effective correctional programming (vol. 1, pp. 9-17). Ottawa, Ontario, Canada: Correctional Service of Canada-Ministry of Supply and Services.

Andrews

D. A.

Bonta

Wormith

S. J.

(2004). The level of service/case management inventory (ls/cmi). Toronto, Ontario, Canada: Multi-Health Systems.

Austin

Coleman

Peyton

Johnson

(2003). Reliability and validity study of the lsi-r risk assessment instrument. Washington, DC: The Institute on Crime, Justice, and Corrections.

Barnoski

Aos

(2003). Washington’s offender accountability act: An analysis of the department of corrections’ risk assessment. Olympia, WA: Washington State Institute for Public Policy.

Belknap

(2007). The invisible woman: Gender, crime and justice (3rd ed.). Belmont, CA: Wadsworth.

Bonta

Motiuk

L.L.

(1985). Utilization of an interview-based classification instrument: A study of correctional halfway houses. Criminal Justice & Behavior, 12, 333-352.

Chesney-Lind

(1989). Girls’ crime and woman’s place: Toward a feminist model of female delinquency. Crime and Delinquency, 35, 5-29.

Dowden

Andrews

D. A.

(1999). What works for female offenders: A meta-analytic review. Crime & Delinquency, 45, 438-452.

10.

Dowdy

E. R.

Lacy

Unnithan

(2002). Correctional prediction and the level of supervision inventory. Journal of Criminal Justice, 30, 29-39.

11.

Fass

T. L.

Heilbrun

Dematteo

Fretz

(2008). The LSI-R and the COMPAS: Validation data on two risks-needs tools. Criminal Justice & Behavior, 35, 1095-1108.

12.

Folsom

Atkinson

J. L.

(2007). The generalizability of the LSI-R and CAT to the prediction of recidivism in female offenders. Criminal Justice & Behavior, 34, 1044-1056.

13.

Fowler

L. T.

(1993). What classification for women? In Classification: A tool for managing today’s offenders(pp. 37-45). Arlington, VA: American Correctional Association.

14.

Flores

A.W.

Lowenkamp

C. T.

Smith

Latessa

E. J.

(2006). Validating the Level of Service Inventory-Revised on a sample of federal probationers. Federal Probation, 70(2), 44-48.

15.

Gendreau

Little

Goggin

(1996). A meta-analysis of the predictors of adult offender recidivism: What works? Criminology, 34, 575-607.

16.

Glaze

L. E.

Bonczar

T. P.

(2009). Probation and parole in the United States 2008 (NJC 228230). Washington, DC: Bureau of Justice Statistics Bulletin.

17.

Herrschaft

B. A.

Veysey

B. M.

Tubman-Carbone

H. R.

Christian

(2009). Gender differences in the transformation narrative: Implications for revised reentry strategies for female offenders. Journal of Offender Rehabilitation, 48, 463-482.

18.

Hollin

C. R.

Palmer

E. J.

(2006). Criminogenic need and women offenders: A critique of the literature. Legal and Criminological Psychology, 11, 179-195.

19.

Holsinger

A. M.

Lowenkamp

C. T.

Latessa

E. J.

(2003). Ethnicity, gender, and the Level of Service Inventory-Revised. Journal of Criminal Justice, 31, 309-320.

20.

Holsinger

A. M.

Lowenkamp

C. T.

Latessa

E. J.

(2006). Exploring the validity of the Level of Service Inventory-Revised with Native American offenders. Journal of Criminal Justice, 34, 331-337.

21.

Holtfreter

Cupp

(2007). Gender and risk assessment: The empirical status of the LSI-R for women. Journal of Contemporary Criminal Justice, 23, 363-382.

22.

Hsu

Caputi

Byrne

M. K.

(2011). The Level of Service Inventory-Revised (LSI-R) & Australian offenders: Factor structure, sensitivity, and specificity. Criminal Justice & Behavior, 38, 600-618.

23.

Kelly

C. E.

Walsh

W. N.

(2008). The predictive validity of the Level of Service Inventory-Revised for drug-involved offenders. Criminal Justice & Behavior, 35, 819-831.

24.

Lowenkamp

C. T.

Bechtel

(2007). The Predictive validity of the LSI-R on a sample of offenders drawn from the records of the Iowa Department of Corrections Management System. Federal Probation, 71(3), 25-29.

25.

Lowenkamp

C. T.

Latessa

(2001). Level of Service Inventory-Revised Validation Study: North Dakota. Cincinnati, OH: University of Cincinnati.

26.

Lowenkamp

C.T.

Latessa

E.J.

(2002). Profiling of Offenders in North Dakota Using the Level of Service Inventory. Cincinnati, OH: University of Cincinnati.

27.

Lowenkamp

C. T.

Latessa

E. J.

(2004). Increasing the effectiveness of community programming through the risk principle: Identifying offenders for residential placement. Criminology & Public Policy, 4(1), 501-528.

28.

Lowenkamp

C. T.

Latessa

E. J.

(2005). Developing successful reentry programs: Lessons learned from the “What Works” research. Corrections Today, 67, 72-77.

29.

Lowenkamp

C. T.

Latessa

E. J.

Holsinger

A. M.

(2006). The risk principle in action: What have we learned from 13,676 offenders and 97 correctional programs? Crime & Delinquency, 51(1), 1-17.

30.

Lowenkamp

C. T.

Lovins

Latessa

E. J.

(2009). Validating the Level of Service Inventory Revised and the Level of Service Inventory: Screening Version with a sample of probationers. The Prison Journal, 89, 192. DOI: 10.1177/0032885509334755

31.

Loza

Simourd

D. J.

(1994). Psychometric evaluation of the Level of Supervision Inventory (LSI) among male Canadian federal offenders. Criminal Justice & Behavior, 21, 468-480.

32.

Manchak

S. M.

Skeem

Douglas

K. S.

Siranosian

(2009). Does gender moderate the predictive utility of the Level of Service Inventory-Revised (LSI-R) for serious violent offenders? Criminal Justice & Behavior, 36, 425-442.

33.

Maruna

(2001). Making good: How ex-convicts reform and rebuild their lives. Washington, DC: American Psychological Association.

34.

Ostermann

(2009). An analysis of New Jersey’s day reporting center and halfway back programs: Embracing the rehabilitative ideal through evidence based practices. Journal of Offender Rehabilitation, 48(2), 139-153.

35.

Ostermann

(2010). Parole? Nope, not for me: Voluntarily maxing out of prison. Crime & Delinquency. Advance online publication. doi:10.1177/0011128710372194.

36.

Paparozzi

Caplan

(2010). A national profile of paroling authorities in America: The strange bedfellow of politics and professionalism. The Prison Journal, 89, 401-425.

37.

Pew Center on the States. (2008). One in 100: Behind bars in America 2008. Washington, DC: The Pew Charitable Trusts.

38.

Pew Center on the States. (2009). One in 31: The long reach of American corrections. Washington, DC: The Pew Charitable Trusts.

39.

Reisig

M. D.

Holtfreter

Morash

(2006). Assessing recidivism risk across female pathways to crime. Justice Quarterly, 23, 384-405.

40.

Salisbury

Van Voorhis

(2009). Gendered pathways: A Quantitative investigation of women probationers’ paths to incarceration. Criminal Justice & Behavior, 36, 541-566.

41.

Salisbury

E. J.

Van Voorhis

Spiropoulous

G. V.

(2009). The predictive validity of a gender-responsive needs assessment: An exploratory study. Crime & Delinquency, 55, 550-585.

42.

Schlager

(2005). Assessing the reliability and validity of the Level of Service Inventory-Revised (LSI-R) on a community correction sample: Implications for corrections and parole policy. (Unpublished dissertation) Rutgers University, Newark, NJ.

43.

Schlager

Simourd

D. J.

(2007). Validity of the Level of Service Inventory-Revised (LSI-R) among African American and Hispanic male offenders. Criminal Justice and Behavior, 34, 545-554.

44.

Smith

Cullen

F. T.

Latessa

E. J.

(2009). Can 14,737 women be wrong? A meta-analysis of the LSI-R and recidivism for female offenders. Criminology & Public Policy, 8(1), 183-208.

45.

Van Voorhis

Salisbury

Wright

Bauman

(2008). Achieving accurate pictures of risk and identifying gender responsive needs: Two new assessments for women offenders. (National Institute of Corrections Summary Report). Washington, DC: National Institute of Corrections.

46.

Van Voorhis

Wright

Salisbury

Bauman

(2010). Women’s risk factors and their contributions to existing risk/needs assessment: The Current status of gender responsive assessment. Criminal Justice & Behavior, 37, 261-288.

47.

Vose

(2008). Assessing the predictive validity of the Level of Service Inventory-Revised: Recidivism among Iowa parolees and probationers (Doctoral dissertation). University of Cincinnati, Cincinnati, OH.

48.

Vose

Cullen

F. T.

Smith

(2008). The empirical status of the Level of Service Inventory. Federal Probation, 72(3), 22-29.

49.

Vose

Lowenkamp

C. T.

Smith

Cullen

F. T.

(2009). Gender and the predictive validity of the LSI-R: A study of parolees and probationers. Journal of Contemporary Criminal Justice, 25, 459-471.

50.

West

H. C.

(2010). Prison inmates at midyear 2009 (NJC 230113). Washington, DC: Bureau of Justice Statistics Statistical Tables.

51.

Westin

(2001). Receiver operating characteristic (ROC) analysis: Evaluating discriminance effects among decision support systems. Retrieved from http://www8.cs.umu.se/research/reports/2001/018/part1.pdf