Clinical Overrides With the YLS/CMI: Predictive Validity and Associated Factors

Abstract

This study explores the use of clinical override with the Youth Level of Service/Case Management Inventory (YLS/CMI), including implications for predictive validity as well as the factors associated with this practice. The sample included 597 justice-involved youth from a metropolitan region in Québec, Canada. The clinical override was used in 32.5% of cases, usually to increase risk levels (30.3% of cases). As found in previous studies, clinical override did not increase the predictive validity of the YLS/CMI. Upward and downward clinical overrides were differently linked to the sociodemographic characteristics and criminal history of the youths in the sample. Criminal History, Peer Relations, Personality/Behavior, and Attitudes/Orientation risk/need domains were positively associated with upward override while Family Circumstances/Parenting, Personality/Behavior, and Attitudes/Orientation risk/need domains were negatively associated with downward override. These results are discussed in relation to the impact clinical override can have on the case management and interventions provided to justice-involved youth.

Keywords

YLS/CMI clinical override recidivism predictive validity

Many clinicians working with youth involved with the justice system use the Youth Level of Service/Case Management Inventory (YLS/CMI; Hoge & Andrews, 2006, 2011), a standardized instrument whose psychometric properties for estimating the risk of recidivism have been confirmed in several studies (e.g., Olver et al., 2009; Schmidt et al., 2005). In exceptional cases, clinicians can use their clinical judgment to override a YLS/CMI risk classification and put an individual in a higher or lower risk category, a practice known as clinical override. However, studies have shown that clinical override decreases the predictive value of the instrument (e.g., Chappell et al., 2013; Guay & Parent, 2018). Deciding how best to combine standardized assessment and clinical judgment in day-to-day practice requires understanding the factors that can play a role in this decision-making process.

Evolution of Risk Assessment

Methods of assessing recidivism risk have evolved significantly over the last 50 years, through four generations of assessment (Bonta & Andrews, 2017). In the first generation, clinicians relied on a dynamic understanding of an individual based on unstructured interviews and clinical judgment (“gut feelings”; Latessa & Lovins, 2010). The second generation involved attempts to standardize risk assessment through the creation of actuarial instruments based on a specific number of static or historical risk factors, such as age or criminal history, statistically associated with recidivism in studies of normative samples (e.g., Hoffman, 1994) or in meta-analyses (e.g., Hanson & Thornton, 2000). The aggregation of ratings of all risk factors is used to establish risk levels. While numerous meta-analyses have shown that structured assessments have higher predictive validity than clinical judgment alone (e.g., Ægisdóttir et al., 2006; Grove & Meehl, 1996), they are atheoretical and provide no information about intervention targets or changes in risk over time (Olver & Wong, 2019).

In the third generation, dynamic risk factors (factors that can be modified through interventions aimed at reducing recidivism risk), such as problems with substance use or antisocial attitudes, were added to second-generation risk assessment instruments. Meta-analyses have demonstrated that dynamic risk factors are as effective as static risk factors in predicting recidivism (Campbell et al., 2009; Gendreau et al., 1996). Third-generation instruments also have a slight advantage over second-generation instruments in statistically significant predictions (Campbell et al., 2009; Schwalbe, 2007; Yang et al., 2010). Finally, fourth-generation measures make it possible to assess static risk factors (e.g., criminal history), criminogenic needs (i.e., dynamic risk factors associated with recidivism), and responsivity factors as well as providing indications for case management, including changes in the level of risk until case closure (Campbell et al., 2009; Latessa & Lovins, 2010; Olver & Wong, 2019). A meta-analysis by Campbell et al. (2009) based on five effect sizes found that fourth-generation measures are more effective than their predecessors.

The YLS/CMI

The YLS/CMI is a fourth-generation instrument used with youth between 12 and 18 years of age who have been convicted of a crime. It is one of the most widely used structured risk and need assessment instrument in many jurisdictions, such as Canada (Catchpole & Gretton, 2003; Schmidt et al., 2005, 2016; Viljoen et al., 2014), the United States (Campbell et al., 2014; Chappell et al., 2013; Guy et al., 2014), Singapore (Chu et al., 2014), and the United Kingdom (Vaswani & Merone, 2014). While there is no clear guidance about reassessment frequency when using the YLS/CMI (Hoge & Andrews, 2006, 2011), some researchers have suggested that youth should be reassessed regularly, perhaps as frequently as every 6 months (Viljoen et al., 2014; Vincent et al., 2012).

The first section of the YLS/CMI assesses the presence or absence of 42 risk factors for recidivism that have been grouped into eight broad risk/need domains: Prior and Current Offenses/Dispositions (five items), Family Circumstances/Parenting (six items), Education/Employment (seven items), Peer Relations (four items), Substance Abuse (five items), Leisure/Recreation (three items), Personality/Behavior (seven items), and Attitudes/Orientation (five items; Bonta & Andrews, 2017). These risk/need domains, with the exception of the first, are dynamic. In the second section, the 42 risk factors are totaled, and the individual is determined to be in one of four risk categories (low, moderate, high, or very high). The level of risk (low, moderate, or high) can also be generated for each risk/need domain. The third section allows clinicians to dichotomously (present/absent) assess other needs and special considerations related to family/parents (11 items) and to the individual being evaluated (27 items). These needs are considered to be noncriminogenic (i.e., a change in them is not linked to a change in recidivism risk) and are more related to treatment responsivity. This section helps in determining possible courses of action by identifying issues that can be used to decide which modes and delivery of rehabilitation services will be most effective, given the characteristics of the individual and his or her family. In the fourth section, clinicians can modify the risk level suggested by the YLS/CMI if they have information not considered by the instrument that leads them to a different conclusion. The last two sections (level of supervision and intervention targets) involve case management.

The validity and reliability of the YLS/CMI have been extensively demonstrated. Interrater reliability for a single risk/need domain, measured by the intra-class correlation coefficient, is generally .60 or higher (Rocque & Plummer-Beale, 2014; Schmidt et al., 2005) and is between .78 and .80 for the total score (Catchpole & Gretton, 2003; Rocque & Plummer-Beale, 2014), which is considered good (Hallgren, 2012). Meta-analyses have shown that YLS/CMI results are statistically significant in predictions of general recidivism for males (r ranging from .28 to .33) and females (r ranging from .25 to .40) as well as for violent recidivism for both males (r ranging from .23 to .30) and females (r ranging from .24 to .25; Olver et al., 2009; Pusch & Holtfreter, 2018; Schwalbe, 2008).

The Use of Clinical Override

Making the option of using clinical override available in the YLS/CMI (and other risk assessment instruments) can be seen as a compromise between two opposing positions regarding risk assessment: clinical judgment and actuarial prediction. While clinical judgment arguably makes it possible to take into account a wide range of information and allow for its more sophisticated integration (Ruscio, 2003; Sreenivasan et al., 2010), standardized instruments provide an objective method of assessment, untainted by bias (e.g., ignorance of base rates), heuristics, or contingent factors (e.g., social and political influences) while taking into account all the information needed for a valid risk assessment (Grisso et al., 2003; Wormith et al., 2012). Research works suggest that clinical override should be used only in specific circumstances (Hoge & Andrews, 2011, p. 7) and in no more than 10% of all assessments (Andrews et al., 2004).

Only a few studies of clinical override have focused on youth involved with the justice system. In these studies, the rate of clinical override varies between 7.0% and 14.1% (Chappell et al., 2013; McCafferty, 2017; Vaswani & Merone, 2014), although Schmidt et al. (2016) reported rates more than 40% (74.0% related to youth who had committed a sexual offense and 41.6% to youth who had not committed a sexual offense). Upward clinical override was found to be more frequent than downward, except in a study by Chappell et al. (2013). Clinical override, whether upward or downward, was found to lead to a decrease in the predictive validity of the instrument (McCafferty, 2017; Vaswani & Merone, 2014), with a higher negative impact on results for youth who had committed a sexual offense (Schmidt et al., 2016). The results for clinical override with youth are similar to those in studies of adults involved with the justice system (T. H. Cohen et al., 2016, 2020; Duwe & Rocque, 2018; Guay & Parent, 2018; Orton et al., 2021; Storey et al., 2012; Wormith et al., 2012).

Factors Associated With Clinical Override

Various factors associated with clinical override can influence how clinicians think about the risk posed by the person being assessed. For example, when a study of 92 adults who had committed a sex offense was evaluated by clinicians using the Static-99 (Harris et al., 2003) and then on file by a consensus among a group of researchers, the only item with an unacceptable intraclass correlation (ICC = .15) was the clinical override decision (Hanson et al., 2007). Few studies have examined factors associated with clinical override for either youth (Chappell et al., 2013; McCafferty, 2017; Schmidt et al., 2016) or adults (Guay & Parent, 2018; Orton et al., 2021; Wormith et al., 2012), and only half reported separate results for upward and downward clinical overrides. Because downward and upward clinical overrides do not represent the same view of the person being evaluated—one sees the person as less dangerous and the other as more dangerous—factors identified as associated with these two types of clinical overrides should always be considered separately.

The association between sociodemographic characteristics and clinical override has been investigated in some studies. In contrast to studies of adults (Guay & Parent, 2018; Orton et al., 2021; Wormith et al., 2012), sex has not been found to be associated with clinical override with youth (Chappell et al., 2013; McCafferty, 2017). Age has been positively associated with both upward and downward clinical override for adults (Guay & Parent, 2018) and positively associated with upward clinical override and negatively with downward override for youth (Chappell et al., 2013). Being a White adult has been associated with the use of upward clinical override (Orton et al., 2021), while being a White (versus Black) youth has been associated with less frequent use of downward clinical override (Chappell et al., 2013), although McCafferty (2017) did not find an association between race and clinical override for youth. Although these associations with sociodemographic characteristics reached statistical significance, they were on the small effect size of the range.

Clinicians may also use clinical override for reasons related to an individual’s index offense and criminal history. A sexual index offense is associated with upward clinical override for adults (Guay & Parent, 2018; Orton et al., 2021) but not for youth (McCafferty, 2017). However, clinical override is used more frequently with both adults (Duwe & Rocque, 2018; Storey et al., 2012; Wormith et al., 2012) and youth (Schmidt et al., 2016) who have committed a sex offense than with people who have not. Prior referrals and being under supervision at the time of risk assessment were both positively associated with the use of upward and downward clinical override for youth (Chappell et al., 2013).

A third category of factors associated with clinical override involves the information assessed in the instrument. For adults, the total risk score is negatively associated with clinical override (Wormith et al., 2012) as is the initial risk level for youth (Schmidt et al., 2016). For adults, the Education/Employment domain of the Level of Service/Case Management Inventory (LS/CMI; Andrews et al., 2004) is negatively associated with either combined clinical override (Wormith et al., 2012) or upward override specifically (Guay & Parent, 2018). Family/marital, pro-criminal attitude, and antisocial pattern domains as well as the total score in the specific risk/need domains (Section 2), personal problems (Section 2.1), perpetration history (Section 2.2), prison experience (Section 3), social, health, mental health problems (Section 4), and special responsivity (Section 5) subscales are positively associated with combined (Wormith et al., 2012) and upward (Guay & Parent, 2018) clinical override. The antisocial pattern domain as well as the total scores in the specific risk/need domains (Section 2) and personal problems (Section 2.1) subscales were also positively associated with downward clinical override (Guay & Parent, 2018). No study of youth has evaluated the association between the risk/need domains of the YLS/CMI and clinical override.

Rationale for the Current Study

Previous studies that have evaluated the use of clinical override and the factors associated with it have some limitations. First, twice as many studies of clinical override have been conducted with adults as with youth, which limits the generalizability of results. Second, it is difficult to understand what is associated with the use of either upward or downward clinical override as study results are often contradictory. Finally, only two studies, based on adult samples, have considered all the domains in the risk assessment instrument as factors potentially associated with clinical override. Earlier contradictory results could be an indication that different people use different heuristics in evaluating risk, and thus there may not be specific profiles that lead clinicians to use clinical override. This merits further investigation, as do the factors involved in clinical override as they pertain to a wider sample base. Our research is intended to achieve this by analyzing the use of clinical override with the YLS/CMI in cases involving youth. We first describe the use of clinical override and then examine the association of clinical override use with the predictive validity of the YLS/CMI for general recidivism as well as recidivism involving violent and nonviolent offenses. We then evaluate the association of sociodemographic characteristics (age, sex, and race), criminal history (sexual and violent offenses), totals of the YLS/CMI risk/need domains, and other needs and special considerations scales with combined, upward, and downward clinical override.

Method

Participants

In Québec, youth involved with the justice system through either the Youth Criminal Justice Act (YCJA) or the Youth Protection Act (YPA) become the responsibility of youth centers. There are 15 youth centers in Québec, each with many associated service centers, responsible for monitoring youth completing sentences (which can range from probation to imprisonment) in their respective regions. Our sample was drawn from all youth registered with a youth center in a metropolitan region in Québec who had at least one closed file for a sentence under the YCJA (i.e., the sentence handed down by the court had been completed) between January 1, 2014, and December 31, 2016, and who had at least one YLS/CMI in their file. We used only closed files so that all risk assessments for each individual were available. Of the 635 youths, 30 were excluded because their YLS/CMI was incomplete (e.g., date or scores for too many items were missing) or because the information on recidivism was missing. Eight other participants were excluded based on their age (>20 years) at the time of assessment. Under the YCJA, a youth can continue to be supervised by a youth center until he or she is 20 years old. The final sample therefore consisted of 597 individuals between the ages of 12 and 20 at the time of their first YLS/CMI assessments (see Table 1).

Table 1:

Descriptive Statistics

Sample characteristics	%	n	M	SD
Sociodemographic characteristics
Age at assessment	–	–	16.4	1.4
Female	13.9	83	–	–
Race
White	41.7	249	–	–
Black	31.0	185	–	–
Hispanic	7.0	42	–	–
Arab	13.2	79	–	–
Asian	4.5	27	–	–
Other	2.5	15	–	–
Criminal history
History of violent nonsexual offense	74.2	443	–	–
Number of previous violent nonsexual offenses	–	–	1.8	2.0
History of sexual offense	9.5	57	–	–
Number of previous sexual offense	–	–	0.2	0.7
YLS/CMI Risk/need domains and total score
Prior and Current Offenses/Dispositions (0–5)	–	–	0.9	1.1
Family Circumstances/Parenting (0–6)	–	–	1.9	1.5
Education/Employment (0–7)	–	–	2.6	1.7
Peer Relations (0–4)	–	–	1.7	1.2
Substance Abuse (0–5)	–	–	1.6	1.6
Leisure/Recreation (0–3)	–	–	1.3	0.9
Personality/Behavior (0–7)	–	–	2.3	1.8
Attitudes/Orientation (0–5)	–	–	1.2	1.2
Total score	–	–	13.4	6.8
YLS/CMI Other needs and special considerations scales
Family/parents (0–11)^a	–	–	1.2	1.5
Youth (0–27)^b	–	–	2.8	2.7
Recidivism
General 1-year	37.4	223	–	–
General 2-year	51.3	306	–	–
Violent 1-year	27.0	161	–	–
Violent 2-year	38.9	232	–	–
Nonviolent 1-year	21.1	126	–	–
Nonviolent 2-year	29.9	178	–	–
Clinical override
No clinical override	66.5	397	–	–
Upward	30.3	181	–	–
Downward	3.2	19	–	–

n = 569. ^b n = 566.

Procedure and Measures

Data were obtained from three sources: the archived paper file, the Projet Intégration Jeunesse (PIJ), and the Module d’informations policières (MIP). The PIJ is a computer system used throughout Québec by youth centers responsible for monitoring YCJA sentences. It provides details (date, type of crime, and results of the court process) about all offenses committed by a person below the age of 18 who was prosecuted in the Youth Division of the Court of Quebec. The MIP is a computer system used by police officers with information on all offenses and police interventions that take place in Québec.

A research team composed of two research assistants and a research manager collected all data concerned with risk assessment between February and June 2017 for each file. The sociodemographic characteristics of participants, their criminal history, and recidivism until the age of 18 were extracted from the PIJ computer system in August 2017 with the collaboration of the youth center. Data regarding recidivism after age 18 was extracted from the MIP in March 2018. This research received approval from the ethics committee of the youth center where the research took place and from a judge from the Youth Division of the Quebec Court to allow access to the files without the consent of their subjects.

Sociodemographic Characteristics and Criminal History

Race was based on the data entered by the youth worker in the PIJ computer system and grouped into six categories: White, Black, Hispanic, Arab, Asian, and other. Age at assessment was calculated based on the date of birth and date of completion of the YLS/CMI. Criminal history involving violence (yes/no and number of offenses) was defined in terms of any act, attempted act, or threatened act with the potential to cause physical harm to another person (e.g., homicide and assault) noted in the individual’s criminal record. Criminal history of sexual offenses (yes/no and number of offenses) was defined by the presence of any record of sexual offenses (e.g., sexual assault and sexual exploitation) in the individual’s criminal record (see Table 1).

YLS/CMI and Clinical Override

In Québec, Law 21 establishes that the assessment of a youth under the YCJA can be undertaken only by criminologists, social workers, psychologists, or psychoeducators who are members of their professional order. There is no court-mandated assessment method, and each of the 15 youth centers responsible for monitoring YCJA sentences is free to determine how to evaluate risk assessment. In the youth center where the research took place, the YLS/CMI has been the method of choice for more than 15 years. All clinicians, known as youth workers, receive 2 days of training on YLS/CMI (Hoge & Andrews, 2006) from an internal team of trainers.

The mean number of YLS/CMI assessments in the youth’s files was 2.1 (standard deviation [SD] = 1.6) and ranged from 1 to 11. For youth with more than one YLS/CMI, the first one was used. In Québec, the first risk assessment with the YLS/CMI is generally completed by a youth worker after the youth has been found guilty but before the judge hands down a sentence. The report, based on the YLS/CMI, is intended to help the judge make the most appropriate decision about sentencing, so it can affect a youth’s future as well as determine case management.

Our analysis was based on the initial and final risk level (low, moderate, high, or very high) and the total score for each risk/need domain as well as the total of other needs and special considerations concerning family/parents and youth scales. A higher score was related to a larger number of risk/need factors or other needs and special considerations. Following Vaswani and Merone (2014), if the clinical override section had been left blank by the youth worker, it was assumed that this indicated agreement with the risk level generated directly by the YLS/CMI—the initial risk level. The clinical override was measured dichotomously (combined clinical override; yes/no). Two other dichotomous variables were used to measure upward (upward override/no override) and downward clinical override (downward override/no override; see Table 1).

Recidivism

Recidivism was defined as any new criminal conviction registered in the PIJ or the MIP computer systems. Recidivism for youth below the age of 18 was obtained from the PIJ computer system. The name and date of birth of each individual who had become 18 years old during the follow-up period were forwarded to the police department in the municipality where the research took place and their official criminal records as an adult were obtained from the MIP. This information was used to create a general category of recidivism. Two other categories of recidivism were created, one for violent crimes including sexual offenses (e.g., homicide, attempted murder, sexual and nonsexual assault, robbery) and one for nonviolent (or property) crimes (e.g., breaking and entering, motor vehicle theft, theft, fraud). The follow-up period began at the date the YLS/CMI risk assessment was made or after the individual had been released from custody for those not in the community at the time of the assessment. The follow-up period ended at the date of our request for information on recidivism. The mean for the follow-up period was 4.1 years (SD = 1.2) and ranged from 1.7 to 8.9 years. To control for the impact of variable follow-up periods, two fixed follow-up periods were created for each recidivism measure: 1 year and 2 years.

Analytical Strategy

Descriptive analyses were used to describe the sample characteristics and variables used in this study. The predictive validity of the YLS/CMI for all recidivism measures was assessed using the area under the curve (AUC) values for the receiver operation characteristic (ROC) analysis, with higher values indicating stronger predictive validity. ROC analysis is used to evaluate the accuracy of a continuous or ordinal scale (e.g., YLS/CMI risk level) in predicting a dichotomous outcome (e.g., recidivism). The AUC can be considered an indicator of the probability that a randomly selected recidivist will have a higher score than a randomly selected nonrecidivist. According to Rice and Harris (2005), thresholds for small, medium, and large effect sizes are AUC values of .556, .639, and .714, respectively. The association of clinical override use with predictive validity was assessed by comparing predictive validity estimates of the risk levels before (initial risk level) and after the clinical override (final risk level) for the same participants. The DeLong et al. (1988) test in MedCalc version 20.027 (MedCalc Software Ltd., 2022) was used because it considers the correlated nature of the data. This formula provides a critical z score that must reach ±1.96 to achieve statistical significance at the .05 level. Finally, as in Wormith et al. (2012) and Guay and Parent (2018), partial point-biserial correlations were used to identify factors associated with the use of combined, upward, and downward overrides while taking into account the initial risk level. According to J. Cohen (1988), thresholds for small, medium, and large effect sizes are correlation values of .10, .30, and .50, respectively.

Results

Descriptive statistics

Descriptive statistics are presented in Table 1. The age of the youth involved with the justice system in our sample was 16.4 years on average at the time of their YLS/CMI (SD = 1.4). The majority were male (86.1%; n = 514) and White (41.7%; n = 249) or Black (31.0%, n =185). Three-quarters of the sample had committed a violent offense at some point before their risk assessment was completed, with an average of 1.8 violent offenses. Fewer than 10% had committed a sexual offense. The mean YLS/CMI total score was 13.4, which can be considered a moderate risk level. The recidivism rate was 37.4% and 51.3% for 1-year and 2-year general recidivism, 27.0% and 38.9% for violent recidivism, and 21.1% and 29.9% for nonviolent recidivism. Clinical override had been used in 33.5% (n = 200) of cases.

The distribution of clinical overrides is shown in Table 2. The results below the shaded diagonal line show downward overrides, while the results above it show upward overrides. In 181 (30.3%) of 597 cases, the override had been used to increase the recidivism risk level, while in 19 (3.2%) cases it had been used to reduce it. The 19 downward clinical overrides were used to move youth from a moderate to a low-risk level. The majority of the upward overrides (n = 103) were used to move youth from a moderate to a high-risk level. Except for seven youths, the initial risk level was upgraded or downgraded by only one risk level.

Table 2:

Distribution of Clinical Override by Risk Level

Initial risk level	Final risk level
Initial risk level	Low	Moderate	High	Very high	Total
Low	89	56	10	0	155
Moderate	19	294	103	7	423
High	0	0	14	5	19
Very high	0	0	0	0	0
Total	108	350	127	12	597

Clinical Override and Predictive Validity

As shown in Table 3, AUC values and confidence intervals did not overlap with the chance level of predictive validity (.500) for YLS/CMI without clinical override for 2-year general, violent, and nonviolent recidivism as well as 1-year nonviolent recidivism. However, the AUC values were in the small effect size range (all less than .580). These results, obtained using the French version of the YLS/CMI, were lower than those found with the original English version in meta-analyses on the predictive validity of the instrument (Olver et al., 2009; Pusch & Holtfreter, 2018; Schwalbe, 2007, 2008)

Table 3:

Areas Under the Curve for YLS/CMI Initial and Final Risk Level for General, Violent, and Nonviolent Recidivism

Type of recidivism	Without clinical override (n = 397)	With clinical override (n = 200)
Type of recidivism	Without clinical override (n = 397)	Initial risk level	Final risk level	AUC differences (final-initial)	Z
General recidivism
1-year	.556 [.499, .613]	.613 [.533, .693]	.635 [.554, .715]	.022	0.530
2-year	.574 [.518, .631]	.594 [.515, .672]	.614 [.536, .692]	.020	0.515
Violent recidivism
1-year	.546 [.483, .608]	.630 [.546, .714]	.658 [.574, .742]	.028	0.643
2-year	.563 [.506, .620]	.583 [.501, .665]	.629 [.550, .708]	.046	1.190
Nonviolent recidivism
1-year	.570 [.503, .637]	.581 [.488, .673]	.590 [.495, .685]	.009	0.195
2-year	.572 [.512, .632]	.543 [.457, .628]	.548 [.462, .634]	.005	0.136

Note. 95% confidence intervals in brackets. AUC = area under the curve.

p < .05, two-tailed. ** p <.01, two-tailed. *** p < .001, two-tailed.

For YLS/CMI with clinical override, AUC values and confidence intervals, both before and after the override, did not overlap with chance level only for 1-year and 2-year general and violent recidivism. These results were in the small to medium effect sizes (AUC ranging from .583 and .658), but still lower than those found in meta-analyses. The AUC was slightly better for 1-year (ranging from .613 to .658) compared with 2-year recidivism (ranging from .583 to .629).

The association of clinical override use with predictive validity was assessed by comparing AUC values of the risk levels before (initial risk level) and after the clinical override (final risk level) for the same participants. In general, clinical override use appeared to be associated with a slight increase in the predictive validity of the YLS/CMI across all 1-year and 2-year recidivism measures. However, these differences were small (ranging from .005 to .046) and not statistically significant (all p > .05).

Factors Associated With Clinical Override

Similar to the work of Wormith et al. (2012) and Guay and Parent (2018), to determine what is associated with the use of a clinical override, the relationship between the use (yes/no) of combined, upward, and downward clinical overrides and sociodemographic characteristics, criminal history and YLS/CMI scores was examined. Partial point-biserial correlations controlling for YLS/CMI risk level were computed (see Table 4).

Table 4:

Partial Point-Biserial Correlation Matrix with Sociodemographic Characteristics, YLS/CMI Risk Domains and YLS/CMI Other Needs/Special Considerations With the Use of Clinical Overrides Controlling for YLS/CMI Initial Risk Level

Factors	Combined clinical override	Upward clinical override	Downward clinical override
Sociodemographic characteristics
Age at assessment	.013	−.010	.102*
Female	−.109**	−.101*	−.067
Black versus White	.094	.138**	−.133*
Hispanic versus White	−.048	−.016	−.105
Arab versus White	.050	.075	−.051
Asian versus White	.059	.065	−.017
Other minorities versus White	.020	.046	−.063
Criminal history
History of violent nonsexual offense	.057	.067	−.021
Number of previous violent nonsexual offenses	.167***	.192***	−.063
History of sexual offense	.058	.054	.042
Number of previous sexual offense	.077	.082*	.003
YLS/CMI Risk/need domains and total score
Prior and Current Offenses/Dispositions	.150***	.162***	.011
Family Circumstances/Parenting	.102*	.145***	−.137**
Education/Employment	.127**	.138***	−.005
Peer Relations	.137***	.171***	−.090
Substance Abuse	.021	.022	.004
Leisure/Recreation	.114***	.134**	−.036
Personality/Behavior	.103*	.160***	−.189***
Attitudes/Orientation	.226***	.284***	−.155**
Total score	.273***	.343***	−.185***
YLS/CMI Other needs and special considerations scales
Family/Parents	.015	.020	−.015
Youth	.080	.090*	−.019

p < .05, two-tailed. ** p <.01, two-tailed. *** p < .001, two-tailed.

Among sociodemographic characteristics, only sex (being female) was negatively associated with combined clinical override. Being a male and being Black versus White was associated with upward clinical override, while being older and being White versus Black was associated with downward clinical override. Regarding criminal history, only the number of violent nonsexual offenses in the individual’s criminal record was positively associated with combined clinical override. The number of both violent nonsexual and sexual offenses was positively associated with upward clinical override. According to J. Cohen (1988), all these results were in the small effect size range (r_pb ranging from −.082 to −.192).

All risk/need domains except Substance Abuse were positively associated with combined clinical override (r_pb ranging from .102 to .273) and in the small to medium effect size range. Thus, the higher an offender scored in these risk/need domains, the greater the probability that youth workers would use clinical override. However, the results for upward and downward clinical overrides considered separately create very different portraits and show the importance of distinguishing between upward and downward overrides. All risk/need domains except Substance Abuse were still positively associated with upward clinical override, and the correlation coefficients are slightly higher (r_pb ranging from .134 to .343) but still in the small to medium effect size range. Also, the youth other needs and considerations scale was now positively associated with upward clinical override. Three risk/need domains (Family Circumstances/Parenting, Personality/Behavior and Attitudes/Orientation) and the YLS/CMI total score were negatively associated with downward override (r_pb ranging from −.137 to −.189).

Discussion

The current study examined clinical override in risk assessments using the YLS/CMI. We focused on the frequency of clinical override, the association of clinical override use with predictive validity, and the factors associated with this practice in general and for upward and downward overrides specifically. Our results show that clinical override was used 2 to 3 times more often in our sample than the rate found in previous studies (Chappell et al., 2013; McCafferty, 2017; Vaswani & Merone, 2014). Our clinical override rate was similar to that found by Schmidt et al. (2016), which was 41.6% for youth who had not committed sexual offenses and close to four times greater than the rate recommended by Andrews et al. (2004).

YLS/CMI, Clinical Override, and Predictive Validity

Although we found that the YLS/CMI without clinical override significantly predicted 2-year general, violent, and nonviolent recidivism and the YLS/CMI with clinical override (before and after) significantly predicted general and violent recidivism, the AUC were smaller than those found in meta-analyses of the predictive validity of the YLS/CMI (Olver et al., 2009; Pusch & Holtfreter, 2018; Schwalbe, 2007, 2008) but similar to those in a study carried out in the same jurisdiction as our study (Saint-Louis, 2015). The difference could be explained by the training in the use of the YLS/CMI received by the youth workers, which is provided by an internal team and could be suboptimal.

Contrary to previous studies of youth (McCafferty, 2017; Vaswani & Merone, 2014), we found that clinical override at first risk assessment slightly increases the predictive validity of the YLS/CMI, but these differences were not statistically different. Therefore, even if the direction of our results differs from previous studies, we can only conclude that clinical override use on the first risk assessment is not associated with an improvement in the predictive validity of the YLS/CMI.

Clinical override on the first assessment can have a large impact on youth. In Québec, risk assessment is usually done before sentencing and results can affect the kind (e.g., custody vs. probation) and length of sentence as well as the intensity and duration of the intervention (Olver & Wong, 2019). Clinical overrides are more often used to increase risk level than to decrease it, exposing large numbers of youth to the risk of a more severe sentence and greater intervention intensity as well as the negative impacts of these outcomes (Lipsey, 2009; Luong & Wormith, 2011; Vieira et al., 2009). A longer and more severe sentence may increase exposure to youth who are at higher risk of recidivism as well as disrupting protective elements (Latessa & Lovins, 2010), such as having prosocial friends, attending a good school, or having a job. Clinical override should thus be used very carefully at the first risk assessment given that this practice is not associated with an improvement in the predictive validity of the instrument.

Factors Associated With Upward and Downward Clinical Override

Our results suggest that several factors were associated with combined, upward, and downward clinical override. However, theoretically, none of these factors should have had an effect on the clinical override, because they had already been taken into account in the risk/need domains, involved demographic characteristics, or were part of special needs and considerations, which are not risk factors according to the Risk-Need-Responsivity model that served as the developmental base for the instrument (Bonta & Andrews, 2017).

Being a male was found to be related to the use of upward clinical override, similar to previous studies with adults (Guay & Parent, 2018; Orton et al., 2021; Wormith et al., 2012), but contrasting with other studies of youth (Chappell et al., 2013; McCafferty, 2017). Our results are similar to Chappell et al. (2013) regarding race. Black (vs. White) youth had a higher probability of seeing their initial risk levels increased through a clinical override, whereas older White youth had a higher probability of seeing their initial risk levels decreased. These results support the symbolic threat theory, which suggests that certain stereotypes related to a youth’s demographic characteristics (e.g., ethnicity, age, gender) can influence clinician’s perceptions (Leiber & Fox, 2005). For example, individuals who are Black may be seen as more delinquent, dangerous, and less likely to benefit from intervention (Leiber & Fox, 2005), a perception that can also affect clinical decisions and thus the judicial process (Chappell et al., 2013; Irwin & Real, 2010). Given that clinical override at first risk assessment can have an impact on sentence and intervention intensity, training in use of the instrument should emphasize the effect of sociodemographic characteristics on clinicians’ perceptions of recidivism and risk.

All risk/need domains, except Substance Abuse, were positively associated with upward clinical override. The risk/need domains previously known as the Big Four (Criminal History, Peers Relations, Personality/Behavior, and Attitudes/Orientation) were the factors most strongly associated with upward override (in the small to medium effect size range). These results are similar to those found by Guay and Parent (2018) in the use of the LS/CMI with adults, except for Peer Relations. The case of downward clinical override was different: A lower score on Family Circumstances/Parenting, Personality/Behavior, and Attitudes/Orientation was associated with the use of downward override (all in the small effect size range). A youth without antisocial indicators (personality and attitude) and with a prosocial family (warm and supervising) was perceived to be at lower risk than suggested by the risk level of the YLS/CMI. In Guay and Parent’s (2018) study with adults, only the antisocial pattern was associated with a downward clinical override.

Antisocial personality traits and attitudes as well as association with delinquent peers are considered to be the most difficult risk/need domains to deal with in interventions involving individuals involved with the justice system, either because they are seen as unchangeable, complex, or difficult to affect in a short follow-up period of time (Haqanee et al., 2015; Viglione et al., 2015) or because clinicians feel they lack the resources or the knowledge to make these risk/need domains their main intervention priorities (Viljoen et al., 2019). Given that the mean sentence length for youth involved with the justice system in Québec is 6 months (Lafortune et al., 2015), clinicians may feel that they will not have the time to have a significant impact on these risk/need domains. By increasing the risk level, they also increase intervention intensity, which could give them more time to work on these difficult risk/need domains. Specific training should be provided to help youth workers develop skills that will help them work more effectively in these areas. Cognitive behavior therapies have been shown to be effective with youth (Lipsey, 2009), and specific training in these techniques could enhance probation officers’ skills (Bonta et al., 2019) and reduce recidivism (Bonta et al., 2011, 2021)

In the youth center from which our sample was taken, youth workers had used clinical overrides to adjust intervention intensity (personal communication from a senior manager, November 30, 2021). The frequency of meetings are determined by risk levels, and some youth workers might be tempted to reduce risk in order not to expose offenders to a high frequency of treatment and supervision. This hypothesis has also been raised by Guay and Parent (2018) for the LS/CMI. Hopefully, the new directives and policies implemented recently will reduce the use of clinical override.

Strengths and Limitations of the Study

The composition of the sample makes it possible to generalize the results of the study to all youth involved with the justice system who have been evaluated with the YLS/CMI in youth centers similar to the one where the research took place, given that the sample comprises all youth involved with this center over a 3-year period. Exhaustive collection of data from files helped avoid the inevitable bias associated with the voluntary recruitment of participants, making the study more representative. Despite the fact that it was unlikely that all new crimes (recidivism) committed by the youth in our study had been detected by the justice system and thus included in our study, our results agree with those of other studies based on official data.

Interpretation of our findings must, however, take into consideration certain limitations. First, the study identified factors associated with clinical override but these correlations do not explain why clinicians change the risk level in appraisals of youth involved with the justice system. Discussions of factors related to clinical override are only conjectures. Other factors that could be associated with the use of clinical override were not considered, including clinician characteristics. For example, Garb and Boyle (2015) found that the quality of clinician’s academic training had more effect on the accuracy of clinical predictions than experience. The lack of information about the clinicians responsible for suggesting overrides, given the variety of their experience and theoretical orientations, is an important limitation of this study. Future research might consider whether there is any association between clinicians’ gender, academic training, or years of experience and their propensity to use clinical override.

Conclusion

This study explores various factors potentially associated with clinical override as well as the association of clinical override with the predictive validity of YLS/CMI. Like previous studies with the YLS/CMI (Schmidt et al., 2016; Vaswani & Merone, 2014), the LS/CMI (Guay & Parent, 2018; Orton et al., 2021; Wormith et al., 2012) and other risk assessment instruments for youth (McCafferty, 2017) or adults (T. H. Cohen et al., 2016, 2020; Duwe & Rocque, 2018; Storey et al., 2012), our results show that clinical override use is not associated with an improvement in the predictive validity of the instrument. They also suggest that considering clinical override by combining upward and downward override may give an inaccurate portrait of the factors associated with this practice. Some characteristics of the youth being assessed (being a male, Black, and with a violent/sexual criminal history) may be the basis of a stereotype of the recidivist in clinicians’ minds, biasing their perceptions and leading them to consider individuals with these characteristics as at higher risk than they really are. Furthermore, clinicians may associate higher scores in risk/need domains that have been identified in previous research as more difficult to change (Haqanee et al., 2015; Viglione et al., 2015; Viljoen et al., 2019) with difficulty in providing efficient treatment, leading them to make harsher judgments in assessments. It would therefore be relevant in future research to explore whether clinicians’ attributes, such as skill in developing relationships with clients or individual characteristics (e.g., gender, academic training), are associated with clinical override.

Footnotes

Authors’ Note:

We have no known conflict of interest to disclose. We received funding from Grant 430- 2016-00313 from the Social Sciences and Humanities Research Council of Canada. The authors would also like to acknowledge the financial support of the Montreal Youth in Difficulty Institute in the writing of this article.

ORCID iDs

Geneviève Parent

Marie-Pier Bilodeau

Jean-Pierre Guay

Geneviève Parent, PhD, is a professor in the Department of Psychoeducation and Psychology at the University of Quebec in Outaouais in Canada. Her research focuses primarily on risk assessment (risk and protective factors) and treatment of justice-involved youth as well as on the implementation of evidence-based practices in rehabilitation centers for youth.

Marie-Pier Bilodeau, DPs, is a psychologist working with justice-involved youth and youth with behavior problems in a youth center in Montréal, Canada. She conducts assessments and provides individual and group treatment as well as clinical support to clinicians working with these youth.

Catherine Laurier, PhD, is a professor in the Department of Psychoeducation at the University of Sherbrooke. Her research focuses on the mental health of justice-involved youth and the impact of traumatic events experienced (or not) in the context of delinquency. In addition, she is interested more generally in media that promote psychosocial integration of adolescents, including sports and physical activities.

Jean-Pierre Guay, PhD, is professor of criminology at University of Montréal, Scientific Director of Forensia training center at Philippe Pinel National Institute of Legal Psychiatry, and researcher at the International Center for Comparative Criminology. His research focuses on assessment of risk and protective factors and on the potential uses of artificial intelligence to improve clinical training.

References

Ægisdóttir

White

M. J.

Spengler

P. M.

Maugherman

A. S.

Anderson

L. A.

Cook

R. S.

Nichols

C. N.

Lampropoulos

G. K.

Walker

B. S.

Cohen

Rush

J. D.

(2006). The meta-analysis of clinical judgment project: Fifty-six years of accumulated research on clinical versus statistical prediction. The Counseling Psychologist, 34(3), 341–382. https://doi.org/10.1177/0011000005285875

Andrews

Bonta

Wormith

J. S.

(2004). Level of Service/Case Management Inventory. https://doi.org/10.1037/t05029-000

Bonta

Andrews

D. A.

(2017). The psychology of criminal conduct. Routledge.

Bonta

Bourgon

Rugge

Pedneault

C. I.

Lee

S. C.

(2021). A system-wide implementation and evaluation of the Strategic Training Initiative in Community Supervision (STICS). Journal of Criminal Justice, 74, 101816. https://doi.org/10.1016/j.jcrimjus.2021.101816

Bonta

Bourgon

Rugge

Scott

T. L.

Yessine

A. K.

Gutierrez

(2011). An experimental demonstration of training probation officers in evidence-based community supervision. Criminal Justice and Behavior, 38(11), 1127–1148. https://doi.org/10.1177%2F0093854811420678

Bonta

Rugge

Bourgon

Wanamaker

K. A.

(2019). A conceptual replication of the Strategic Training Initiative in Community Supervision (STICS). Journal of Experimental Criminology, 15(3), 397–419. https://doi.org/10.1007/s11292-019-09371-4

Campbell

Onifade

Barnes

Peterson

Anderson

Davidson

Gordon

(2014). Screening offenders: The exploration of a youth level of service/case management inventory (YLS/CMI) brief screener. Journal of Offender Rehabilitation, 53(1), 19–34. https://doi.org/10.1080/10509674.2013.861314

Campbell

M. A.

French

Gendreau

(2009). The prediction of violence in adult offenders: A meta-analytic comparison of instruments and methods of assessment. Criminal Justice and Behavior, 36(6), 567–590. https://doi.org/10.1177%2F0093854809333610

Catchpole

R. E. H.

Gretton

H. M.

(2003). The predictive validity of risk assessment with violent young offenders: A 1-year examination of criminal outcome. Criminal Justice and Behavior, 30(6), 688–708. https://doi.org/10.1177/0093854803256455

10.

Chappell

A. T.

Maggard

S. R.

Higgins

J. L.

(2013). Exceptions to the rule? Exploring the use of overrides in detention risk assessment. Youth Violence and Juvenile Justice, 11(4), 332–348. https://doi.org/10.1177/1541204012470849

11.

Chu

C. M.

Lee

Zeng

(2014). The utility of the YLS/CMI-SV for assessing youth offenders in Singapore. Criminal Justice and Behavior, 41(12), 1437–1457. https://doi.org/10.1177/0093854814537626

12.

Cohen

(1988). Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence Erlbaum Associates.

13.

Cohen

T. H.

Lowenkamp

C. T.

Bechtel

Flores

A. W.

(2020). Risk assessment overrides: Shuffling the risk deck without any improvements in prediction. Criminal Justice and Behavior, 47(12), 1609–1629. https://doi.org/10.1177/0093854820953449

14.

Cohen

T. H.

Pendergast

VanBenschoten

S. W.

(2016). Examining overrides of risk classifications for offenders on federal supervision. Federal Probation, 80(1), 12–21.

15.

DeLong

E. R.

DeLong

D. M.

Clarke-Pearson

D. L.

(1988). Comparing the areas under two or more correlated receiver operating characteristic curves: A nonparametric approach. Biometrics, 44, 837–845. https://doi.org/10.2307/2531595

16.

Duwe

Rocque

(2018). The home-field advantage and the perils of professional judgment: Evaluating the performance of the Static-99R and the MnSOST-3 in predicting sexual recidivism. Law and Human Behavior, 42(3), 269–279. https://doi.org/10.1037/lhb0000277

17.

Garb

H. N.

Boyle

P. A.

(2015). Understanding why some clinicians use pseudoscientific methods: Findings from research on clinical judgment. In Lilienfeld

S. O.

Lynn

S. J.

Lohr

J. M.

(Eds.), Science and pseudoscience in clinical psychology (2th ed., p.19–41). The Guildford Press.

18.

Gendreau

Little

Goggin

(1996). A meta-analysis of the predictors of adult offender recidivism: What works! Criminology, 34(4), 575–608. https://doi.org/10.1111/j.1745-9125.1996.tb01220.x

19.

Grisso

Malamuth

N. M.

Barbaree

Quinsey

Knight

(2003). Risk assessment: Discussion of the section. Annals of the New York Academy of Sciences, 989(1), 236–246. https://doi.org/10.1111/j.1749-6632.2003.tb07309.x

20.

Grove

W. M.

Meehl

P. E.

(1996). Comparative efficiency of informal (subjective, impressionistic) and formal (mechanical, algorithmic) prediction procedures: The clinical–statistical controversy. Psychology, Public Policy, and Law, 2(2), 293–323. http://dx.doi.org/10.1037/1076-8971.2.2.293

21.

Guay

J.-P.

Parent

(2018). Broken legs, clinical overrides, and recidivism risk: An analysis of decisions to adjust risk levels with the LS/CMI. Criminal Justice and Behavior, 45(1), 82–100. https://doi.org/10.1177/0093854817719482

22.

Guy

L. S.

Nelson

R. J.

Fusco-Morin

S. L.

Vincent

G. M.

(2014). What do juvenile probation officers think of using the SAVRY and YLS/CMI for case management, and do they use the instruments properly? International Journal of Forensic Mental Health, 13(3), 227–241. https://doi.org/10.1080/14999013.2014.939789

23.

Hallgren

K. A.

(2012). Computing inter-rater reliability for observational data: An overview and tutorial. Tutorials in Quantitative Methods for Psychology, 8(1), 23–34. https://doi.org/10.20982/tqmp.08.1.p023

24.

Hanson

R. K.

Harris

A. J.

Scott

T. L.

Helmus

(2007). Assessing the risk of sexual offenders on community supervision: The Dynamic Supervision Project (Vol. 5). Public Safety Canada.

25.

Hanson

R. K.

Thornton

(2000). Improving risk assessments for sex offenders : A comparison of three actuarial scales. Law and Human Behavior, 24(1), 119–136. https://doi.org/10.1023/A:1005482921333

26.

Haqanee

Peterson-Badali

Skilling

(2015). Making “what works” work: Examining probation officers’ experiences addressing the criminogenic needs of juvenile offenders. Journal of Offender Rehabilitation, 54(1), 37–59. https://doi.org/10.1080/10509674.2014.980485

27.

Harris

Phenix

Hanson

R. K.

Thornton

(2003). Static-99 coding rules: Revised 2003. Department of the Solicitor General of Canada.

28.

Hoffman

P. B.

(1994). Twenty years of operational use of a risk prediction instrument: The United States Parole Commission’s Salient Factor Score. Journal of Criminal Justice, 22(6), 477–494. https://doi.org/10.1016/0047-2352(94)90090-6

29.

Hoge

R. D.

Andrews

D. A.

(2006). Youth Level of Service/Case Management Inventory (YLS/CMI) user’s manual. Multi-Health Systems.

30.

Hoge

R. D.

Andrews

D. A.

(2011). Youth Level of Service/Case Management Inventory 2.0 (YLS/CMI 2.0) user’s manual. Multi-Health Systems.

31.

Irwin

H. J. F.

Real

D. L.

(2010). Unconscious influences on judicial decision-making: The illusion of objectivity. Mcgeorge Law Review, 42(1), 1–18.

32.

Lafortune

Royer

M. N.

Rossi

Turcotte

M. E.

Boivin

Cousineau

M.-M.

Dionne

Drapeau

Faubert

Guay

J.-P.

Fenchel

Laurier

Meilleur

Trépanier

(2015). La loi sur le système de justice pénale pour les adolescents sept ans plus tard: portrait des jeunes, des trajectoires et des pratiques (FQRSC 2011-TA-144097). https://frq.gouv.qc.ca/app/uploads/2021/08/resume_loi_systeme_justice_penale_adolescents.pdf

33.

Latessa

E. J.

Lovins

(2010). The role of offender risk assessment: A policy maker guide. Victims and Offenders, 5(3), 203–219. https://doi.org/10.1080/15564886.2010.485900

34.

Leiber

M. J.

Fox

K. C.

(2005). Race and the impact of detention on juvenile justice decision making. Crime & Delinquency, 51(4), 470–497. https://doi.org/10.1177/0011128705275976

35.

Lipsey

M. W.

(2009). The primary factors that characterize effective interventions with juvenile offenders: A meta-analytic overview. Victims and Offenders, 4, 124–147. https://doi.org/10.1080/15564880802612573

36.

Luong

Wormith

J. S.

(2011). Applying risk/need assessment to probation practice and its impact on the recidivism of young offenders. Criminal Justice and Behavior, 38, 1177–1199. https://doi.org/10.1177/0093854811421596

37.

McCafferty

J. T.

(2017). Professional discretion and the predictive validity of a juvenile risk assessment instrument: Exploring the overlooked principle of effective correctional classification. Youth Violence and Juvenile Justice, 15(2), 103–118. https://doi.org/10.1177/1541204015622255

38.

MedCalc Software Ltd. (2022). MedCalc: Easy-to-use statistical software (version 20.027) [Computer software]. https://www.medcalc.org

39.

Olver

M. E.

Stockdale

K. C.

Wormith

J. S.

(2009). Risk assessment with young offenders : A meta-analysis of three assessment measures. Criminal Justice and Behavior, 36(4), 329–353. https://doi.org/10.1177/0093854809331457

40.

Olver

M. E.

Wong

S. C.

(2019). Offender risk and need assessment: Theory, research, and applications. In Devon

L. L.

Polaschek

A. D.

Hollin

C. R.

(Eds.), The Wiley international handbook of correctional psychology (pp. 461–475). John Wiley.

41.

Orton

L. C.

Hogan

N. R.

Wormith

J. S.

(2021). An examination of the professional override of the Level of Service Inventory–Ontario Revision. Criminal Justice and Behavior, 48(4), 421–441. https://doi.org/10.1177/0093854820942270

42.

Pusch

Holtfreter

(2018). Gender and risk assessment in juvenile offenders: A meta-analysis. Criminal Justice and Behavior, 45(1), 56–81. https://doi.org/10.1177/0093854817721720

43.

Rice

M. E.

Harris

G. T.

(2005). Comparing effect sizes in follow-up studies: ROC area, Cohen’s. D, and R. Law and Human Behavior, 29(5), 615–620. https://doi.org/10.1007/s10979-005-6832-7

44.

Rocque

Plummer-Beale

(2014). In the eye of the beholder? An examination of the inter-rater reliability of the LSI-R and YLS/CMI in a correctional agency. Journal of Criminal Justice, 42(6), 568–578. https://doi.org/10.1016/j.jcrimjus.2014.09.011

45.

Ruscio

(2003). Holistic judgment in clinical practice. The Scientific Review of Mental Health Practice, 2(1), 33–48.

46.

Saint-Louis

(2015). L’Inventaire des risques et des besoins liés aux facteurs criminogènes (IRBC): évaluation des propriétés métriques de l’instrument [Master’s thesis, University of Montréal]. Papyrus. https://papyrus.bib.umontreal.ca/xmlui/bitstream/handle/1866/13770/St-Louis_Sophie_2015_memoire.pdf?sequence=2&isAllowed=y

47.

Schmidt

Hoge

R. D.

Gomes

(2005). Reliability and validity analyses of the Youth Level of Service/Case Management Inventory. Criminal Justice and Behavior, 32(3), 329–344. https://doi.org/10.1177/0093854804274373

48.

Schmidt

Sinclair

S. M.

Thomasdóttir

(2016). Predictive validity of the Youth Level of Service/Case Management Inventory with youth who have committed sexual and non-sexual offenses: The utility of professional override. Criminal Justice and Behavior, 43(3), 413–430. https://doi.org/10.1177/0093854815603389

49.

Schwalbe

C. S.

(2007). Risk assessment for juvenile justice: A meta-analysis. Law and Human Behavior, 31(5), 449–462. https://doi:10.1007/s10979-006-9071-7

50.

Schwalbe

C. S.

(2008). A meta-analysis of juvenile justice risk assessment instruments: Predictive validity by gender. Criminal Justice and Behavior, 35(11), 1367–1381. https://doi.org/10.1177/0093854808324377

51.

Sreenivasan

Weinberger

L. E.

Frances

Cusworth-Walker

(2010). Alice in actuarial-land: Through the looking glass of changing Static-99 norms. Journal of the American Academy of Psychiatry and the Law Online, 38(3), 400–406.

52.

Storey

J. E.

Watt

K. A.

Jackson

K. J.

Hart

S. D.

(2012). Utilization and implications of the Static-99 in practice. Sexual Abuse, 24(3), 289–302. https://doi.org/10.1177/1079063211423943

53.

Vaswani

Merone

(2014). Are there risks with risk assessment? A study of the predictive accuracy of the Youth Level of Service-Case Management Inventory with young offenders in Scotland. British Journal of Social Work, 44(8), 2163–2181. https://doi.org/10.1093/bjsw/bct059

54.

Vieira

T. A.

Skilling

T. A.

Peterson-Badali

(2009). Matching court-ordered services with youths’ treatment needs: Predicting treatment success with young offenders. Criminal Justice and Behavior, 36, 385–401. https://doi.org/10.1177/0093854808331249

55.

Viglione

Rudes

D. S.

Taxman

F. S.

(2015). Misalignment in supervision: Implementing risk/needs assessment instruments in probation. Criminal Justice and Behavior, 42(3), 263–285. https://doi.org/10.1177/0093854814548447

56.

Viljoen

J. L.

Nicholls

T. L.

Cruise

K. R.

Desmarais

S. L.

Webster

C. D.

(2014). Short-term assessment of risk and treatability: Adolescent version (START:AV): User guide. Mental Health, Law, and Policy Institute.

57.

Viljoen

J. L.

Shaffer

C. S.

Muir

N. M.

Cochrane

D. M.

Brodersen

E. M.

(2019). Improving case plans and interventions for adolescents on probation: The implementation of the SAVRY and a structured case planning form. Criminal Justice and Behavior, 46(1), 42–62. https://doi.org/10.1177/0093854818799379

58.

Vincent

G. M.

Guy

L. S.

Grisso

(2012). Risk assessment in juvenile justice: A guidebook for implementation. Models for Change.

59.

Wormith

J. S.

Hogg

Guzzo

(2012). The predictive validity of a general risk/needs assessment inventory on sexual offender recidivism and an exploration of the professional override. Criminal Justice and Behavior, 39(12), 1511–1538. https://doi.org/10.1177/0093854812455741

60.

Yang

Wong

S. C.

Coid

(2010). The efficacy of violence prediction: A meta-analytic comparison of nine risk assessment tools. Psychological Bulletin, 136(5), 740–767. https://psycnet.apa.org/doi/10.1037/a0020473