The Reliability and Validity of the Measure of Criminal Attitudes and Associates and the Pride in Delinquency Scale in a Mixed Sex Sample of Justice-Involved Youth

Abstract

It is unclear if self-report measures of criminal attitudes and associates—developed and validated predominately on adult male offender samples—can or should be used with justice-involved girls. With a sample of 300 justice-involved youth (100 females, 200 males), this study examined the reliability and validity of the Measure of Criminal Attitudes and Associates (MCAA) and the Pride in Delinquency Scale (PIDS). Both measures evidenced reliability and convergent and discriminant validity in both sexes. The MCAA and the PIDS consistently predicted general recidivism for males (majority of area under the curve [AUCs; 7 out of 10] ranged from .60 to .68), but not for females (majority of AUCs [6 out of 8] ranged from .50 to .59). Both measures predicted violent recidivism across sex, with the majority of AUCs ranging from .60 to .67. More work is needed with exclusive female samples to better understand and operationalize criminal attitudes through a gender-informed lens.

Keywords

criminal attitudes gender recidivism youth

Following decades of androcentric correctional research, gender-responsive researchers (e.g., Gobeil, Blanchette, & Stewart, 2016) are making concerted efforts to engage in the explicit study of juvenile and adult female offenders. Important gender differences have been elucidated as a result; for example, female offenders have generally experienced significantly higher rates of childhood abuse and childhood adversity than their male counterparts (Bloom, Owen, & Covington, 2003; Vitopoulos, Peterson-Badali, Skilling, & Brown, 2018). Qualitative research findings have convincingly illustrated that for the vast majority of deeply entrenched justice-involved females, repeated forms of childhood abuse and neglect lead to maladaptive coping (e.g., substance abuse, internalizing mental health problems) and criminalized survival strategies (e.g., prostitution, drug use, and trafficking) culminating in incarceration (Belknap, 2015). Olver, Stockdale, and Wormith’s (2014) comprehensive meta-analytic review of the Level of Service/Case Management Inventory (Andrews, Bonta, & Wormith, 2004) and its variants has illustrated that although certain risk factors appear to predict criminal recidivism equally well for both genders (e.g., criminal history, criminal associates, and employment deficits), other domains are more salient predictors of crime for either males or females. Specifically, their results indicated that substance abuse and personal/emotional deficits are slightly stronger predictors of recidivism for females than males, whereas having an antisocial personality pattern is a more salient predictor for males. There is also mounting evidence that gender-responsive risk assessment tools and gender-responsive correctional programming designed specifically for female offenders can produce superior correctional outcomes to gender-neutral or inherently male-based approaches (Gobeil et al., 2016). Notwithstanding the prominence of criminal attitudes and criminal associates as criminogenic needs and treatment targets in the mainstream literature (Bonta & Andrews, 2017; Walters, 2012), researchers who study female offenders either have afforded little focus to criminal attitudes and/or associates or have instead focused almost exclusively on one specific type of criminal association—criminal romantic partners, or more broadly, unhealthy relationships (Kreis, Schwannauer, & Gillings, 2014). Consequently, research that explores how mainstream correctional treatment targets like attitudes and associates function in the context of female offending is warranted.

Criminal Attitudes and Associates, Gender, and Criminal Recidivism

Two influential meta-analyses published in the 1990s cemented the importance of criminal attitudes and associates as criminogenic treatment targets (Andrews et al., 1990; Gendreau, Little, & Goggin, 1996). Regrettably, these meta-analytic reviews did not perform analyses on samples disaggregated by sex. Simourd and Andrews (1994) addressed this issue in a correlational meta-analysis, illustrating that among adolescent offenders, a composite criminal attitudes/associates category comprised of 53 effect sizes correlated equally well with criminal behavior irrespective of sex (males: r = .40; females: r = .39).

Dowden and Andrews (1999) conducted the first female-focused correctional treatment outcome meta-analysis involving 26 studies, evaluating the extent to which the Risk-Need-Responsivity (RNR) framework generalized to female offender correctional outcomes. They concluded that the RNR framework is just as germane to female offenders as it is to males, with RNR-related reductions in recidivism commensurate with those observed among correctional programs delivered to male offenders adhering to the RNR principles (Bonta & Andrews, 2017).

Two additional published meta-analyses examined sex differences in attitudes and associates. Hubbard and Pratt (2002) found that criminal attitudes (r = .18, k = 3) and criminal associates (r = .53, k = 2) were predictors of female delinquency (but based on a very small number of individual effects). Olver et al.’s (2014) meta-analysis found that the Level of Service criminal attitudes domain score predicted general recidivism equally well for females (r = .23, k = 20) as males (r = .21, k = 35), as did criminal associates (females: r = .27, k = 20; males: r = .26, k = 37).

Collectively, these meta-analyses suggest that the global categories of criminal attitudes and associates are generally as important for female offenders as they are for male offenders. However, they do not address which (if any) specific elements of either global construct may be more or less relevant for one sex or the other. Furthermore, due to the absence of primary studies, the meta-analytic findings provide no insight into the applicability of male-developed (labeled “gender-neutral”) attitudinal/associate self-report measures to justice-involved females.

There is some preliminary evidence to suggest that attitudes overtly supportive of deviance are more predictive of criminal behavior for males than for females (Heimer, 1996). Indeed, qualitative interviews conducted with adolescent girls involved in the criminal justice system indicate that many of these individuals do not have a profound attachment to their delinquent activities and may even express disdain about being labeled delinquent (Chesney-Lind & Shelden, 2014). Moreover, current conceptualizations of the “antisocial attitudes” construct as defined in the gender-neutral literature may be neglecting attitudinal elements surrounding criminal conduct that are potentially unique and/or more salient to females, such as crime as a means of survival or to preserve a relationship (Bloom et al., 2003).

Self-Report Measures of Criminal Attitudes and Associates

Although most standardized risk/need assessment tools measure attitudes and associates, multimodal assessment achieved by augmenting interview-based risk assessment tools with self-report measures is an endorsed standard of practice in the field of correctional psychology (Hoge, 2012). Furthermore, there is evidence that among offender populations, self-report measures predict recidivism just as well as rater-based methods (Loza & Green, 2003).

Current Study

Drawing upon samples of both male (n = 200) and female (n = 100) justice-involved youth, this study explores the psychometric properties (the reliability [internal consistency] and validity [convergent, discriminant, and predictive]) of two self-report measures originally designed to assess criminal attitudes and associates among male offenders. Specifically, we examine the reliability and validity of the Pride and Delinquency Scale (PIDS; Shields & Whitehall, 1991), a measure of criminal attitudes originally developed for adolescent male offenders, and the Measure of Criminal Attitudes and Associates (MCAA; Mills, Kroner, & Forth, 2002), a measure of criminal attitudes and criminal associates originally developed for adult male offenders.

These measures were chosen because they have not been extensively validated among justice-involved youth and also because, collectively, they measure core constructs reflected in prominent attitudinal theories such as Walters’s (2012) criminal thinking styles (specifically—entitlement), Bandura’s (2002) construct of moral disengagement, Sykes and Matza’s (1957) related notion of neutralization techniques (rationalizations and justifications), and Ajzen’s (2011) classic theory of planned behavior. Importantly, the MCAA is one of the few measures of criminal associates that factors in both the density and frequency of criminal associations—a fundamental tenant of Sutherland and Cressey’s (1970) differential association theory. Finally, the PIDS is a strong measure of the extent to which an individual identifies with criminal others—one of three core criminal attitudes constructs identified by Bonta and Andrews (2017). Furthermore, the constructs and associated measures used to assess convergent and discriminant validity were selected using at least one the following criteria: (a) previously used to establish the validity of the MCAA and/or the PIDS, (b) extensively validated with youth populations, and (c) past research/theory predicts convergence (e.g., aggression and criminal attitudes) or discrimination (e.g., anxiety/depression and criminal attitudes). It is hypothesized that the MCAA and PIDS will be reliable and valid instruments for males. Specifically, both measures are expected to predict recidivism and demonstrate moderate to high correlations with concurrent constructs (e.g., the YLS Antisocial Orientation domain), convergent constructs (e.g., measures of alcohol and/or drug abuse, measures of aggression and externalizing behaviors), and small or nil correlations with discriminant constructs (e.g., self-esteem, self-efficacy, internalizing problems). However, given the androcentric development of both measures and the complete absence of female-specific data for either measure, no female-specific a priori hypotheses are advanced.

Method

Participants

Three hundred and forty consenting justice-involved youth (n_females = 113, 33.2%; n_males = 227, 66.8%) participated in the study. Twenty-eight of the original 340 participants (8.24% of the sample) did not consent to the questionnaire portion of the study and a further 12 participants (0.03% of the sample) either had not been released from custody (n = 6), did not meet the 3-year fixed follow-up criteria (n = 2), or had no available recidivism information (n = 4), resulting in a final sample size of 300 offenders (n_females = 100; 33.3%; n_males = 200; 66.7%). The resulting 300 participants were recruited from six different sites in Ontario, Canada, that service justice-involved youth. Youth were originally assessed at the following locations: probation (20.3%, n = 61), secure custody (50.7%, n = 152), open custody (4.7%, n = 14), or a mental health center mandated to provide court-ordered assessments (24.3%, n = 73). Notably, at the time of the assessment, 41.3% (n = 124) of the sample had been remanded to custody pending trial outcome or sentencing determination, while the remaining 58.7% (n = 176) had been formally adjudicated. Although there were no significant sex differences in the proportion of males versus females remanded to custody, significant sex differences emerged in terms of location, with more females than males being on probation or in open custody; in addition, more males than females had been assessed at the mental health center.

At the time of the original assessment, the sample ranged in age from 12 to 21 (M = 16.80, SD = 1.25); notably 71% of the sample was between the ages of 15 and 17. At follow-up, the sample ranged in age from 17 to 25 (M = 21.40, SD = 1.35). Age did not vary as a function of sex. The sample was ethnically diverse with participants self-identifying as Caucasian (46.30%, n = 139), African/Caribbean (29.70%, n = 89), mixed racial heritage (5.30%, n = 16), Indigenous (4.70%, n = 14), Middle/Far East (5.0%, n = 15), Asian (3.0% n = 9), and Hispanic (2.0% n = 6). Based on a collapsed ethnicity variable (i.e., Caucasian, African/Caribbean, other), a chi-square analysis revealed significant sex differences in ethnic distribution, with more Caucasian females than Caucasian males, and more males than females of African/Caribbean descent.

At the time of the original assessment, participants had been charged with and/or convicted of a variety of violent and nonviolent offenses: homicide-related (2%), serious violence (e.g., kidnapping/forcible confinement, robbery, assault; 54.3%), sex-related (9.3%), less serious person-related (e.g., criminal harassment, uttering threats; 11.4%), weapons-related (14.7%), property/theft/fraud-related (34.4%), drug trafficking (5.4%), drug possession (4.7%), and administration of justice/obstruction of justice (57.5%). Significant sex differences only emerged for sexual offenses (n_females = 1 [1.0%] vs. n_males = 27 [13.5%]; χ² = 12.31, p = .001, Phi = −.203) and weapons-related offenses (n_females = 4 [4.0%] vs. n_males = 40 [20.1%]; χ² = 13.75, p = .001, Phi = −.21). Overall, the study sample was a relatively serious and violent group of justice-involved youth (male and female alike) who are not representative of the larger youth-justice population in Canada; the typical justice-involved youth in Canada has committed a minor, nonviolent crime (e.g., theft) and has either been diverted from the justice system or has received a community-based sanction such as probation (Allen & Superle, 2016).

Measures

MCAA

The MCAA is comprised of two parts. In Part A, the Associates scale asks participants to identify up to four people with whom they spend the most time. Participants are also required to estimate how much of their time they spend with each person. Four questions are posed to assess the extent to which each identified associate is criminally active, asking (in a yes/no format) if each identified associate has ever committed a crime, been to jail, tried to involve the participant in a crime, or has a criminal record. The MCAA generates two measures of criminal associations: the Criminal Friend Index (CFI) and the number of criminal associates.

The CFI is calculated by assigning a number ranging from 1 to 4 to a percentage of time value for each of the four possible identified associates (i.e., 0%-25% = 1, 25%-50% = 2, 50%-75% = 3, and 70%-100% = 4). Next, for each identified associate, the recoded percentage of time value is multiplied by the number of times the respondent answered yes (yes = 1) to the four questions germane to the identified associate’s criminality, thus resulting in a plausible range of 0 to 16 for each associate. Summing the four product terms results in a final CFI score ranging from 0 to 64, with higher scores indicative of denser criminal associations.

Part B of the MCAA consists of a 46-item Attitude scale with four subscales: a 12-item Violence subscale, a 12-item Entitlement subscale, a 12-item Antisocial Intent subscale, and a 10-item Attitudes Toward Associates subscale. Items are rated dichotomously—agree (1) or disagree (0)—and then summed to produce a total overall attitude score ranging from 0 to 46, and four individual subscale scores ranging from 0 to 12 (0 to 10 in the case of the Attitudes Toward Associates subscale).

In the adult male development sample (Mills et al., 2002), the internal consistency of the subscales was acceptable (α = .80 to .84) except for Entitlement (α = .65), although a more recent validation study found acceptable internal consistency for this subscale (α = .80; Bäckström & Björklund, 2008). Table 2 illustrates comparable results in the current study. In terms of predictive validity, area under the curve (AUC) values ranged from .58 to .67 for the four attitude subscales in predicting general recidivism, and from .61 to .70 for the prediction of violent recidivism (Mills, Kroner, & Hemmati, 2004).

Pride in Delinquency Scale (PIDS)

The PIDS (Shields & Whitehall, 1991) is a 10-item scale that measures an individual’s level of comfort with engaging in crime by asking how proud or ashamed the respondent would feel engaging in specific criminal acts (e.g., beating up a child molester). Respondents reply on a scale ranging from −10 (indicative of extreme shame) to +10 (indicative of extreme pride). A constant of 100 is added to each PIDS score, resulting in a final score ranging from 0 to 200, with higher scores indicative of having greater pride in committing the hypothesized crimes. The convergent validity and reliability of the PIDS has been established (Skilling & Sorge, 2014), and the PIDS evidenced acceptable reliability in the present sample for both males and females (see Table 2).

Youth Level of Service–Case Management Inventory (YLS-CMI, 2.0)

The YLS-CMI (Hoge & Andrews, 2011), henceforth simply referred to as the YLS, is a tool developed to assess risk, criminogenic needs, and strengths in male and female adolescent offenders aged 12 to 18 years (Hoge & Andrews, 2011). The full YLS has eight subcomponents, each measuring a domain associated with criminality, with 42 items in total. Each item is scored as present (1) or absent (0). Three subcomponents of the YLS were used in the present study: Peer Relations (four items), Attitudes/Orientation (five items), and Personality/Behavior (seven items). The tool is scored based on an interview and file review, with higher scores indicating higher risk/needs within each subcomponent. Cronbach’s alphas for our study (Attitudes: α = .53 for males, α = .62 for females; Peers: α = .69 for males, α = .64 for females; Personality/Behavior: α = .63 for males, α = .58 for females) were comparable with the development sample.

Achenbach’s Youth Self-Report (YSR)

Achenbach’s YSR measure (Achenbach, 1991) is an extensively validated, 112-item self-report measure of emotional and behavioral problems developed for youth aged 11 to 18 years. The YSR can be collapsed into two overarching composite scales representing internalizing (21 items, assessing anxiety, depression, and somatic complaints) and externalizing (32 items, assessing rule-breaking and aggression) behaviors. Responses are recorded on a 3-point Likert-type scale on which participants rate the veracity of each statement as applied to them in the past 6 months (i.e., 0 = not true to 3 = very often). For the present sample, Cronbach’s alphas were acceptable for males (Internalizing: α = .87; Externalizing: α = 91) and females (Internalizing: α = .89; Externalizing, α = 90).

Alcohol and Drug Use Scales

The Alcohol Use Disorders Identification Test (AUDIT; Babor, Higgins-Biddle, Saunders, & Monteiro, 2001) is a well-validated, 10-item scale originally developed by the World Health Organization (WHO) that assesses three domains: Hazardous Alcohol Use (three items), Dependence Symptoms (three items), and Harmful Alcohol Use (four items). Responses are provided on a 5-point Likert-type scale (0 = never, 4 = 4 or more times weekly) based on the respondent’s experience with alcohol in the previous 12 months, with higher scores indicating a higher level of problematic drinking. The reliability for the present sample was high for both males (α = .88) and females (α = .89). The Drug Abuse Screening Test–Adolescents (DAST-A; Martino, Grilo, & Fehon, 2000) is a 27-item, commonly used and validated scale for adolescents adopting a binary yes/no response format for questions about drug use. Higher scores indicate a greater level of drug use problems. Reliability for the present sample was high for both males (α = .90) and females (α = .90).

Aggression Questionnaire (AQ)

The AQ scale (Buss & Warren, 2000) is a 34-item self-report questionnaire with demonstrated reliability and convergent validity. Each AQ item describes an aggression-related characteristic rated on a 5-point Likert-type scale (1 = not at all like me to 5 = completely like me). Higher scores are indicative of more aggression. Internal consistency for the AQ total scale was high in our sample (males: α = .95; females: α = .95).

Rosenberg Self-Esteem Scale (RSES)

The RSES (Rosenberg, 1965) is an extensively validated 10-item self-report measure of global self-esteem. It consists of 10 statements related to overall feelings of self-worth or self-acceptance. Each item is rated on a 4-point Likert-type scale ranging from strongly agree (3) to strongly disagree (0), with higher scores indicative of a greater level of self-esteem. Internal consistency in our sample was high (males: α = .84; females: α = .87).

General Self-Efficacy Scale (GSES)

The GSES (Bosscher & Smit, 1998) is a 12-item short version developed from the original 17-item General Self-Efficacy Scale (Sherer, Maddux, Mercandante, & Rogers, 1982) with well-documented reliability and validity. Each item is rated on a 5-point Likert-type scale (1 = strongly agree to 5 = strongly disagree). Higher scores are associated with greater degrees of self-reported persistence, initiative, and effort. Internal consistency in the current sample was high for males (α = .84) and females (α = .86).

Recidivism

Recidivism was coded using electronic files obtained from the Ministry of Community Safety and Correctional Services (Ontario) and the Ministry of Children and Youth Services (Ontario), as well as hard copy versions of Canadian Police Information Centre (CPIC) files from the Royal Canadian Mounted Police (RCMP). The RCMP database captures criminal convictions occurring anywhere in Canada, whereas the Ontario Ministry databases only capture convictions incurred in the province of Ontario. By considering both youth (i.e., Ministry of Children and Youth Services) and adult conviction data sources (i.e., Ministry of Community Safety and Correctional Services, and the RCMP), recidivism counts include both youthful (under 18 years of age) and adult (18 years of age and above) convictions. Using these combined sources, general recidivism was defined as any new criminal conviction (excluding technical violations; dichotomously coded: 0 = no, 1 = yes) occurring within a 3-year fixed follow-up period. Violent recidivism (dichotomously coded: 0 = no, 1 = yes) included the following convictions involving threats or actual harm to personal safety: homicide-related, robbery, assault, any sexual offenses, threats, kidnapping, weapons-related, or arson.

Procedures

After all necessary ethical and legal clearances were obtained, each consenting youth participated in a semistructured interview (with a trained student) of approximately 2 to 3 hr in length and completed a battery of self-report questionnaires of approximately 1 hr in length. The researchers also accessed official file information while on site for collateral scoring to complete a demographic and offense coding manual. Based on their level of participation, youth were compensated up to CAD$30.00 in gift cards and/or canteen funds.

Results

The percentage of missing data ranged from 3.3% for the YLS subscales to 30.3% for the AQ total score. A further inspection of the nature and pattern of missing data indicated that the data were most likely missing at random (MAR). Thus, multiple imputation was deemed the appropriate analytic strategy for addressing missing data (Allison, 2002). Specifically, pooled statistical estimates were generated for 25 imputed datasets using the multiple imputation feature available in SPSS Version 25.0. In terms of interpreting statistical findings, we prioritized magnitude of effect sizes over statistical significance given that males outnumbered females (2:1 ratio) in our sample, coupled with the problems associated with statistical significance testing (Singh, 2017).

Descriptives

As Table 1 illustrates, no substantial gender differences emerged with regard to criminal attitudes when measured via the MCAA or the PIDS. However, males did evidence slightly higher levels of criminal attitudes when measured using the YLS Attitude domain, albeit the effect size (d = .19) is considered small (Cohen, 1988). Female youth did evidence slightly more problems with criminal peers when measured by the MCAA, with effect sizes falling in the small (.11 < d < .35) to moderate range (.36 < d < .65; Cohen, 1988); however, no noticeable gender differences emerged for the YLS Peer domain.

Table 1:

Descriptive Statistics by Gender

Measure	Males(n = 200)^a	Females(n = 100)^a	t test	Cohen’s d^b
Measure	M (SD)	M (SD)	t test	Cohen’s d^b
MCAA (total)	25.57 (8.95)	25.18 (8.64)	0.35	0.04
Violence	6.28 (3.11)	5.77 (3.13)	1.31	0.16
Entitlement	6.77 (2.50)	6.39 (2.44)	1.23	0.15
Antisocial intent	5.86 (3.33)	5.96 (3.24)	−0.26	−0.03
Associates	6.66 (2.65)	7.07 (2.39)	−1.28	−0.16
Criminal peers	1.84 (1.42)	2.28 (1.25)	−2.57**	−0.32
Criminal Friend Index	10.58 (10.69)	14.26 (9.26)	−2.89**	−0.36
PIDS (total)	80.50 (41.18)	76.26 (42.32)	0.81	0.10
YLS (total)	20.46 (8.12)	19.99 (7.52)	0.75	0.06
Attitudes	2.00 (1.35)	1.74 (1.34)	1.62	0.19
Peers	2.57 (1.29)	2.61 (1.24)	−0.27	−0.03
Substance abuse	2.40 (1.69)	2.31 (1.71)	0.44	0.05
Personality/behavior	3.75 (1.89)	3.94 (1.67)	−0.86	−0.11
AQ	94.04 (25.06)	102.42 (24.27)	−2.71**	−0.34
YSR (internal tot)	14.99 (10.30)	22.42 (12.87)	−5.05***	−0.66
YSR (external tot)	23.13 (11.51)	27.12 (10.96)	−2.88**	−0.35
AUDIT	8.58 (8.29)	10.34 (8.54)	−1.65	−0.21
DAST	8.34 (6.80)	9.79 (7.18)	−1.63	−0.21
RSES	20.81 (4.76)	18.81 (5.59)	2.11**	0.40
GSES	42.44 (7.20)	40.42 (8.04)	2.02*	0.27

Note. MCAA = Measure of Criminal Attitudes and Associates; PIDS = Pride in Delinquency Scale; YLS = Youth Level Service; AQ = Aggression Questionnaire; YSR = Youth Self-Report Questionnaire; AUDIT = Alcohol Use Disorders Identification Test; DAST = Drug Abuse Screening Test; RSES = Rosenberg Self-Esteem Scale; GSES = General Self-Efficacy Scale.

n = full sample based on pooled correlations from 25 imputed datasets. ^bNegative d values are indicative of females scoring higher; positive d values are indicative of males scoring higher.

p < .05. **p < .01. ***p < .001.

Female youth scored substantially higher than their male counterparts in regard to internalizing problems (d = .66). Females also evidenced slightly more problems with both alcohol and drug abuse when measured using the AUDIT and the DAST; however, both corresponding effect sizes were small (d = −.21). No noticeable gender differences emerged when substance abuse was measured using the YLS. Notably, female youth scored higher than male youth for externalizing problems (d = −.35). Finally, males evidenced higher levels of self-esteem (d = .40) and self-efficacy (d = .27) relative to females.

Recidivism

Fifty-eight percent of the sample was reconvicted for at least one criminal offense during the 36-month fixed follow-up and 42.7% was reconvicted for a violent offense. Males were more likely than females to recidivate both generally (63.5% vs. 47%, χ² = 7.45, p = .006) and violently (48.5% vs. 31%, χ² = 8.35, p = .004). However, the magnitude of this gender difference was small, with Phi = −.16 and −.17, respectively.

Reliability of MCAA and PIDS

As Table 2 illustrates, the MCAA Attitude total and subscale scores demonstrated acceptable internal consistency for both genders, with Cronbach’s alphas comparable with those found in the adult male development sample (Mills et al., 2002). Notably, the Entitlement subscale alphas were somewhat lower than those associated with the remaining MCAA subscales. The PIDS demonstrated strong internal consistency for both males and females, with all alpha values exceeding .87.

Table 2:

Cronbach’s Alphas for the MCAA and the PIDS

Measure (no. of items)	Males	Females
Measure (no. of items)	Alpha (n)^a	Alpha (n)^a
MCAA (46)	.91 (153)	.90 (76)
Violence (12)	.83 (163)	.83 (80)
Entitlement (12)	.65 (169)	.63 (88)
Antisocial intent (12)	.85 (172)	.83 (87)
Associates (10)	.82 (174)	.78 (91)
PIDS (10)	.88 (181)	.87 (90)

Note. MCAA = Measure of Criminal Attitudes and Associates total score; PIDS = Pride in Delinquency Scale.

n = SPSS does not perform multiple imputation for reliability analyses; hence, n fluctuates due to missing data.

Irrespective of sex, mean item-total correlations were above the recommended threshold of .30 for all subscales (Nunnally & Bernstein, 1994), with one exception: the mean item-total correlation for the Entitlement subscale was slightly below .30 for males and females (see Table 3). With few exceptions, mean interitem correlations were generally all within the recommended range of .10 to .30 (Nunnally & Bernstein, 1994). No sex differences were noted.

Table 3:

Mean Item-Total and Interitem Correlations for the MCAA Attitude Scales and PIDS

Measure (no. of items)	Males		Females
Measure (no. of items)	Mean item total r (n)^a	Mean interitem r (n)^a	Mean item total r (n)^a	Mean interitem r (n)^a
MCAA total (46)	.40 (153)	.18 (153)	.37 (76)	.16 (76)
Violence (12)	.48 (163)	.28 (163)	.49 (80)	.29 (80)
Entitlement (12)	.29 (169)	.13 (169)	.27 (88)	.12 (88)
Antisocial (12)	.52 (172)	.32 (172)	.48 (87)	.28 (87)
Associates (10)	.50 (174)	.31 (174)	.46 (91)	.28 (91)
PIDS (10)	.61 (181)	.42 (181)	.60 (90)	.41 (90)

Note. MCAA = Measure of Criminal Attitudes and Associates; PIDS = Pride in Delinquency Scale.

n = SPSS does not perform multiple imputation for reliability analyses; hence, n fluctuates due to missing data.

Intercorrelations between the MCAA Subscales and the PIDS

As Table 4 illustrates, the intercorrelations between all of the MCAA total and subscales were in the small to large range (rs ranging between .13 and .87), irrespective of sex. Notably, the PIDS demonstrated small to moderate correlations with all of the MCAA subscales (rs ranging between .25 and .63). Once again, the pattern of correlations was similar for both male and female youth.

Table 4:

MCAA and PIDS Intercorrelations

Variable	1	2	3	4	5	6	7	8
Males (n = 200)^a
1. MCAA^b total	—
2. Violence	.79***	—
3. Entitlement	.65***	.42***	—
4. Antisocial	.85***	.54***	.38***	—
5. Associates	.77***	.44***	.29***	.64***	—
6. Criminal peers	.47***	.20**	.18**	.51***	.54***	—
7. CFI	.51***	.26***	.21**	.55***	.54***	.78***	—
8. PIDS	.63***	.42***	.32***	.61***	.56***	.44***	.46***	—
Females (n = 100)^a
1. MCAA^b total	—
2. Violence	.76***	—
3. Entitlement	.74***	.43***	—
4. Antisocial	.87***	.55***	.52***	—
5. Associates	.68***	.26**	.37***	.55***	—
6. Criminal peers	.40***	.29**	.13	.38***	.45***	—
7. CFI	.45**	.29**	.25*	.37***	.50***	.77***	—
8. PIDS	.42***	.27**	.25**	.40***	.37***	.27**	.26**	—

Note. MCAA = Measure of Criminal Attitudes and Associates; PIDS = Pride in Delinquency Scale; CFI = Criminal Friend Index.

n = full sample based on pooled correlations from 25 imputed datasets. ^bMCAA = total Attitudes score.

p < .05. **p < .01. ***p < .001.

Convergent and Discriminant Validity of the MCAA and PIDS

The convergent validity of both the MCAA and the PIDS was examined using the following measures: the YLS (Attitudes, Peers, and Personality domains), the YSR (total Externalizing score), the DAST, AUDIT, and the AQ (see Table 5). All convergent validity correlations were generally small to moderate for both sexes with the exception of the correlation between the YLS Personality domain and Entitlement for males (r = .08). In addition, for females, no significant correlations emerged between self-reported aggression and the PIDS nor the Personality domain of the YLS and the MCAA’s Associates measures. Both the MCAA and PIDS demonstrated acceptable discriminant validity given that the majority of correlations were either negative or did not exceed .10 (see Table 6).

Table 5:

Convergent Validity: Correlations of the MCAA and PIDS With Theoretically Relevant Measures

Measure	MCAA	Violence	Entitlement	Intent	Associates	No. of peers	CFI	PIDS
Males (n = 200)^a
YLS Attitude	.37***	.23**	.19**	.34***	.37***	.36***	.37***	.34***
YLS Peers	.40***	.22***	.15*	.39***	.46***	.37***	.45***	.41***
YLS Per	.28**	.20**	.08	.25**	.33**	.29**	.27**	.26**
Extern	.49***	.31***	.22***	.47***	.48***	.43***	.48***	.47***
DAST	.36***	.20**	.10*	.34***	.46***	.32***	.39***	.36***
AUDIT	.42***	.31***	.20**	.37***	.43***	.28***	.37***	.31***
AQ	.50***	.37***	.25***	.46***	.45***	.36***	.37***	.48***
Females (n = 100)^a
YLS Attitude	.51***	.45***	.41***	.40***	.28**	.24*	.20*	.28**
YLS Peers	.43***	.22**	.32**	.37***	.44***	.27**	.28**	.28**
YLS Per	.35**	.32**	.24*	.29**	.19	.13	.09	.25*
Extern	.54***	.44***	.36***	.43***	.40***	.23*	.25**	.29**
DAST	.25*	.05	.17	.25**	.32**	.22*	.24*	.25*
AUDIT	.44***	.28**	.28**	.41***	.38***	.23*	.20*	.29**
AQ	.46***	.43***	.34***	.37***	.27**	.22*	.18*	.15

Note. MCAA = Measure of Criminal Attitudes and Associates, total attitudes score; PIDS = Pride in Delinquency Scale; CFI = Criminal Friend Index; YLS = Youth Level of Service; YLS Per = YLS Personal; Extern = Externalizing Symptoms subscale of the Youth Self-Report Questionnaire; DAST = Drug Abuse Screening Test; AUDIT = Alcohol Use Disorders Identification Test; AQ = Aggression Questionnaire, total score.

n = full sample based on pooled correlations from 25 imputed datasets.

p < .05. **p < .01. ***p < .001.

Table 6:

Discriminant Validity: Correlations Between the MCAA Attitudes, PIDS, and Theoretically Nonrelevant Measures

Measure	MCAA	Violence	Entitlement	Antisocial	Associates	PIDS
Males (n = 200)^a
YSR internalizing	.14*	.11	.10	.09	.13*	.17*
RSES	−.15*	−.17*	−.05	−.14*	−.07	−.15*
GSES	−.09	−.11	−.02	−.12	−.01	−.08
Females (n = 100)^a
YSR internalizing	.003	.02	−.02	−.06	.09	−.03
RSES	−.02	.04	−.03	−.04	−.04	.05
GSES	−.22*	−.10	−.18	−.21*	−.19*	−.21*

Note. MCAA = Measure of Criminal Attitudes and Associates, total score; PIDS = Pride in Delinquency Scale; YSR = Youth Self-Report; RSES = Rosenberg Self-Esteem Scale; GSES = General Self-Efficacy Scale.

n = full sample based on pooled correlations from 25 imputed datasets.

p < .05.

Predictive Criterion Validity of the MCAA and PIDS

The AUC is an accuracy statistic that indicates the probability that given a randomly selected recidivist and nonrecidivist, the recidivist will score higher than the nonrecidivist on the predictor variable. The AUC provides an effect size, such that a value of .56 corresponds to a Cohen’s d of .20 (small effect), a .64 corresponds to a d of .50 (medium effect), and a .71 corresponds to a d of .80 (large effect; Rice & Harris, 2005). While Rice and Harris (2005) use the term medium effect to describe AUCs between .64 and .70, we use the term modest.

As Table 7 illustrates, most MCAA subscales and the PIDS performed adequately in predicting general recidivism among male youth (all AUCs ranged from .60 to .68, except for Entitlement, generating an AUC of .57). In contrast, for females, the majority of corresponding AUCs were considerably lower, ranging from .51 to .59. However, as with the male sample, the MCAA total score (AUC = .64) and the MCAA Violence subscale (AUC = .68) emerged as modest predictors of general recidivism among females. A series of formal, pairwise tests (Hanley & McNeil, 1983) comparing male AUCs with corresponding female AUCs yielded no statistically significant sex differences at the p < .05 level; however, nonoverlapping confidence intervals (CIs) were evident for the CFI, number of Criminal Peers, and the PIDS, indicating meaningfully stronger indices of prediction for males. In general, the rater-based YLS Antisocial Orientation domain performed on par with the self-reported MCAA total score and Violence subscale, although associated levels of predictive validity were slightly higher for females.

Table 7:

Criterion Predictive Validity: Receiver Operating Characteristic AUC for General and Violent Recidivism

Measure	Males (n = 200)^a AUC [95% CI]	Females (n = 100)^a AUC [95% CI]
	General recidivism
MCAA	.68 [.66, .69]	.64 [.62, .66]
Violence	.68 [.67, .70]	.68 [.66, .70]
Entitlement	.57 [.56, .59]	.59 [.56, .61]
Antisocial Intent	.65 [.64, .67]	.59 [.57, .61]
Associates	.62 [.60, .64]	.58 [.56, .60]
CFI	.63 [.62, .65]	.54 [.52, .56]
Number of Criminal Peers	.60 [.59, .62]	.51 [.49, .54]
PIDS	.65 [.63, .66]	.59 [.57, .62]
YLS Attitudes domain	.68 [.60, .76]	.69 [.58, .79]
	Violent recidivism
MCAA	.62 [.60, .63]	.67 [.65, .70]
Violence	.60 [.59, .62]	.67 [.64, .69]
Entitlement	.58 [.56, .60]	.62 [.59, .64]
Antisocial Intent	.61 [.59, .62]	.64 [.61, .66]
Associates	.57 [.56, .59]	.59 [.57, .62]
CFI	.60 [.58, .61]	.63 [.60, .65]
Number of Criminal Peers	.55 [.53, .56]	.54 [.52, .57]
PIDS	.62 [.60, .63]	.61 [.58, .63]
YLS Attitudes domain	.63 [.52, .75]	.63 [.52, .75]

Note. AUC = area under the curve; CI = confidence interval; MCAA = Measure of Criminal Attitudes and Associates, Total Score; CFI = Criminal Friend Index; PIDS = Pride in Delinquency Scale; YLS = Youth Level Service Inventory.

SPSS’s multiple imputation algorithm does not pool AUC values, thus meta-analytic procedures were used (SPSS syntax provided by K. Babchishin, personal communication, November 13, 2017).

A different pattern of results emerged with regard to violent recidivism. The majority of the MCAA subscales predicted violent recidivism more strongly for females than males, albeit no pairwise statistically significant sex differences emerged (at the p < .05 level) and the 95% CIs overlapped in most cases with the exception of the MCAA total and Violence subscale, suggesting that these scales are actually better predictors of violent recidivism for females than males. The PIDS performed adequately and to the same degree for both males and females and no statistically significant sex differences emerged (at the p < .05 level). Notably, once again for the males, the MCAA total score and Violence subscale performed on par with YLS Antisocial Orientation domain; however, for females, the MCAA total score and Violence subscale slightly outperformed the YLS, albeit with overlapping CIs evident.

Incremental Predictive Criterion Validity of the MCAA and the PIDS

The ability of the MCAA and the PIDS to add incrementally to the prediction of recidivism above static criminal history was also assessed. A binary logistic regression analysis was conducted to predict general recidivism among males. Simultaneous entry of the following four predictors was performed: the MCAA’s Violence subscale (Attitude Towards Violence [ATV]; the strongest univariate MCAA attitudinal predictor), the MCAA’s CFI, the PIDS total score, and the YLS Criminal History domain. Only the MCAA’s Violence subscale (odds ratio [OR] = 1.22, p = .002, 95% CI = [1.10, 1.39]) and the YLS Criminal History domain (OR = 1.52, p = .001, 95% CI = [1.22, 1.90]) added incrementally to the prediction of general recidivism; the PIDS (OR = 1.00, p = .66, 95% CI = [0.99, 1.01]) and the CFI did not (OR = 1.01, p = .63, 95% CI = [0.97, 1.05]). Interestingly, a parallel analysis among the female subsample evidenced only one significant predictor—the MCAA Violence subscale (OR = 1.23, p = .01, 95% CI = [1.05, 1.44]); the remaining predictors were not significant—YLS Criminal History (OR = 1.21, p = .213, 95% CI = [0.90, 1.65]), PIDS (OR = 1.00, p = .515, 95% CI = [0.99, 1.02]), and CFI (OR = 0.987, p = .639, 95% CI = [0.94, 1.04]). However, the relatively smaller sample of females may have contributed to the nonsignificant finding for the YLS Criminal History. Further analyses using violent recidivism as the outcome variable revealed that only YLS Criminal History predicted violent recidivism among males (OR = 1.27, p = .01, 95% CI = [1.05, 1.53]); the remaining ORs were nonsignificant; for females, no variables emerged as significant predictors, although the MCAA Violence subscale approached statistical significance (OR = 1.16, p = .08, 95% CI = [0.982, 1.36]).

Discussion

Drawing upon a sample of 300 justice-involved youth (100 females, 200 males), this study examined the reliability and validity of two purportedly gender-neutral self-report measures of criminal attitudes and associates—the MCAA (Mills & Kroner, 1999) and the PIDS (Shields & Whitehall, 1994). The MCAA evidenced strong internal consistency across all attitudinal subscales for both sexes, with one exception. The Entitlement subscale evidenced relatively weak internal consistency, a finding congruent with the original development sample (Mills & Kroner, 1999). The MCAA total attitudinal score and all four attitudinal subscales demonstrated strong discriminant validity (for both sexes), yielding negative, nil, or very small positive correlations with measures of generalized internalizing symptoms, self-esteem, and general self-efficacy. As expected, the convergent validity of the MCAA was established for males with moderate to high correlations emerging throughout. For the most part, parallel results emerged for the females; however, the MCAA Violence subscale was not related to drug abuse and the MCAA’s CFI was not related to the YLS Personality domain, potentially suggesting gendered associations between violence-supportive attitudes, drug abuse, and personality features.

More nuanced findings emerged in regard to predictive validity. The strongest predictive validity for both types of recidivism was found with the MCAA total score and the Violence subscale. Although speculative, it is possible that the Violence subscale performed particularly well for this sample of females given their higher than normal prevalence of violent index offenses and self-reported aggression (relative to the typical justice-involved female youth who has most likely engaged in nonviolent crimes; see Allen & Superle, 2016). Noteworthy, the MCAA total and Violence subscales essentially predicted both general and violent recidivism as well as the rater-based YLS Antisocial Orientation domain for both sexes, thereby providing further evidence in support of self-appraisal measures in the context of recidivism prediction. Interestingly, attitudes toward violence emerged as a predictor of general recidivism in both sexes while controlling for criminal history, attesting to the importance of this dynamic construct in the criminal reoffending process.

The PIDS demonstrated internal consistency and discriminant and convergent validity in both sexes; for females, however, the aggression scale score did not correlate with the PIDS, potentially suggesting greater heterogeneity among female-perpetrated violence—being either expressive (arguably captured by the aggression scale) or instrumental (arguably captured by the PIDS). It has also been suggested that female aggression is qualitatively different from that exhibited by males (Morash, Kashy, Cobbina, & Smith, 2018). In our study, the convergent validity analysis involving the PIDS illustrated that although the PIDS did not correlate with measures of self-reported aggression among females, it did correlate with measures of self-reported aggression among males. Tapping into unique predictors of aggression for females requires a more gendered methodological approach, with future research examining whether aggression-related crimes committed by females are more deeply rooted in trauma than these crimes are for males.

The PIDS predicted violent recidivism modestly well and to the same degree in both genders, but for general recidivism, the PIDS predicted modestly for males but less so for females. Pride in criminal behavior may be a more male-specific facet of criminal attitudes in the context of general criminality, possibly because pride in delinquent acts could support machoism in male criminal identities. This hypothesis is consistent with the finding that males score higher than females on masculinity measures (Zell, Krizan, & Teeter, 2015). Conversely, like their male counterparts, pride in criminal behavior may be salient for some high risk female youth but only in the context of violence.

Contrary to the gender-responsive hypothesis that justice-involved females tend to score lower on indices of criminal attitudes than their male counterparts (see Morash et al., 2018), no meaningful sex differences in mean scores emerged for the MCAA or the PIDS. Noteworthy, on the MCAA total score, both males and females evidenced a mean score equal to the 91st percentile reported in the adult male development sample (Mills & Kroner, 1999). Potentially reflective of the higher risk nature of our youth sample, this finding may also explain the MCAA’s somewhat lower predictive validity in the current context relative to its performance in adult male samples. The difference between youth and adults could also be explained by adolescents actually having greater antisocial attitudes and more criminal associates than adults, consistent with age-crime curves, showing criminal behavior peaking during adolescence for both males and females (D’Unger, Land, & McCall, 2002). We underscore the importance of developing population-specific norms and attending to developmental differences in criminal attitudes.

Another trend was the observation that the females either (a) evidenced elevated symptomatology on measures that assessed internalizing and externalizing problems or (b) scored lower on hypothesized protective factors—namely, self-esteem and self-efficacy. These findings confirm what gender-responsive theorists have repeatedly argued: that justice-involved girls and women present with particularly high levels of mental health and addiction-related needs (Bloom et al., 2003) compared with their male counterparts.

A gender-responsive perspective on the assessment of criminal associations emphasizes the contribution of unhealthy relationships with romantic intimate partners to criminal conduct. In the context of our study, all three MCAA indices of criminal association rendered strong convergent and discriminant validity irrespective of sex, with mixed results in terms of predictive validity. It is noteworthy that although the Cohen’s d results consistently demonstrated higher levels of criminal peer problems among females than among males, these gender differences in prevalence did not translate into criminal peers being a stronger predictor of recidivism for females. This finding is an important reminder that high prevalence of a problem is not synonymous with predictive criterion validity (Kraemer et al., 1997).

With respect to peer associations, relational-cultural theorists maintain that females are socialized to attach ultimate value to relationships (Jordan, 2010). As such, it is somewhat counter-intuitive that a global measure of antisocial associates should produce stronger criminogenic effects among males compared with females. However, our pattern of findings is consistent with past research (e.g., Piquero, Gover, MacDonald, & Piquero, 2005), indicating that antisocial peers are more predictive of delinquency in males than in females. Furthermore, subsequent empirical studies have supported the contention that while criminal peers are important predictors of recidivism in males, having a criminal romantic partner is a more potent predictor of crime among females (e.g., Benda, 2005). Notwithstanding the Benda (2005) study, additional research examining intimate relationships as predictors of recidivism for females rendered inconclusive results (Kreis et al., 2014), although Kreis et al. underscore that relationship factors may interact with other factors (e.g., depression, anxiety) in predicting recidivism in females.

To achieve a sample size comprised of a sufficient number of female youth to render meaningful gender comparisons, it was necessary to employ a sampling strategy that resulted in a mixed sample of justice-involved youth from community and custodial settings who had either been formally adjudicated or remanded to custody. This sample heterogeneity may diverge from extant published studies but was nonetheless necessary given the primary focus of the research—to assess gender similarities and differences in a meaningful way. To reiterate, our sample was not representative of the general youth-justice population in Canada (Allen & Superle, 2016), though our results do reveal important information regarding a small, yet significant segment of the youth-justice population that, although rarely studied, will potentially cost Canadian society roughly CAD$16 million dollars over the long term (Day & Koegl, 2015). Indeed, very few quantitative studies have compared serious adolescent female offenders with their adolescent male counterparts (see Lanctôt, 2018, for a notable exception). The goal of our study was to assess the psychometric properties of two, decidedly male-based conceptualizations of criminal attitudes; thus, we could not capture nuanced gender differences in the nature or expression of criminal attitudes beyond the item content reflected in our chosen measures. Also, despite the conceptual richness of our measures, they did not capture rejection of convention—a core attitudinal component articulated in the correctional literature (Bonta & Andrews, 2017).

Moving forward, gender-based validation research on both the MCAA and the PIDS is critical. First, gender-informed construct validation research with sufficiently powered samples is needed for both measures—including their underlying factor structure—given that developing a solid foundation for correctional intervention requires identifying the specific attitudes or thinking patterns most germane to criminal conduct (Mitchell & Tafrate, 2012). Multimethod testing to address the possible effects of method factors (e.g., item wording) on convergent validity (see Bagozzi & Yi, 1991) should also be conducted by sex. In addition, it is arguably necessary to disaggregate these larger constructs to appreciate sex differences in their criminogenic impact. Second, concurrent testing of gender-neutral measures with female-specific variables such as poverty, abuse, and relational items is essential to ensure that theorized female needs are fully represented in correctional assessment and research.

Third, multiwave, longitudinal research—both qualitative and quantitative—is needed to disentangle temporal ordering effects. Specifically, when and how do certain justice-involved youth develop criminal attitudes? Do criminal attitudes play a larger role in the onset or maintenance of criminal conduct? How does gender moderate these potential temporal ordering effects? In addition, more research regarding the prevalence of criminal attitudes among the more serious and criminally entrenched female offenders versus typical justice-involved females is warranted.

In sum, clinical implications garnered from this study indicate that attitudes and associates appear to be valid criminogenic treatment targets for both male and female youth, echoing findings of past research. Importantly, our results suggest that until truly gender-informed scales are developed (i.e., using development samples of girls and women) and validated, the MCAA total and Violence subscales can be used with both justice-involved girls and boys to augment rater-based risk assessment and treatment planning. However, more research is required in regard to the remaining MCAA’s subscales with justice-involved youth, particularly justice-involved females, before firm practice recommendations can be made; most notable is the poor performance of the MCAA’s CFI among our female sample. Our results suggest that the PIDS can be used with justice-involved boys to assess pride in antisocial behaviors and to assess risk to reoffend. More PIDS-focused research is required among justice-involved females to render conclusive practice statements, although our results do suggest this measure may be useful in the prediction of violent reoffending among high risk girls.

Footnotes

Authors’ Note:

The authors thank the youth, justice organizations, and research assistants who made this research possible. This research was supported by a grant from the Social Sciences and Humanities Research Council (SSHRC, 410-2009-1020).

Heidi R. O’Hagan is a registered psychotherapist (qualifying) and a graduate of the Master of Arts program in the Department of Psychology, Carleton University, with a focus on forensic psychology and research involving adolescents, gender differences, and crime.

Shelley L. Brown is an associate professor with the Department of Psychology at Carleton University. Her research interests include risks and strengths, complex trauma, and violence among justice-involved females.

Natalie J. Jones is a senior evaluator at the Social Sciences and Humanities Research Council (SSHRC) of Canada and adjunct research professor in the Department of Psychology at Carleton University. Her primary research interests lie in the development of strengths-based and gender-informed risk assessment and intervention strategies for justice-involved populations.

Tracey A. Skilling is a psychologist and clinician scientist at the Centre for Addiction and Mental Health and an assistant professor in the Department of Psychiatry at the University of Toronto with a graduate appointment in the Department of Applied Psychology and Human Development. Her research interests include antisocial behavior, mental health issues, substance use in adolescents, and risk assessment issues in both male and female young offenders. Her current projects include studies evaluating risk assessment practices and understanding the desistance process with adolescent male and female offenders.

References

Achenbach

T. M.

(1991). Manual for the Youth Self-Report and 1991 profile. Burlington, VT: Department of Psychiatry, University of Vermont.

Ajzen

(2011). The theory of planned behavior: Reactions and reflections. Psychology & Health, 26, 1113-1127. doi:10.1080/08870446.2011.613995

Allen

M. K.

Superle

(2016). Youth crime in Canada, 2014. Ottawa, Ontario, Canada: Statistics Canada. Retrieved from https://www.statcan.gc.ca/pub/85-002-x/2016001/article/14309-eng.htm

Allison

P. D.

(2002). Missing data: Sage University papers series on quantitative applications in the social sciences. Thousand Oaks, CA: SAGE.

Andrews

D. A.

Bonta

Wormith

S. J.

(2004). Level of Service/Case Management Inventory: An offender assessment system. Toronto, Ontario, Canada: Multi-Health Systems.

Andrews

D. A.

Zinger

Hoge

R. D.

Bonta

Gendreau

Cullen

F. T.

(1990). Does correctional treatment work? A psychologically informed meta-analysis. Criminology, 28, 369-404. doi:10.1111/j.1745-9125.1990.tb01330.x

Babor

T. F.

Higgins-Biddle

J. C.

Saunders

J. B.

Monteiro

M. G.

(2001). The Alcohol Use Disorders Identification Test: Guidelines for use in primary care (2nd ed.). Geneva, Switzerland: World Health Organization.

Bäckström

Björklund

(2008). The Measures of Criminal Attitudes and Associates (MCAA). Criminal Justice and Behavior, 35, 1398-1410. doi:10.1177/0093854808322239

Bagozzi

R. P.

(1991). Multitrait-multimethod matrices in consumer research. Journal of Consumer Research, 17(4), 426-439.

10.

Bandura

(2002). Selective moral disengagement in the exercise of moral agency. Journal of Moral Education, 31, 109-119. doi:10.1080/0305724022014322

11.

Belknap

(2015). The invisible woman: Gender, crime and justice (4th ed.). Stamford, CT: Cengage Learning.

12.

Benda

B. B.

(2005). Gender differences in life-course theory of recidivism: A survival analysis. International Journal of Offender Therapy and Comparative Criminology, 49, 325-342. doi:10.1177/0306624X04271194

13.

Bloom

Owen

Covington

S. S.

(2003). Gender-responsive strategies for women offenders: Research, practice, and guiding principles for women offenders (NIC Accession Number 018017). Retrieved from https://nicic.gov/gender-responsive-strategies-research-practice-and-guiding-principles-women-offenders

14.

Bonta

Andrews

D. A.

(2017). The psychology of criminal conduct (6th ed.). New York, NY: Routledge.

15.

Bosscher

R. J.

Smit

J. H.

(1998). Confirmatory factor analysis of the General Self-Efficacy Scale. Behavior Research and Therapy, 36, 339-343. doi:10.1016/S0005-7967(98)00025-4

16.

Buss

A. H.

Warren

W. L.

(2000). Aggression Questionnaire (AQ): Manual. Los Angeles, CA: Western Psychological Services.

17.

Chesney-Lind

Shelden

R. G.

(2014). Girls, delinquency, and juvenile justice (4th ed.). Chichester, UK: John Wiley.

18.

Cohen

(1988). Statistical power analysis for the behavioral sciences (2nd ed.). New York, NY: Academic Press.

19.

Day

Koegl

C. J.

(2015, June). The monetary costs of crime for seven criminal trajectories in an Ontario sample of offenders. Paper presented at 76th Annual Convention of the Canadian Psychological Association, Ottawa, Ontario, Canada.

20.

Dowden

Andrews

(1999). What works for female offenders: A meta-analytic review. Crime & Delinquency, 45, 438-452. doi:10.1177/0011128799045004002

21.

D’Unger

A. V.

Land

K. C.

McCall

P. L.

(2002). Sex differences in age patterns of delinquent/criminal careers: Results from Poisson latent class analyses of the Philadelphia cohort study. Journal of Quantitative Criminology, 18, 349-375. doi:10.1023/A:1021117626767

22.

Gendreau

Little

Goggin

(1996). A meta-analysis of the predictors of adult offender recidivism: What works! Criminology, 34, 575-608. doi:10.1111/j.1745-9125.1996.tb01220.x

23.

Gobeil

Blanchette

Stewart

(2016). A meta-analytic review of correctional interventions for women offenders: Gender-neutral versus gender-informed approaches. Criminal Justice and Behavior, 43, 301-322. doi:10.1177/0093854815621100

24.

Hanley

J. A.

McNeil

B. J.

(1983). A method of comparing the areas under Receiver Operating Characteristic curves derived from the same cases. Radiology, 148, 839-843. doi:10.1148/radiology.148.3.6878708

25.

Heimer

(1996). Gender, interaction, and delinquency: Testing a theory of differential social control. Social Psychology Quarterly, 59, 39-61. doi:10.2307/2787118

26.

Hoge

R. D.

(2012). Forensic assessments of juveniles: Practice and legal considerations. Criminal Justice and Behavior, 39, 1255-1270. doi:10.1177/0093854812444024

27.

Hoge

R. D.

Andrews

D. A.

(2011). Youth Level of Service/Case Management Inventory 2.0. Toronto, Ontario, Canada: Multi-Health Systems.

28.

Hubbard

D. J.

Pratt

T. C.

(2002). A meta-analysis of the predictors of delinquency among girls. Journal of Offender Rehabilitation, 34(3), 1-13. doi:10.1300/J076v34n03_01

29.

Jordan

J. V.

(2010). Theories of psychotherapy: Relational-cultural therapy. Washington, DC: American Psychological Association.

30.

Kraemer

H. C.

Kazdin

A. E.

Offord

D. R.

Kessler

R. C.

Jengen

P. S.

Kupfer

D. J.

(1997). Coming to terms with risk. Archives of General Psychiatry, 54, 337-343. doi:10.1001/archpsyc.1997.01830160065009

31.

Kreis

M. K. F.

Schwannauer

Gillings

(2014). Relational risk factors for reoffending in women: A systematic review. International Journal of Forensic Mental Health, 13, 381-393. doi:10.1080/14999013.2014.962713

32.

Lanctôt

(2018). Gender-responsive programs and services for girls in residential centers: Meeting different profiles of rehabilitation needs. Criminal Justice & Behavior, 45, 101-120. doi:10.1177/0093854817733495

33.

Loza

Green

(2003). The Self-Appraisal Questionnaire: A self-report measure for predicting recidivism versus clinician-administered measures: A 5-year follow-up study. Journal of Interpersonal Violence, 18, 781-797. doi:10.1177/0886260503253240

34.

Martino

Grilo

C. M.

Fehon

D. C.

(2000). Development of the Drug Abuse Screening Test for Adolescents (DAST-A). Addictive Behaviors, 25, 57-70. doi:10.1016/S0306-4603(99)00030-1

35.

Mills

J. F.

Kroner

D. G.

(1999). Measures of Criminal Attitudes and Associates User Guide. Kingston, Ontario, Canada: Author. doi:10.13140/2.1.4785.4081

36.

Mills

J. F.

Kroner

D. G.

Forth

A. E.

(2002). Measures of Criminal Attitudes and Associations (MCAA): Development, factor structure, reliability, and validity. Assessment, 9, 240-253. doi:10.1177/1073191102009003003

37.

Mills

J. F.

Kroner

D. G.

Hemmati

(2004). The Measure of Criminal Attitudes and Associates (MCAA): The prediction of general and violent recidivism. Criminal Justice and Behavior, 31, 717-733. doi:10.1177/0093854804268755

38.

Mitchell

Tafrate

R. C.

(2012). Conceptualization and measurement of criminal thinking: Initial validation of the Criminogenic Thinking Profile. International Journal of Offender Therapy and Comparative Criminology, 56, 1080-1102. doi:10.1177/0306624X11416197

39.

Morash

Kashy

D. A.

Cobbina

J. E.

Smith

S. W.

(2018). Characteristics and context of women probationers and parolees who engage in violence. Criminal Justice and Behavior, 45, 381-401. doi:10.1177/009385481771903

40.

Nunnally

Bernstein

(1994). Psychometric theory (3rd ed.). New York, NY: McGraw-Hill.

41.

Olver

M. E.

Stockdale

K. C.

Wormith

J. S.

(2014). Thirty years of research on the Level of Service scales: A meta-analytic examination of predictive accuracy and sources of variability. Psychological Assessment, 26, 156-176. doi:10.1037/a0035080

42.

Piquero

N. L.

Gover

A. R.

MacDonald

J. M.

Piquero

A. R.

(2005). The influence of delinquent peers on delinquency: Does gender matter? Youth & Society, 36, 251-275. doi:10.1177/0044118X04265652

43.

Rice

M. E.

Harris

G. T.

(2005). Comparing effect sizes in follow-up studies: ROC area, Cohen’s d, and r. Law and Human Behavior, 29, 615-620. doi:10.1007/s10979-005-6832-7

44.

Rosenberg

(1965). Society and the adolescent self-image. Princeton, NJ: Princeton University Press.

45.

Sherer

Maddux

J. E.

Mercandante

Rogers

R. W.

(1982). The Self-Efficacy Scale: Construction and validation. Psychological Reports, 51, 663-671. doi:10.2466/pr0.1982.51.2.663

46.

Shields

I. W.

Whitehall

G. C.

(1991, December). The Pride in Delinquency Scale. Paper presented at the Eastern Ontario Correctional Psychologists’ Winter Conference, Burritts Rapids, Ontario, Canada.

47.

Shields

I. W.

Whitehall

G. C.

(1994). Neutralization and delinquency among teenagers. Criminal Justice and Behavior, 21, 223-235. doi:10.1177/0093854894021002003

48.

Simourd

Andrews

D. A.

(1994). Correlates of delinquency: A look at gender differences. Forum on Corrections Research, 6, 26-31. Retrieved from https://www.ncjrs.gov/App/Publications/abstract.aspx?ID=152033

49.

Singh

C. D.

(2017). P-value shake-up proposed: Big names in statistics recommend tightening threshold for significance in biomedical science. Nature, 548, 16-17. doi:10.1038/nature.2017.22375

50.

Skilling

Sorge

G. B.

(2014). Measuring antisocial values and attitudes in justice-involved youth: Evaluating the psychometric properties of the Pride in Delinquency and Criminal Sentiments Scale-Modified. Criminal Justice and Behavior, 41, 992-1007. doi:10.1177/0093854814521415

51.

Sutherland

E. H.

Cressey

D. R.

(1970). Principles of criminology (6th ed.). New York, NY: Lippincott.

52.

Sykes

G. M.

Matza

(1957). Techniques of neutralization: A theory of delinquency. American Sociological Review, 22, 664-670. doi:10.2307/2089195

53.

Vitopoulos

Peterson-Badali

Skilling

Brown

(2018). The relationship between trauma, recidivism risk, and reoffending in male and female juvenile offenders. Manuscript submitted for publication.

54.

Walters

G. D.

(2012). Criminal thinking and recidivism: Meta-analytic evidence on the prevalence and incremental validity of the Psychological Inventory of Criminal Thinking Styles (PICTS). Aggression and Violent Behavior, 17, 272-278. doi:10.1016/j.avb.2012.02.010

55.

Zell

Krizan

Teeter

S. R.

(2015). Evaluating gender similarities and differences using metasynthesis. American Psychologist, 70, 10-20. doi:10.1037/a0038208