The effects of aftercare/resettlement services on crime and violence in children and youth: A systematic review

Study objectives 1–3

For objectives 1–3, included studies were restricted to rigorous or moderately rigorous research designs, including randomized controlled trials and quasi-experimental comparison group designs in which participant baseline variables (e.g., criminal history, age, sex) were examined for observable between-group differences. Eligible comparators were similar groups of youth who did not participate in an aftercare/resettlement program or receive reentry services. Single group pretest-post-test designs, as well as comparison group designs deemed weak in baseline matching (i.e., with groups found inequivalent on important factors such as risk level and criminal history, or baseline group comparability not provided), were excluded.

Study objective 4

For objective 4, included studies were restricted to implementation assessments of an aftercare/resettlement intervention, without restriction on study research design, or requirement for quantitative outcomes of violence and crime. Both quantitative and qualitative research were included. Qualitative studies (or mixed methods studies with a qualitative component) identified through the search were appraised using the Critical Appraisal Skills Program (CASP) qualitative research checklist (2018). Included studies met, at minimum, the requirements of questions 1–2 on the checklist; specifically, the study provided a clear statement of the aims of the research, and a qualitative methodology was appropriate to meet these aims. See Supporting Information: Appendix 1 for an overview of the checklist.

Study objective 5

Data for objective 5 were obtained if information on program intervention theory was presented in any of the studies included for objectives 1–4, or if studies relevant to an aftercare/resettlement theory of change were identified in the search of gray literature. Included studies were not restricted to those presenting a program evaluation as long as they presented a discussion or description of aftercare/resettlement intervention theory of change.

Study objective 6

For objective 6, data were obtained if analyses on program cost–benefit, cost-effectiveness, or cost-per-participant served were presented in any of the studies included for objectives 1–5.

Primary outcomes

Secondary outcomes

For objectives 1–3 and 5–6, no additional outcomes were considered. For objective 4, qualitative data that did not fit within the a priori outcomes categories were coded inductively, allowing other relevant outcomes to present themselves.

Study objectives 1–3

For objectives 1–3, an extensive coding scheme involving approximately 75 variables was used to extract data from each study. The coded variables fell into categories of general study characteristics (e.g., publication type, program delivery year), program components (e.g., vocational skills, family involvement), information about the study sample (e.g., age, racial mix, gender mix, sample size), research design characteristics (e.g., type of research design, baseline group differences, outcome measure, follow-up period), and outcomes (e.g., means and standard deviations, frequencies, group sample sizes, t-statistics).

Study objective 4

For objective 4, implementation studies were coded using 37 variables. Extracted information related to study details (e.g., program name, program description), study quality (e.g., reporting practices, ethical considerations, limitations), data characteristics (e.g., types of data presented, methods of data analysis, types of participants), and process evaluation outcomes (e.g., program fidelity, service provision, casework feasibility, communication, staffing). After initial coding on these 37 a priori variables was complete, further data extraction took place by identifying emergent variables based on trends across the included studies. This process is described in more detail under Data Synthesis.

Study objective 5

For objective 5, any information concerning the intervention's theory of change was extracted, for example, theorized pathways for participants as they move between program activities towards desired outcomes.

Study objective 6

For objective 6, studies were coded on a series of variables related to any intervention cost analyses (e.g., cost–benefit, cost-effectiveness, cost in relation to participants served). Identified variables include any program or staffing costs, cost per participant, annual costs or budget, or costs related to the justice system (e.g., incarceration, court processes, police/arrests, etc.), and findings of any cost analysis or comparison made. Notably, the studies varied considerably in terms of the type and extent of the cost analysis or comparison presented, and the resulting coding and data extracted were limited.

See the draft coding form in Supporting Information: Appendix 2 for more details.

Study objectives 1–3

Studies were assessed by two independent reviewers for potential risk of bias using either the Cochrane Risk of Bias in Non-randomized Studies of Interventions (ROBINS-I; Sterne et al., 2016) or the Cochrane Risk of Bias tool for randomized studies (RoB-2; Higgins et al., 2019). Non-randomized studies were scored across seven domains to determine a rating of no information, critical, serious, moderate, or low risk of bias, and randomized studies were scored across five domains to determine a rating of high, some concerns, or low risk of bias. Judgments were then visualized using robvis, a tool used to create ‘traffic light’ charts for risk of bias judgments (McGuinness & Higgins, 2020). See the section on Sensitivity Analysis for testing concerning publication bias and influential studies, as well as the section on Moderator Analyses which outlines plans for the exploration of study- and intervention-related characteristics as potential moderators of treatment impact.

Study objective 4

We assessed the quality of included qualitative studies using an adapted version of the Critical Appraisal Skills Program qualitative study checklist (CASP, 2018), making judgments on the adequacy of reporting, data collection, presentation, analysis, and validity of the conclusions drawn. The checklist is included in Supporting Information: Appendix 1. We excluded studies of particularly low quality at this stage (Noyes et al., 2019), as well as studies in which questions 1–2 on the checklist were assessed as “No.” The remaining studies were classified as high, moderate, or low quality. Each item on the checklist was coded as “yes,” “no,” or “can't tell” with respect to whether a component was adequately addressed. Using similar guidelines as presented in Templeton et al. (2017), studies were rated as high quality if they had 8–10 “yes” ratings, moderate quality with 5–7 “yes” ratings, and low quality with 4 or fewer “yes” ratings.

The results of the quality appraisal are reported in Table 10. Further, as described in the section on Sensitivity Analysis, a sensitivity analysis of qualitative evidence was performed to assess the contributions of individual studies to the synthesized process evaluation findings.

Study objective 5

Any study presenting discrepant or conflicting theory of change information was discussed by the research team until a decision about inclusion was reached.

Study objective 6

To assess methodological quality and risk of bias, in our protocol we described plans to assess studies using the Consensus on Health Economic Criteria (CHEC-list) developed by Evers et al. (2005). For this checklist, studies are rated on a series of 19 criteria to calculate a global assessment of study quality. However, the final sample for this objective was small, with only seven studies reporting any kind of cost assessment. Of these studies, only three provided a detailed cost–benefit assessment; a majority provided very minimal cost data. Due to a lack of cost data provided and minimal cost analyses, most studies would have scored very low on the checklist, resulting in even less usable data. Given this, and the fact that the current sample was unable to be synthesized due to the low number of studies overall, formal quality assessments were not conducted.

Study objectives 1–3

All relevant effect size data were extracted from each study, and all potential effect sizes were calculated. For univariate pooled analyses, a key assumption in meta-analysis is the independence of observations (Card, 2011). With respect to the inclusion of effect sizes for objectives 1–3, to ensure independence several a priori decision rules were defined and implemented when appropriate: (a)

If multiple reports present data from the same population or study (e.g., a dissertation and a journal article, or two separate articles using overlapping research samples), all available information will be used but the study will only be counted a single time.

Study objectives 4 and 6

For study objectives 4 and 6, if multiple reports present process evaluation or cost–benefit data from the same population or study (e.g., a dissertation and a journal article, or two separate articles used overlapping research samples), all relevant information will be coded but the study counted only once.

Study objective 5

For objective 5, the concept of study independence is not relevant; any information uncovered that could be used to refine the theory of change will be incorporated.

Study objectives 1–3

For objectives 1–3, we examined tau², Cochran's Q-statistics and I ² statistics (Borenstein et al., 2017; Higgins, 2003). We also disaggregated the available outcome measures into smaller, more commensurate sets of outcomes (i.e., rearrest, reconviction, and reincarceration).

Study objective 4

In addition to the exclusion of studies that did not fit study inclusion criteria (e.g., the evaluated program was not aftercare, participants were not youth), studies were removed if they did not evaluate program implementation or fidelity. For example, studies that were qualitative impact evaluations (e.g., where the focus was on participants' perceived outcomes) were excluded.

Study objective 5

Assessment of heterogeneity is not relevant for objective 5 (theory of change).

Study objective 6

For objective 6, as noted in the protocol we intended to assess heterogeneity by pooling outcomes based on various study and program characteristics to account for potentially variability (e.g., program setting, number of participants served, length of program operation). Unfortunately, this was not possible due to the small sample size and lack of available cost data.

Study objectives 1–3

The two main approaches to modeling data in meta-analysis are fixed effects and random effects models. Both models weight each study by its inverse variance, however, the calculation of weights is based on the assumed source of variability between studies (Card, 2011; Egger & Smith, 2001). A fixed effects model assumes that any between-subject variability is the result of sampling error and occurs only by chance, while random effects models assume that between-study heterogeneity is important and is due to factors other than random subject-level sampling error (Card, 2011; Lipsey & Wilson, 2001). In the current analysis, given the expectation of a multitude of between-study differences, we used restricted maximum likelihood (REML) random effects estimators with the Hartung-Knapp-Sidik-Jonkman (HKSJ) variance correction. All analyses were conducted in Stata/SE 18.0, results are presented in forest plots. Analyses were conducted separately for three groups of outcomes: arrest, conviction, and incarceration.

Study objective 4

For outcome 4, both qualitative and quantitative data were collected to identify barriers and facilitators to program implementation and fidelity to intervention design. The extraction of qualitative data was in part based on an a priori set of expected findings, derived from a preliminary review of the literature. Specifically, broader categories of variables, such as communication, continuity of care, organization partnerships, and case management guided coding in a deductive manner wherein process data were coded within these predetermined variables either as either program “challenges” or “successes.” For reported data that did not fit into these pre-defined variables, an inductive thematic approach was taken to code and analyze these findings using qualitative data analysis software,

Coding followed the three stages of aftercare/resettlement: (1) pre-release programming and planning, including risk assessments and case management, (2) intensive transition structures and processes, including throughcare services, and (3) post-release supervision and services, including intensive supervision and reintegration programming. However, many studies reported process data that did not belong to one of these stages; therefore, some data were coded as “overall” findings. See Table 1 for example from the coding scheme, guided by deductive and inductive variables in each of the three stages of aftercare/resettlement. A complete list of a priori variables can be found in the coding form (Supporting Information: Appendix 2).

Study objectives 1–3

We investigated between-study heterogeneity of effects through subgroup (moderator) analysis to examine whether certain categorical variables could explain some of the variability in effect sizes (Card, 2011; Lipsey & Wilson, 2001). Dichotomized variables were related to study characteristics, sample characteristics, intervention components, and program delivery. A total of 24 potential moderators were examined. Only those variables in which each subgroup had a minimum of 3 effect sizes were included in the analyses, resulting in a maximum of 16 moderators in the analyses. Associations among moderators within each of the three outcome categories were assessed in an effort to address potential collinearity; two correlations were statistically significant for the arrest outcome. These included bias rating and research design (only the bias rating was used as a moderator), and group therapy and mentoring (only the mentoring variable was used). Crosstabulations were computed between moderators within each of the four categories (study, sample, intervention, and program delivery characteristics) for each of the three program outcomes (arrest, conviction, incarceration); no significant associations were observed.

With respect to study characteristics (all identified a priori), the following four variables were examined: peer-reviewed publication type (yes vs. no), research design (randomized control trial vs. quasi-experimental design), treatment group sample size (<75 participants vs. 75+ participants), and risk of bias score (low/moderate risk vs. serious risk

Sample characteristics (both identified a priori) included: sample race (predominantly Caucasian/mixed race vs. predominantly racial minority

Intervention characteristics were coded for presence of the following 7 components (yes vs. no; all components were identified a priori)

Last, each program was coded based on who was involved in delivery of program services, including the following three non-mutually exclusive options

Subgroup analyses were conducted using the analog to the ANOVA method, which separates the total variability (Q _T) into the within-group variation (Q _w = the summed Q-statistics for each of the two groups in the analysis) and that which can be explained by the categorical variable (the between-group variation; Q _b = the difference between the total and within-Q-statistics). A statistically significant Q _b suggests the two categories are producing significantly different effect sizes and the difference is due to more than sampling error (Lipsey & Wilson, 2001).

In addition, random effects meta-regression models were implemented, incorporating multiple moderators into the same model (Hedges & Olkin, 1985). This approach served to reduce potential confounding across moderators. As in standard linear regression analysis, the coefficients derived from a meta-regression describe how the dependent variable changes with a unit increase in the predictor (moderator) variables. We considered the use of robust variance estimation (RVE) so that dependent effect sizes could be included in the meta-regression models without violating the assumption of independence (see Hedges et al., 2010; Pustejovsky & Tipton, 2021; Tanner-Smith & Tipton, 2013). However, based on simulations provided by Hedges et al. (2010) and Tipton (2013), RVE appeared unlikely to perform well (with respect to empirical coverage) for the given number of effect sizes per study and total number of studies in our set. As such, standard meta-regression models were implemented using no more than one effect size per study. We used REML to estimate the additive component of tau², with the Knapp Hartung variance correction (Tipton et al., 2019a, 2019b). Given the relatively small number of studies for each outcome, moderator variables were grouped into three separate models for the outcomes of arrest and conviction, and two separate models for the outcome of incarceration.

Study objective 4

Subgroup analysis of qualitative studies was conducted based on themes uncovered from the process evaluation evidence synthesis for aftercare/resettlement programming as a whole, as well as for each of the three stages of aftercare/resettlement. Specifically, overarching themes and findings were derived, along with subgroup analyses based on findings related to the custodial phase of intervention, the transition phase, and the post-custodial phase.

Study objective 5

Subgroup analysis and investigation of heterogeneity is not relevant for objective 5 (theory of change).

Study objective 6

Subgroup analysis of intervention cost-related data was intended to be analyzed in smaller sets based on various characteristics, such as intervention setting (e.g., for those programs delivered in a hybrid prison/community setting than solely in the community), program size (e.g., low vs. high numbers of participants served), program length of operation (e.g., newer vs. more established programs), geographic location (e.g., United States vs. United Kingdom). However, the sample included only seven eligible studies (with 8 independent program sites) and a lack of consistency in reported data, which precluded any subgroup analysis.

Study objectives 1–3

For objectives 1–3, we tested sensitivity of the findings to strongly influential studies by conducting a remove-one-study influence analysis (Tobias, 1999). To do so, each study in the meta-analysis was omitted, one at a time, and the pooled effect was recalculated without that study to determine whether its removal had a notable impact on the pooled findings of the meta-analysis. Further, we tested the sensitivity of findings by using research design bias as a treatment moderator, and calculating the pooled effect separately for low/moderate risk versus serious risk studies.

Study objective 4

A sensitivity analysis of the synthesized process evaluation findings was performed to assess the contributions of qualitative studies. By examining both the “frequency” (i.e., if certain conclusions are reliant on data from a single study) and “thickness” (i.e., the depth of understanding that studies bring to conclusions) of study outcomes within the qualitative evidence synthesis findings, we considered the influence and profile of individual studies (i.e., if studies with low or high methodological quality were more influential), and possible biases that shaped the synthesized findings (Carroll et al., 2012, 2013).

In addition, the CASP (2018) qualitative study quality appraisal checklist was used as a guide while conducting the sensitivity analysis (see Supporting Information: Appendix 1). The level of quality observed among qualitative studies (high, moderate, or low in methodological quality, based on a cut off score modeled after Templeton et al. (2017), was used to examine study contributions to synthesized process evaluation findings. Where conclusions were based primarily on studies with low or moderate methodological quality, we tempered our conclusions appropriately, or removed themes altogether.

Study objectives 5 and 6

Sensitivity analysis is not relevant for objective 5 (theory of change) or objective 6 (intervention cost).

Intervention studies

The present sample included 15 eligible studies assessing intervention impact. See Table 2 for a detailed summary of each study.

These 15 studies represented 21 individual program implementation sites

With respect to program characteristics, the vast majority of interventions (k = 19, 90%) were implemented both pre- and post-release, and were primarily handled by a case manager (k = 20, 95%). Programs included additional involvement from other program staff (62%), community-based service providers (71%), probation officers (29%), and counselors (10%; categories were not mutually exclusive). Specific program components included behavioral or emotional supports (76% of programs), academic supports (71%), vocational training (76%), life skills development (62%), housing assistance (48%), individual therapy (67%), group therapy (14%), family therapy or services (62%), mentoring (43%), substance use treatment (71%), and parental involvement (86%).

A total of 6 (29%) sites were evaluated using a randomized controlled trial, while the remaining 15 (71%) utilized a non-randomized design with a matched comparison group. Of the 21 sites, 57% were scored as having low/moderate research design bias, while 43% were scored as having serious bias concerns. Outcomes were most commonly measured between 12 and 17 months post release (57%). Study samples were primarily male, with 8 (38%) utilizing all male samples, as opposed to mixed gender samples (62%). Overall participant age ranged from 11 to 21 years, with means ranging from 15.2 to 17.3. Samples primarily included participants of an ethnic minority group (i.e., <30% White; 62%). Sample size at baseline ranged from 61 to 850, with a mean of 235 (SD = 190.2); 57% of the sites had a sample of over 75 participants. See Table 3 for a presentation of study characteristics for the full sample, as well as by outcome measure.

Implementation studies

The systematic search resulted in a total of 19 publications which provided process evaluations of 18 reentry programs in 27 program sites.

As shown in Table 5, studies were published between 1998 and 2021 (median = 2012), primarily in the United States. Data were commonly collected through surveys and interviews with participants or program staff (e.g., aftercare/resettlement workers, case managers, stakeholders), observations and field notes, focus groups, and/or case records. Qualitative interviewing was used as a method of data collection in all 27 implementation sites. Further, observations (k = 14), surveys of program materials (k = 12) and case records (k = 11), focus groups (k = 10), and questionnaires (k = 10) were used to collect data. Methods of data analysis in the 19 studies were varied and often not stated explicitly, but included ethnographic analysis, narrative synthesis, and thematic analysis. Last, in the 27 implementation sites, the most common sources for data were program administrators (k = 18) and case managers (18). Other sources included community partners (k = 13), participants (k = 7), parents (6), custodial staff (k = 4), probation officers (k = 3), school staff (k = 2), counselors (k = 1), and policymakers (k = 1).

Cost assessment studies

A total of 7 studies (8 program sites) were found to report on cost assessment data. Of these, 3 assessed a program implemented in the United States, 1 in Canada, and 3 in the United Kingdom. Three assessments were reported in journal articles, with 4 in reports. See Table 6 for an overall summary of the studies.

Intervention studies

The 250 studies excluded from the analysis of intervention evaluation data are summarized in Table 7, with detailed exclusion reasons. Of these 250 studies, the majority were excluded for not having an evaluation of a program (k = 79) or because the program discussed was not an aftercare or resettlement program (k = 57). Many studies were “near misses”; for example, where samples were a mix of youth with custodial sentences and those without but data were not presented separately (e.g., Lane et al., 2005; Lindquist et al., 2014; Savage et al., 2010). Similarly, some studies had treatment samples with mean ages that were between 19 and 20 years old, just above the cutoff of 18 years (e.g., Beausoleil et al., 2017; Josi & Sechrest, 1999).

Implementation studies

Nineteen of the 265 full text publications were included in the qualitative synthesis analysis of implementation data, and 212 were excluded for not having relevant implementation data. An additional 40 studies were excluded for not meeting other study eligibility requirements, as summarized in Table 8.

Cost studies

Of the 265 retrieved studies, 258 were excluded from the synthesis of cost assessment data. 23 were excluded for not reporting any appropriate cost assessment data, and an additional 3 were excluded as they did not meet program eligibility requirements (e.g., sample mean age, non-commensurate program), despite reporting on relevant cost data. See Table 9.

Intervention studies

The included intervention studies consisted of 4 RCT designs (6 program sites), and 11 non-randomized designs (15 program sites). The RCTs were assessed for risk of bias and overall study quality using the Revised Cochrane Risk of Bias tool for randomized trials (RoB-2; Higgins et al., 2019), while the non-randomized studies were assessed the Cochrane Risk of Bias in Non-randomized Studies of Interventions (ROBINS-I; Sterne et al., 2016). With respect to the RCT designs, risk of bias was assessed across five domains relating to the randomization process, deviations from the intended intervention, missing outcome data, outcome measurement, and selection of the reported result. All six of the reported program sites received an overall judgment of “some concerns” indicating moderate levels of risk. This is primarily a result of two factors. The first is in relation to the randomization process, as no study supplied enough information to determine whether the allocation sequence was concealed from participants and as such, we could not confidently code this domain was low risk. Additionally, all studies were determined to have “some concerns” in relation to the selection of reported results (domain 5) as no studies provided information regarding whether a pre-specified analysis plan (e.g., trial protocol) was followed. As these studies were primarily government reports, it is unlikely there was a formal, approved scientific study protocol developed.

Regarding the non-randomized designs, risk of bias was assessed across seven domains related to confounding, selection of participants, classification of interventions, deviation from intended interventions, missing outcome data, outcome measurement, and selection of the reported result. Six program sites received a judgment of moderate, six of serious, and two of critical. The main domain of concern was that of bias due to confounding. Due to the nature of the assessed interventions, control of potential confounding variables to a similar level as RCT designs is not feasible. Many of the interventions occur in the community, with a variety of potential influences that are out of the evaluators' scope of control. As such, all studies received a judgment of moderate or higher on that domain, contributing to the final overall judgments. Six program sites were scored as moderate for confounding due to evaluators attempts to control for some potential confounds, such as utilizing multivariate analyses (e.g., Abrams et al., 2008; Andersson Vogel et al., 2014; Bouffard & Bergseth, 2008; Liberman et al., 2021; Stafford & Glassner, 2012) or propensity score matching (e.g., Liberman et al., 2021). We note that these studies and methods were unable to account for all possible confounding factors. The remaining studies were judged as serious or critical risk for confounding due to minimal efforts to control for confounding.

Based on these tools, the evaluations of 12 program sites were assessed as being low/moderate risk for bias, while 7 were deemed serious risk (see Figures 3 and 4, generated using robvis; McGuinness & Higgins, 2020).

OPEN IN VIEWER

Figure 3

Risk of bias assessments for randomized design studies (RoB-2).

OPEN IN VIEWER

Figure 4

Risk of bias assessments for non-randomized design studies (ROBINS-I).

Implementation studies

Qualitative implementation studies were assessed for quality and risk of bias using the Critical Appraisal Skills Program qualitative study checklist (CASP, 2018). The present sample included 19 publications

Cost studies

As discussed in the protocol, methodological quality and risk of bias of studies presenting a cost analysis were intended to be assessed using the Consensus on Health Economic Criteria (CHEC-list) developed by Evers et al. (2005). However, given the small sample of cost-related studies (k = 7) identified and the lack of available cost data, we determined that this step was not useful, and the checklist was not implemented.

Reporting bias – Arrest outcome

The funnel plot for the set of studies examining arrest was reasonably symmetric (Figure 6), with three studies outside the 95% confidence limits. Egger's test for small-study effects was not significant (−0.379, t = −0.41, p = 0.687).

OPEN IN VIEWER

Figure 6

Funnel plot for arrest outcome (k = 14).

Sensitivity analysis – Arrest outcome

Figure 7 presents findings from the remove-one-study influence analysis. Results indicate that the non-significant aggregate treatment effect is robust; the removal of any of the 14 effect sizes does not change the overall substantive finding.

OPEN IN VIEWER

Figure 7

Influence analysis for arrest outcome (k = 14).

Given potential differences in findings related to the time period of the outcome measure, sensitivity analyses were conducted by limiting study pools to those effect sizes measured at similar time points. Doing so allows for more homogenous analyses with respect to the follow-up time periods examined in each study. Specifically, one analysis included the two studies which measured outcomes at 3 months post-release. The pooled odds ratio was 1.215 and non-statistically significant (t = 0.398, p = 0.759). Similarly, a pool of the two studies measuring outcomes at 18 months produced a non-statistically significant odds ratio of 0.698 (t = −1.152, p = 0.455). Last, a pooled analysis limited to the eight studies measuring outcomes at 12 months was conducted; the resulting odds ratio was 1.00 (t = 0.003, p = 0.998). None of these analyses resulted in a substantive change to the original finding of no treatment impact on the outcome of arrest.

Moderator analysis – Arrest outcome

Thirteen potential moderators of aftercare/resettlement treatment impact on arrest outcomes were examined. As noted previously, only moderators with a minimum of 3 effect sizes per group were included; see Table 11 for an overview of all findings.

Two variables related to study design were included: bias score and treatment group sample size. The bias score rating was found to significantly moderate the impact of the aftercare/resettlement intervention (Q _B  = 7.60, p < 0.01). Specifically, 9 of the 14 studies were rated as low/moderate risk (64%); these studies produced a positive, non-statistically significant odds ratio of 1.191. For the 5 studies rated as serious risk (36%), the pooled effect was negative and statistically significant (OR = 0.698, p < 0.05). See Figure 8. These findings indicate that studies rated as having notable bias were related to negative treatment effects.

OPEN IN VIEWER

Figure 8

Forest plot for research bias moderator for arrest outcome (k = 14).

No other variables were found to be significant moderators of treatment impact, include two variables related to sample characteristics (sample ethnicity and sample gender), six program/intervention characteristics (whether the program included life skills training, housing, individual therapy, family therapy, mentoring, or substance use treatment), and three variables related to program delivery (whether the program was delivered by probation officers, program staff, or community service providers (not mutually exclusive)). See Table 11.

To examine the joint impact of moderator variables, we implemented three random effects meta-regression models (with the Knapp Hartung correction technique). The first model included the two study design characteristics and the two study sample characteristics, the second model incorporated the six program component characteristics, and the third model included the three program delivery characteristics. The model F-statistics were all nonsignificant; indicating that, when looked at jointly, none of the variables were important moderators of aftercare/resettlement intervention impact on arrests.

In addition, to examine the impact of timing of outcome measure, we implemented a random effects meta-regression (with the Knapp Hartung correction) to the full set of 17 studies with time to outcome measured in weeks. The model was implemented with time to outcome as the sole predictor, and in combination with the two study design characteristics and the two study sample characteristics. Both model F-statistics were non-significant, and none of the variables were important moderators of treatment effect on arrests.

Reporting bias – Conviction outcome

Figure 10 presents a funnel plot showing all studies falling within the 95% confidence intervals; Egger's test for small-study effects was not significant (bias = −1.231, t = −1.81, p = 0.098).

OPEN IN VIEWER

Figure 10

Funnel plot for conviction outcome (k = 13).

Sensitivity analysis – Conviction outcome

Results of the influence analysis indicate that the pooled finding is not robust; removing any of 8 of the 13 effect sizes is sufficient to reduce the aggregate LOR to a non-statistically significant level favoring the treatment group. For example, removing the study by Liberman et al. (2021) - Virginia would result in a pooled odds ratio of 1.172 (i.e., LOR = 0.159, 95%CI [−0.053, 0.372]). See Figure 11.

OPEN IN VIEWER

Figure 11

Influence analysis for conviction outcome (k = 13).

Upon close examination, the aftercare intervention evaluated by Abrams et al. (2008) is arguably different than the other programs in the set, as it is the only intervention which takes place solely during a custodial phase.

OPEN IN VIEWER

Figure 12

Forest plot for conviction outcome: Removing 1 study (k = 12).

Given potential differences related to the time period in which outcomes were measured, additional sensitivity analyses were conducted by limiting study pools to effect sizes measured at similar time points. Specifically, the two studies which measured outcomes at 24 months post-custodial release were pooled, resulting in a non-statistically significant odds ratio of 1.317 (t = 1.363, p = 0.403). In addition, a pooled analysis using only the 10 studies measuring outcomes at 12 months was conducted; the resulting LOR of 0.139 (95%CI [−0.102, 0.379]) was not statistically significant, with a corresponding odds ratio of 1.149 (t = 1.305, p = 0.224).

Overall, results of the sensitivity testing suggest that the aggregate impact of aftercare/resettlement programs on conviction is marginal, and is acutely sensitive to the addition or removal of individual studies. Strong conclusions based on these findings are not warranted.

Moderator analysis – Conviction outcome

Twelve potential moderators of the impact of aftercare/resettlement interventions on conviction outcomes were examined. As noted previously, only moderators with a minimum of three effect sizes per group were included; see Table 12 for a summary of findings.

Four variables related to study design were included: peer-reviewed status, research design, bias score, and treatment group sample size. While none of the variables were found to significantly moderate the impact of the aftercare/resettlement intervention (i.e., non-significant between-groups heterogeneity statistics), two of the subgroups produced significant effects on their own.

Experimental design

A significant, positive subgroup effect on the outcome of conviction was found for the group of nine quasi-experimental studies (OR = 1.254, z = 2.371, p = 0.018); see Figure 13.

OPEN IN VIEWER

Figure 13

Forest plot for experimental design moderator for conviction outcome (k = 13).

Treatment group sample size

As shown in Figure 14, the aggregate impact of studies with samples of over 75 participants was positive and statistically significant (LOR = 0.229, z = 2.508, p = 0.012).

OPEN IN VIEWER

Figure 14

Forest plot for treatment group sample size moderator for conviction outcome (k = 13).

Two variables related to sample characteristics were included: sample ethnicity and sample gender. The between-group heterogeneity statistic was non-significant for both analyses, indicating that neither were significant moderators of treatment impact. However, samples that included predominantly minority group participants resulted in a positive and significant overall pooled effect (LOR = 0.284, z = 2.437, p = 0.015). See Figure 15.

OPEN IN VIEWER

Figure 15

Forest plot for sample ethnicity moderator for conviction outcome (k = 13).

Sample ethnicity

With respect to program components, no significant moderating effect was found for the moderators of housing, individual therapy, group therapy, or mentoring.

Last, program delivery by probation officers and community service providers was examined. While no significant between-group heterogeneity was identified, the group of 10 studies involving a program which did not include probation officers resulted in a positive, statistically significant (LOR = 0.212, z = 2.284, p = 0.022). See Figure 16.

OPEN IN VIEWER

Figure 16

Forest plot for probation officer delivery moderator for conviction outcome.

Delivery by probation officers

In addition, the subgroup of nine studies which involved community service providers produced a positive pooled LOR of 0.237, z = 2.339, p = 0.019. See Figure 17 and Table 12.

OPEN IN VIEWER

Figure 17

Forest plot for community service provider delivery moderator for conviction outcome.

OPEN IN VIEWER

Table 12

Conviction outcome: Summary of moderator analyses (k = 13).

Type	Moderator	Definition	N (%) per group	LOR, z, p-value, and heterogeneity measures
Study characteristics	Peer-review	No	10 (77%)	LOR = 0.194, z = 1.931, p = 0.053
		Yes	3 (23%)	LOR = 0.180, z = 1.109, p = 0.267
				Q T = 11.69, p = 0.471
				Q B = 0.01, p = 0.942
	Research design	Non-RCT	9 (69%)	LOR = 0.227, z = 2.371, p = 0.018*
		RCT	4 (31%)	LOR = 0.047, z = 0.249, p = 0.804
				Q T = 11.69, p = 0.471
				Q B = 0.73, p = 0.394
	Bias score	Low/medium	8 (62%)	LOR = 0.161, z = 1.637, p = 0.102
		Serious	5 (38%)	LOR = 0.276, z = 1.618, p = 0.106
				Q T = 11.69, p = 0.471
				Q B = 0.34, p = 0.559
	Tx group sample size	<75	4 (42%)	LOR = −0.080, z = −0.332, p = 0.740
		75+	9 (69%)	LOR = 0.229, z = 2.508, p = 0.012*
				Q T = 11.69, p = 0.471
				Q B = 1.45, p = 0.229
Sample characteristics	Sample ethnicity	White/mixed	6 (46%)	LOR = 0.081, z = 0.649, p = 0.517
		Minority	7 (54%)	LOR = 0.284, z = 2.437, p = 0.015*
				Q T = 11.69, p = 0.471
				Q B = 1.40, p = 0.236
	Sample gender	All male	4 (31%)	LOR = 0.183, z = 1.444, p = 0.149
		Mixed gender	9 (69%)	LOR = 0.196, z = 1.695, p = 0.090
				Q T = 11.69, p = 0.471
				Q B = 0.01, p = 0.941
Program components	Housing1	No	8 (67%)	LOR = 0.122, z = 0.889, p = 0.374
		Yes	4 (33%)	LOR = 0.217, z = 1.758, p = 0.079
				Q T = 11.48, p = 0.404
				Q B = 0.26, p = 0.610
	Individual therapy2	No	4 (36%)	LOR = −0.153, z = −0.631, p = 0.528
		Yes	7 (64%)	LOR = 0.199, z = 1.708, p = 0.088
				Q T = 10.82, p = 0.372
				Q B = 1.71, p = 0.191
	Group therapy3	No	7 (70%)	LOR = 0.186, z = 1.173, p = 0.241
		Yes	3 (30%)	LOR = 0.050, z = 0.249, p = 0.803
				Q T = 10.82, p = 0.288
				Q B = 0.29, p = 0.591
	Mentoring3	No	6 (60%)	LOR = 0.117, z = 0.745, p = 0.456
		Yes	4 (40%)	LOR = 0.160, z = 0.789, p = 0.430
				Q T = 10.82, p = 0.288
				Q B = 0.03, p = 0.867
Program delivery	Probation officers	No	10 (77%)	LOR = 0.212, z = 2.284, p = 0.022*
		Yes	3 (23%)	LOR = 0.069, z = 0.319, p = 0.749
				Q T = 11.69, p = 0.471
				Q B = 0.36, p = 0.546
	Community service providers	No	4 (29%)	LOR = 0.075, z = 0.477, p = 0.633
		Yes	9 (64%)	LOR = 0.237, z = 2.339, p = 0.019*
				Q T = 11.69, p = 0.471
				Q B = 0.74, p = 0.389

One study did not provide information.

Two studies not did provide information.

Three studies did not provide information.

p < 0.05.

Reporting bias – Incarceration outcome

The funnel plot shows some degree of asymmetry, with studies clustered in the lower left quadrant. Egger's test was not significant (bias = −1.832, t = −1.47, p = 0.191). See Figure 19.

OPEN IN VIEWER

Figure 19

Funnel plot for incarceration outcome (k = 8).

Sensitivity analysis – Incarceration outcome

The influence analysis involving removal of each study one at a time did not produce any substantively different findings; in other words, the aggregate (lack of) effect on incarceration outcomes is negative and robust and no study exerts a disproportionate effect. See Figure 20.

OPEN IN VIEWER

Figure 20

Influence analysis for incarceration outcome (k = 8).

As an additional sensitivity analyses, the two studies measuring outcomes at 15 and 24 months were dropped from the set, leaving only the six studies using the commensurate 12 month outcome period. No substantive difference in findings emerged; the pooled odds ratio was negative and non-statistically significant at 0.807 (t = −1.144, p = 0.304).

Moderator analysis – Incarceration outcome

Despite the lack of evidence of heterogeneity in the pooled set of studies, six variables were examined as potential moderators for the incarceration outcome; see Table 13 for an overview of findings. No significant between-group heterogeneity statistics were observed; two subgroups produced notable pooled effects. First, in terms of the bias score rating, studies rated as having serious bias were related to a statistically significant negative pooled odds ratio of 1.718 (LOR = −0.541, z = −2.445, p < 0.05). In other words, studies rated as high bias had a negative effect on incarceration; see Figure 21.

OPEN IN VIEWER

Figure 21

Forest plot for bias score moderator for incarceration outcome (k = 8).

Gender

Second, with respect to gender, interventions serving samples of all male participants were related to statistically significant, negative effects on the outcome of incarceration (LOR = −0.345, z = −2.203, p < 0.05). See Figure 22 and Table 13.

OPEN IN VIEWER

Figure 22

Forest plot for gender moderator for incarceration outcome (k = 8).

OPEN IN VIEWER

Table 13

Incarceration outcome: Summary of moderator analyses (k = 8).

Type	Moderator	Definition	N (%) per group	LOR, z, p-value, and heterogeneity measures
Study characteristics	Research design	Non-RCT	5 (63%)	LOR = −0.170, z = −1.369, p = 0.171
		RCT	3 (38%)	LOR = −0.229, z = −1.138, p = 0.255
				Q T = 9.56, p = 0.215
				Q B = 0.06, p = 0.803
	Bias score	Low/medium	5 (63%)	LOR = −0.081, z = −0.679, p = 0.497
		Serious	3 (38%)	LOR = −0.541, z = −2.445, p = 0.014*
				Q T = 9.56, p = 0.215
				Q B = 3.33, p = 0.068
	Tx group sample size	<75	3 (38%)	LOR = −0.381, z = −1.516, p = 0.129
		75+	5 (63%)	LOR = −0.144, z = −1.240, p = 0.215
				Q T = 9.56, p = 0.215
				Q B = 0.73, p = 0.392
Sample characteristics	Sample ethnicity	White/mixed	3 (38%)	LOR = −0.263, z = −1.776, p = 0.076
		Minority	5 (63%)	LOR = −0.106, z = −0.708, p = 0.479
				Q T = 9.56, p = 0.215
				Q B = 0.55, p = 0.458
	Sample gender	Mixed gender	3 (38%)	LOR = −0.054, z = −0.377, p = 0.706
		All male	5 (63%)	LOR = −0.345, z = −2.203, p = 0.028*
				Q T = 9.56, p = 0.215
				Q B = 1.89, p = 0.169
Program components	Housing	No	3 (38%)	LOR = −0.229, z = −1.138, p = 0.255
		Yes	5 (63%)	LOR = −0.170, z = −1.369, p = 0.171
				Q T = 9.56, p = 0.215
				Q B = 0.06, p = 0.803

p < 0.05.

To examine the joint impact of moderators, two random effects meta-regression models were implemented (with the Knapp Hartung correction technique). The first model included the three study characteristics, and the second model included the two study sample characteristics and the one program component characteristic. Again, no significant model statistics were noted; none of the variables examined are important moderators of aftercare/resettlement intervention impact on the outcome incarceration.

To examine the impact of timing of outcome measure, we implemented a random effects meta-regression (with the Knapp Hartung correction) to the full set of 8 studies with time to outcome measured in weeks. The model was implemented with time to outcome as the sole predictor, and in combination with the three study design characteristics. Both model F-statistics were non-significant, and none of the variables were important moderators of treatment effect on incarceration.

Tables 14 and 15 present summaries of the findings from the main analyses and the moderator analyses.

OPEN IN VIEWER

Table 14

Summary of main analysis findings.

Outcome category	No. of effect sizes	Pooled OR (LOR)	95% prediction interval (OR)	t (p-value)	tau2, Q-statistic (p-value) I 2 statistic
Arrest	14	OR = 1.044	[0.527, 2.075]	0.335	tau2 = 0.081,
					Q = 24.04
		(LOR = 0.043)		(p = 0.743)	(p = 0.031*)
					I 2 = 45.9%
Conviction	13	OR =3.287	[1.000, 1.462]	2.256	tau2 = 0.000,
					Q = 11.69
		(LOR = 1.190)		(p = 0.044*)	(p = 0.471)
					I 2 = 0%
Incarceration	8	OR =0.806	[2.203, 1.433]	−1.674	tau2 = 0.039,
					Q = 9.56
		(LOR = −0.216)		(p = 0.138)	(p = 0.215).
					I 2 = 26.8%

OPEN IN VIEWER

Table 15

Summary of moderator analyses.

Moderator	Arrest	Conviction	Incarceration
Study characteristics
Peer review	n/a	No peer review: LOR = 0.194, z = 1.931, p = 0.053 Peer reviewed: LOR = 0.180, z = 1.109, p = 0.267 Q B = 0.01, p = 0.942	n/a
Research design	n/a	Non-RCT: LOR = 0.227, z = 2.371, p = 0.018* RCT: LOR = 0.047, z = 0.249, p = 0.804 Q B = 0.73, p = 0.394	Non-RCT: LOR = −0.170, z = −1.369, p = 0.171 RCT: LOR = −0.229, z = −1.138, p = 0.255 Q B = 0.06, p = 0.803
Bias score	Low bias: LOR = 0.175, z = 1.926, p = 0.054 Serious bias: LOR = −0.360, z = −2.099, p = 0.036* Q B = 7.60, p = 0.006**	Low bias: LOR = 0.161, z = 1.637, p = 0.102 Serious bias: LOR = 0.276, z = 1.618, p = 0.106 Q B = 0.34, p = 0.559	Low bias: LOR = −0.081, z = −0.679, p = 0.497 Serious bias: LOR = −0.541, z = −2.445, p = 0.014* Q B = 3.33, p = 0.068
Tx group sample size	<75: LOR = 0.131, z = 0.915, p = 0.360 75+: LOR = 0.024, z = 0.249, p = 0.804 Q B = 0.38, p = 0.537	<75: LOR = −0.080, z = −0.332, p = 0.740 75+: LOR = 0.229, z = 2.508, p = 0.012* Q B = 1.45, p = 0.229	<75: LOR = −0.381, z = −1.516, p = 0.129 75+: LOR = −0.144, z = −1.240, p = 0.215 Q B = 0.73, p = 0.392
Sample characteristics
Sample ethnicity	White/mixed: LOR = −0.106, z = −0.605, p = 0.545 Minority: LOR = 0.101, z = 1.121, p = 0.262 Q B = 1.11, p = 0.293	White/mixed: LOR = 0.081, z = 0.649, p = 0.517 Minority: LOR = 0.284, z = 2.437, p = 0.015* Q B = 1.40, p = 0.236	White/mixed: LOR = −0.263, z = −1.776, p = 0.076 Minority: LOR = −0.106, z = −0.708, p = 0.479 Q B = 0.55, p = 0.458
Sample gender	Mixed gender: LOR = 0.089, z = 0.898, p = 0.369 All male: LOR = −0.003, z = −0.020, p = 0.984 Q B = 0.29, p = 0.591	Mixed gender: LOR = 0.183, z = 1.444, p = 0.149 All male: LOR = 0.196, z = 1.695, p = 0.090 Q B = 0.01, p = 0.941	Mixed gender: LOR = −0.054, z = −0.377, p = 0.706 All male: LOR = −0.345, z = −2.203, p = 0.028* Q B = 1.89, p = 0.169
Program components
Life skills	No: LOR = −0.083, z = −0.716, p = .474 Yes: LOR = 0.088, z = 0.643, p = 0.520 Q B  = 0.91, p = 0.341	n/a	n/a
Housing	No: LOR = −0.052, z = −0.489, p = 0.625 Yes: LOR = 0.206, z = 1.673, p = 0.094 Q B  = 2.52, p = 0.112	No: LOR = 0.122, z = 0.889, p = 0.374 Yes: LOR = 0.217, z = 1.758, p = 0.079 Q B  = 0.26, p = 0.610	No: LOR = −0.229, z = −1.138, p = 0.255 Yes: LOR = −0.170, z = −1.369, p = 0.171 Q B  = 0.06, p = 0.803
Individual therapy	No: LOR = −0.268, z = −1.189, p = 0.234 Yes: LOR = 0.035, z = 0.360, p = 0.719 Q B  = 1.52, p = 0.217	No: LOR = −0.153, z = −0.631, p = 0.528 Yes: LOR = 0.199, z = 1.708, p = 0.088 Q B  = 1.71, p = 0.191	n/a
Group therapy	n/a	No: LOR = 0.186, z = 1.173, p = 0.241 Yes: LOR = 0.050, z = 0.249, p = 0.803 Q B  = 0.29, p = 0.591	n/a
Family therapy	No: LOR = 0.043, z = 0.349, p = 0.727 Yes: LOR = 0.069, z = 0.651, p = 0.515 Q B  = 0.03, p = 0.869	n/a	n/a
Mentoring	No: LOR = 0.242, z = 1.498, p = 0.134 Yes: LOR = −0.120, z = −1.139, p = 0.255 Q B = 3.52, p = 0.060	No: LOR = 0.117, z = 0.745, p = 0.456 Yes: LOR = 0.160, z = 0.789, p = 0.430 Q B = 0.03, p = 0.867	n/a
Subs use treat	No: LOR = 0.098, z = 0.799, p = 0.424 LOR = 0.028, z = 0.258, p = 0.796 Q B = 0.19, p = 0.665	n/a	n/a
Program delivery
Probation officers	No: LOR = −0.016, z = −0.176, p = 0.860 Yes: LOR = 0.289, z = 1.773, p = 0.076 Q B = 2.66, p = 0.103	No: LOR = 0.212, z = 2.284, p = 0.022* Yes: LOR = 0.069, z = 0.319, p = 0.749 Q B = 0.36, p = 0.546	n/a
Program staff	No: LOR = 0.127, z = 1.109, p = 0.267 Yes: LOR = −0.009, z = −0.078, p = 0.938 Q B = 0.72, p = 0.397	n/a	n/a
Community service providers	No: LOR = 0.006, z = 0.046, p = 0.963 Yes: LOR = 0.091, z = 0.885, p = 0.376 Q B = 0.27, p = 0.605	No: LOR = 0.075, z = 0.477, p = 0.633 Yes: LOR = 0.237, z = 2.339, p = 0.019* Q B = 0.74, p = 0.389	n/a

p < 0.05;

p < 0.01.

Funding and program continuance

Theme 1: A lack of long-term funding often results in program termination or poor service delivery.

Several studies noted that programs had been terminated during the study period, in some cases due to funding issues (Barton et al., 2008; Liberman et al., 2021). In multiple program sites, a lack of leadership and difficulties operating the program past a term of grant funding led to program termination. For example, in its Alabama site, the Targeted Re-Entry (TR) program suffered from a lack of administrative commitment and a weakened community network, which led to the decision to “…discontinue TR programming when the funding from the national [Boys and Girls Clubs of America] office was due to expire in December, 2006” (Barton et al., 2008, p. 26). Similarly, a reentry program implemented in Texas was terminated due to a lack of funding after Second Chance Act grant funding expired, while services in California and Oklahoma sites were reduced and enrollment waned (Liberman et al., 2021). Though it is not clear if the Green Reentry program is still running, Lindquist et al. (2014) also noted that “…at the conclusion of grant period, it was unclear how the overall planning and coordination of green activities and the oversight of specific green projects would be accomplished” (pp. 6–23). Though not all programs experience funding issues, programs seem to experience instability and challenges towards program fidelity due to a lack of funding, perhaps hampering the effectiveness of such programs.

Where funding was noted as a program success, improvements in service delivery were described. For example, additional funds provided to an Arkansas program resulted in “…the addition of two case managers to the [Arkansas Division of Youth Services] staff, a reduction in the role of the current aftercare provider, ongoing training in case management (to include the [Targeted Reentry] staff), and the development of a mentoring program in partnership with a local university” (Barton et al., 2008, p. 41). Similarly, Iutcovich and Pratt (1998) found that in Philadelphia, an adequate budget allowed smooth operation of the Pennsylvania Intensive Aftercare Program, while Miller and MacGillivray (2002) reported that service expansions were made possible by combining funding sources. Last, in its Virginia site, a Second Chance Act reentry program was able to renew its funding and “…funding for the program [continued to be] uninterrupted” throughout the study period (Liberman et al., 2021).

Staffing and organization-level findings

Theme 2: Aftercare/resettlement programs experience high levels of staff turnover due to a lack of attractive job features.

Perhaps the most commonly noted challenge to program delivery and functioning was a high level of staff turnover (Barton et al., 2008; Dum & Fader, 2013; Flynn et al., 2003; Hazel et al., 2008; Hazel et al., 2012; Iutcovich & Pratt, 1998; Jain et al., 2018; Lindquist et al., 2014; Wiebush et al., 2005). While aftercare/resettlement programs may experience high staff turnover for a number of reasons, poor compensation, working conditions, staff demoralization, health and retirement benefits, qualification levels, and burnout were all noted as contributing factors (Barton et al., 2008; Dum & Fader, 2013; Flynn et al., 2003; Iutcovich & Pratt, 1998; Lindquist et al., 2014; Miller & MacGillivray, 2002). For example, in the Arkansas implementation of the Targeted Re-Entry program, “issues of insufficient training and the lack of clinical supervision for the staff would come up again and again over the course of the project and [were] never completely resolved within the organization itself” (Barton et al., 2008, p. 41). In the Pennsylvania Intensive Aftercare programs, “managers experienced challenges in finding and retaining qualified, experienced staff, and cited the nature of the work and compensation issues as contributing to staff turnover and ‘burnout’” (Iutcovich & Pratt, 1998, p. 12).

Theme 3: Poor leadership and organizational functioning often compromise aftercare/resettlement program delivery.

A lack of leadership, administrative changes, and poor supervision were cited in some studies (Barton et al., 2008; Flynn et al., 2003; Liberman et al., 2021). For example, in the Wisconsin implementation of the Targeted Re-Entry program, there were difficulties establishing an office which would supervise aftercare/resettlement staff, new management was not suited to supervising the program, and “the [Targeted Re-Entry] project did not always receive the kind of attention it needed” (Barton et al., 2008, p. 46). Poor relationships between staff and leadership were also noted. In the Network Aftercare System, Flynn et al. found that “some of the staff seemed to feel that while they could communicate with those below them, upper management were not listening to their needs or being supportive of them … comments about opinions or issues “not being taken seriously” by supervisors were raised” (2003, p. 130).

A number of studies cited issues regarding role definition and conflicting authority. Staff and partners were often unsure of their responsibilities or the responsibilities of others, leading to inefficiencies and time-wasting (Flynn et al., 2003; Hazel & Hampson, 2015; Hazel et al., 2008; Ipsos MORI, 2012; Andersson Vogel et al., 2014; Wright et al., 2012). For instance, in the Leaving Care Project, “when the coordinators started their work, no specific job description existed, but was formulated along the way” (Andersson Vogel et al., 2014, p. 250). In other cases, staff felt that they lacked the authority or power to properly do their jobs (Dum & Fader, 2013; Flynn et al., 2003; Andersson Vogel et al., 2014). For example, aftercare/resettlement workers in the Powelton Aftercare program “felt powerless because they lacked any ability to impose meaningful consequences upon their clients” (Dum & Fader, 2013, p. 794), indicating a lack of efficiency in implementing the program as intended and likely impacting the extent to which the program is adequately affecting participants.

Theme 4: Staff and leadership often show high levels of care and dedication in their work.

Some successes noted regarding staff included a high level of care towards youth, relevant and extensive prior experience, and dedication to the program (Barton et al., 2008; Dum & Fader, 2013; Miller & MacGillivray, 2002; Wiebush et al., 2005). Successful cooperation and communication were also common themes throughout studies. Good teamwork and interagency cooperation were cited as program successes, often exemplified by frequent meetings, strong communication, established community partnerships, and continuity of care across jurisdictions or program phases (Flynn et al., 2003; Ipsos MORI, 2012; Liberman et al., 2021; Lindquist et al., 2014; Wiebush et al., 2005). Regarding program leadership, strong management and administrative support, consistent leadership, and proper program oversight were found in some studies (Barton et al., 2008; Ipsos MORI, 2012; Lindquist et al., 2014; Wiebush et al., 2005).

Program fidelity

Theme 5: Program fidelity is often hampered by weak or poorly communicated program theory, aims, and operation.

Interventions seemed to suffer from a lack of program fidelity in many cases. For example, programs operated under various theoretical bases by different staff (e.g., treatment vs. punishment-based), provided inconsistent or irrelevant case planning, emphasized some stages of the aftercare/resettlement process more than others, or changed their goals after experiencing bureaucratic challenges (Andersson Vogel et al., 2014; Barton et al., 2008; Flynn et al., 2003; Hazel & Hampson, 2015; Hazel et al., 2008; Jain et al., 2018; Liberman et al., 2021; Miller & MacGillivray, 2002). While Flynn et al. (2003) found that program administrators were not using the same treatment philosophies, Barton et al. similarly noted that “stakeholders had little understanding of the big picture of IAP and TR in the early stages of this project” (2008, p. 50). In the RESET program, Hazel and colleagues note that “…there were fewer consensuses about the aims and objectives of RESET … there had been a lack … communication about what the project was really about and what it is meant to do” (2008, pp. 30–31). Andersson Vogel et al. illustrated the salience of program fidelity, finding that “in the end, [the Leaving Care project] could rather be described as 24 different projects loosely connected by a few mutual grounds than as a structured program or a systematic way of working” (2014, p. 256).

Theme 6: Administrative duties are not always aligned with direct service delivery, which can hamper program fidelity.

Additionally, challenges related to service provision were noted throughout the process analyses. For example, paperwork responsibilities were noted as a barrier to service provision, where staff had little time to spend with youth because they had to fulfill administrative tasks (Abrams et al., 2008; Dum & Fader, 2013; Hazel et al., 2008). Similarly, administrative meetings and emails were perceived to take time away from front-line work and service delivery in the RESET program (Hazel et al., 2008). On the other hand, missing, duplicated, or incorrectly completed paperwork can also waste time, and universal and straightforward documentation procedures (e.g., making sure that staff members know the purpose of forms and case files, or having inter-agency information sharing systems) were identified as an unfulfilled need in some programs (Andersson Vogel et al., 2014; Barton et al., 2008; Hazel et al., 2012; Iutcovich & Pratt, 1998; Wright et al., 2012). Documentation and case plans were sometimes not made, or transition teams were not in place, leaving youth without proper aftercare/resettlement services (Andersson Vogel et al., 2014; Barton et al., 2008).

Caseload feasibility

Theme 7: Caseload feasibility is variable across aftercare/resettlement programs; however, where caseloads are high, program fidelity may be diminished.

Findings on staff to participant caseload ratios and casework feasibility were mixed. Three studies reported caseloads that were too high, where low staff to participant ratios impeded program delivery (Citizens' Committee for Children of New York, NY, 2000; Flynn et al., 2003; Liberman et al., 2021). In the Texas implementation of an aftercare/resettlement program, “downsizing of the Houston District Parole Office” was the reason for increased caseloads (Liberman et al., 2021, p. 10). In addition to managing too many cases, aftercare/resettlement workers must juggle numerous responsibilities, which may hamper their ability to provide participants with high quality case management. For example, in the London Youth Reducing Reoffending Program, “Resettlement Brokers were faced with the challenge of trying to secure education or employment opportunities for young people whilst also addressing their wider needs, such as community issues and family relations” (Ipsos MORI, 2012, p. 7). However, another four studies noted that caseloads were small and met program standards (Barton et al., 2008; Hazel et al., 2008; Iutcovich & Pratt, 1998; Wiebush et al., 2005). In Colorado, small caseloads and a team approach to case-management resulted in “constant interaction and communication around assessment, planning, service delivery, and monitoring issues for each youth … more coordinated interventions for youth … [and], a source of mutual support for staff, which proved critical given the difficult nature of the IAP population” (Wiebush et al., 2005, p. 28).

Parents and families

Theme 8: A high degree of parent involvement is needed for successful program implementation.

The engagement of parents and families was frequently discussed as a major challenge to successful transition processes or as weak points of the programs. For instance, there was a need for greater parent and family involvement, or for better services dedicated to involving families (Abrams et al., 2008; Jain et al., 2018). In fact, process findings often showed that parents were generally uninvolved in the lives of youth clients, or in the transition processes of their children (Andersson Vogel et al., 2014; Citizens' Committee for Children of New York, NY, 2000; Dum & Fader, 2013; Lindquist et al., 2014; McKay et al., 2014). In some cases parents were identified as problematic and as barriers to program success, especially when they were noted as experiencing behavioral or substance use issues, not fulfilling their roles as guardians (e.g., facilitating condition incompliance, failing to enroll youth in school, or missing court dates), or when they had adversarial relationships with their child or program staff (Dum & Fader, 2013; Lindquist et al., 2014; McKay et al., 2014).

While failures related to parent and family involvement could stem from multiple areas, some findings may provide context towards this problem. For example, where parents are non-English speakers, it may be difficult for them to engage in the reentry process. Jain et al. note that in an aftercare/resettlement program in California, families and youth of underrepresented ethnicities may have disproportionally experienced communication problems:

“There was a low availability of translators and individuals who could communicate with youth and their families in a culturally responsive way, especially for underrepresented populations like some specific dialects and/or less common Southeast Asian languages” (2018, p. 3673).

In the Targeted Re-Entry program, the Wisconsin site tackled this barrier by bringing in a volunteer to translate for Spanish-speaking parents (Barton et al., 2008). Two additional barriers to family involvement stood out as reported in the three sites of the Tribal Green Reentry Program evaluation (McKay et al., 2014). First, parents were reported to be unaware of their roles in the transition process or the ways in which they could be involved in their child's program involvement (McKay et al., 2014). Second, it seemed that parents were sometimes kept in the dark with regard to their child's progress in the aftercare/resettlement program, or they were not notified of changes in transition plans (McKay et al., 2014).

Parent engagement was noted as a success in only three studies: Hazel et al. (2008), McKay et al. (2014), and Sinclair et al. (2021). In the Tribal Green Reentry Initiative, though some parents were found to be unsupportive, others were found to act as proponents of program engagement and transition success; parents were able to emotionally support their children while they were enrolled in the aftercare/resettlement programs, to encourage adherence to program conditions, and to promote youth engagement in programming (McKay et al., 2014). Similarly, Sinclair et al. (2021) found that transition specialists coordinated with client families to promote youth engagement.

Contacts in custodial facilities

Theme 9: Minimal contact between aftercare/resettlement workers and youth in custodial facilities compromises the ability to develop strong support relationships.

According to the theory of change presented in this review, contact between program staff and youth participants should be made early on, ideally at the start of the incarceration period (Bateman et al., 2013; Byrnes et al., 2002; Travis & Petersilia, 2001). Both Iutcovich and Pratt (1998; Philadelphia site) and Liberman and colleagues (2021; California, Texas, and Virginia sites) found that contacts were fewer than expected in the pre-release stage. For example, though programs aimed to facilitate multiple contacts between case managers and youth in custodial facilities, oftentimes only one contact could be made (Iutcovich & Pratt, 1998; Liberman et al., 2021). Generally, the purpose of these meetings was characterized as “[introducing youth to] the program and services provided” (Liberman et al., 2021, p. 10). Similarly, contact between program staff and the parents/families of clients was also minimal at the pre-release stage (Iutcovich & Pratt, 1998; Liberman et al., 2021). For example, Iutcovich and Pratt note that “case managers had only one contact documented with 13 of 32 families [in the pre-release phase], making an average of less than one contact per family” (1998, p. 46). In some cases, it seemed that long distances between community and custodial facilities, or caseloads with clients in multiple far-apart custodial facilities, were a barrier to contacts between aftercare/resettlement workers and their clients (Barton et al., 2008; Hazel & Hampson, 2015; Hazel et al., 2008; Liberman et al., 2021; Wright et al., 2012). Where caseworkers were assigned from the communities that young custody-leavers would be released to, they could not travel to custodial facilities to meet; alternatively, if aftercare/resettlement teams were close to custodial facilities but the young person was far from home, it became difficult to coordinate release supports in release destination communities (Barton et al., 2008; Hazel & Hampson, 2015; Liberman et al., 2021; Wright et al., 2012). Minimal contact between case managers and youth in the custodial phase of aftercare/resettlement directly compromises an important component of aftercare/resettlement programming: the development of trusting support relationships that allow youth to buy-in to programs and be motivated to change (Hazel et al., 2017).

Only one study noted major successes in pre-release contacts (Wiebush et al., 2005). In the Virginia implementation of the Intensive Aftercare Program (IAP), Wiebush et al. found that:

“In addition to ongoing telephone communication between the case managers and parole officers while the youth was at the institution, [aftercare] parole officers made monthly visits to see the youth and case managers at the institution. The parole staff also contacted parents twice per month while youth were institutionalized, and the institutional case managers traveled to Norfolk periodically to meet with the parents of youth on their caseloads. From the perspective of [aftercare] managers and staff, the improved coordination and communication in planning and service delivery were [the program's] major accomplishments” (2005, p. 41).

For the IAP sites in Colorado, Nevada, and Virginia, visits to participants who were incarcerated were made by case managers or parole officers at least once per month, and “the monthly number of face-to-face contacts between [Intensive Aftercare Program] youth and parole officers was at least twice that which occurred in the [treatment as usual] group” (Wiebush et al., 2005, p. 26).

Short sentences

Theme 10: Sentence lengths are often short, leaving very little time for transition planning, preparation activities, or securing post-release supports.

A number of studies noted that young people who had shorter sentences received insufficient aftercare/resettlement support while in custody (Barton et al., 2008; Hazel & Hampson, 2015; Hazel et al., 2008; Hazel et al., 2012; Liberman et al., 2021; Wright et al., 2012). For example, in California, Liberman et al. (2021) noted that young people who had shorter lengths of incarceration were not always able to participate in pre-release programming, and were sometimes not eligible to be enrolled in re-entry services. Similarly, in the Targeted Re-Entry program, the Arkansas site found that young people with shorter lengths of incarceration did not have enough time to complete two pre-release courses that were required for enrollment in the aftercare/resettlement program (Barton et al., 2008). It is clear that, while long periods of incarceration are no doubt harmful to young people, re-entry preparation takes time and cannot always be completed in one or 2 months.

Further, some studies found that aftercare/resettlement staff simply did not have enough time to put proper supports in place for youth before they were released back into the community. For instance, procuring living arrangements for young custody-leavers is a particularly important component of positive re-entry, but short sentences may prevent housing to be secured (Hazel et al., 2012). In addition, practices such as temporary release, while incredibly helpful in gradually bringing young people back to their communities, take a long time to put in place and cannot be offered when sentence lengths are short (Hazel & Hampson, 2015; Wright et al., 2012).

Working with custodial staff and institutions

Theme 11: A lack of communication between custodial staff and aftercare/resettlement workers creates challenges in transition processes and gaps in service.

Working relationships between aftercare/resettlement program staff or administration and custodial institutions appeared to pose a major challenge in delivering reentry programming. Some studies found that custodial staff had a limited understanding of, support for, or even awareness of the aftercare/resettlement programs (Barton et al., 2008; Wiebush et al., 2005; Wright et al., 2012). For example, in one facility, access to aftercare/resettlement programming was treated as a reward by one custodial facility, rather than as treatment, which hampered the process of preparing for release (Barton et al., 2008).

Some studies noted that poor relationships/partnerships between program staff and custodial facilities contributed to a low number of client-aftercare worker contacts, delays in creating treatment plans, or difficulties with involving families. A commonly cited issue was the failure of custodial facilities to communicate scheduled release dates to aftercare/resettlement staff (Barton et al., 2008; Iutcovich & Pratt, 1998; Liberman et al., 2021). For instance, Barton and colleagues note that “staff were not even aware of when releases were scheduled, and youths were being released without the [aftercare program] case managers being notified” (2008, p. 45). Similarly, Liberman et al. note:

“Reentry staff were often not informed of when a youth would be released, which hampered their ability to contact youth and their family to develop an individualized plan to guide the transition home. In many cases reentry planning was not able to occur until the youth had returned home” (2021, p. 10).

The need for advanced knowledge of client release dates was directly related to a program's ability to engage youth in program services prior to release, to create effective transition plans, and to allow youth and case managers to develop positive relationships (Barton et al., 2008; Iutcovich & Pratt, 1998; Liberman et al., 2021).

Service provision during release from custodial facilities

Theme 12: Aftercare/resettlement programs often fail to ensure youth arrive in the community with immediate supports.

The continuance of service provision is perhaps the most emphasized component of the intensive transition stage in aftercare/resettlement programming. When youth experience large gaps in services during their transition from custody to their communities, it hampers their ability to lead prosocial lives (Hazel et al., 2017). Several studies noted gaps in care during the time immediately following a youth's release from custody (Abrams et al., 2008; Barton et al., 2008; Citizens' Committee for Children of New York, NY, 2000; Hazel & Hampson, 2015; Hazel et al., 2012; Iutcovich & Pratt, 1998; Sinclair et al., 2021; Wright et al., 2012). For example, one study noted that while youth may have medical care available in custody, “once [youth] are discharged … Medicaid does not automatically continue unless the young person's family meets the standard eligibility criteria, which requires either no wages or very low earnings that do not exceed the minimum wage” (Citizens' Committee for Children of New York, NY, 2000, p. 18). Iutcovich and Pratt found that in the Pennsylvania Intensive Aftercare programs, “once released from institutional placement, approximately 50 percent of the youth were not contacted within the first 24 hours” (1998, p. 12). School enrollment immediately after release, or continuation of GED programming, was also noted as a common gap in services (Abrams et al., 2008; Citizens' Committee for Children of New York, NY, 2000; Dum & Fader, 2013; Hazel et al., 2012; Iutcovich & Pratt, 1998).

Some studies noted a need for better service provision surrounding basic life necessities, like food security and housing (Abrams et al., 2008; Hazel & Hampson, 2015; Hazel et al., 2012; Lindquist et al., 2014; Miller & MacGillivray, 2002; Wright et al., 2012). In particular, housing presents a challenge to the re-entry process, as accommodations are often inappropriate or unstable for young people (e.g., without proper familial support or supervision), far away from custodial sentences (e.g., where youth are returning to their communities, but are placed in far-from-home institutions that makes it difficult to schedule pre-release home visits), not permanent (e.g., where young people are placed in temporary group homes), or difficult to secure (Abrams et al., 2008; Hazel & Hampson, 2015; Hazel et al., 2012; Miller & MacGillivray, 2002; Wright et al., 2012). In the North West Resettlement Consortium, it was found that:

“young people are not currently deemed to be homeless until they exit custody, which means that local authority homelessness departments will not consider their needs until they have left custody … young people are left unsure of their future housing situation throughout the whole custodial period” (Hazel et al., 2012, p. 43).

Related to the issue of gaps in service is the motivation of a young person to change, or lack thereof when proper supports are not put in place. This barrier to a young person's inner change process is illustrated saliently in Hazel and Hampson's evaluation of the Resettlement Broker Project, where a practitioner stated that “…children tend to be full of hope and determination at release, but then typically they become disillusioned when promised services fail to materialize … the moment of opportunity has passed” (2015, p. 84).

Imperative to service continuity are stable and long-term staff members, who develop relationships with youth and become familiar with their specific needs. In discussing the many hours it takes to develop these relationships, one stakeholder in the RESET evaluation noted “…I don't think it's any coincidence that [the young people are] getting on with their support worker … the worker having the time to build the relationship, I can't stress that enough” (Hazel et al., 2008, p. 53). Where youth were able to develop these long-term relationships with staff, the relationships were generally cited as being very beneficial (Abrams et al., 2008; Hazel & Hampson, 2015; Hazel et al., 2008; Ipsos MORI, 2012). However, in programs which suffered from high staff turnover, these relationships may have been interrupted or multidisciplinary teams may not have been consistent (Jain et al., 2018). For example, in the Network Aftercare System, Flynn et al. found that a lack of consistent staff was a barrier to relationships between participants and aftercare/resettlement workers, and that “staff need to understand and buy into the IAP model … this level of knowledge and commitment is difficult to attain when turnover is high or when positions remain unfilled” (2003, p. 136).

Community partnerships

Theme 13: Poor relationships with community organizations and a lack of pre-specified services negatively impacts service access.

Related to proper service provision are the relationships that aftercare/resettlement programs have with community partners. While youth are ideally linked with community services and agencies prior to release or immediately upon release, this is often not the case. This may be due to a lack of, or generally poor quality, relationships with community service providers and partners, which was noted in a number of studies (Abrams et al., 2008; Barton et al., 2008; Hazel et al., 2012; Iutcovich & Pratt, 1998; Jain et al., 2018; Miller & MacGillivray, 2002; Wiebush et al., 2005). Further, one study noted that community organizations were hesitant to engage with young people who had offended (Hazel et al., 2012).

Difficulties in creating partnerships with specialized services, such as drug and alcohol treatments, family therapy, or services for youth with disabilities, were also noted as a challenge after release from custody (Abrams et al., 2008; Barton et al., 2008; Sinclair et al., 2021; Wiebush et al., 2005). In the Targeted Re-Entry program, the Arkansas site had “little success involving the faith community in reentry programming”; similarly, in Alabama, “the [Targeted Re-Entry] program attempted to involve the faith community and Volunteers of America with little success” (Barton et al., 2008, p. 36; p. 23). Further, the age at which youth engage in the program may not be taken into account when organizing supports; for example, Sinclair et al. (2012) noted that “there was … a lack of transition from youth to adult services … one transition specialist mentioned the need for age-appropriate substance use program, for example, alcoholics anonymous” (p. 10).

Follow-up and long-term engagement

Theme 14: Service provision is lacking after the intensive transition stage.

Though aftercare/resettlement services will generally taper off once youth have successfully reintegrated into their communities, aftercare/resettlement programming generally aims to engage youth for, at minimum, multiple months after their release from custody, in order to produce long-term benefits. However, a number of studies cited problems related to follow-up care. For example, Abrams et al. (2008) noted that aftercare workers rarely maintained contact with their clients after 6 weeks post-release, despite program guidelines stating that bi-monthly contact was to be made for another 2 months; in this case, youth felt unsupported by the program. In the RESET program, aftercare/resettlement staff felt that the program was not long enough, and that they weren't given enough time to deliver program services or to facilitate positive re-entry experiences for their clients (Hazel et al., 2008). Similarly, Dum and Fader (2013) noted that youth were cut off from various program components before they were ready to be without these services. In the Powelton Aftercare program, aftercare workers felt that they were put “in a no-win position … if clients were staying out of trouble, that was considered a success that led to termination of services” (Dum & Fader, 2013, p. 798). Youth can also “[age] out and [be] dropped by supports,” leaving them with no further support (Sinclair et al., 2021, p. 10). The problem of clients who age out of youth services has been noted across multiple implementation studies, with many noting that young people who turn 18 or become legal adults are cut off from their re-entry teams and services (Hazel & Hampson, 2015; Hazel et al., 2008; Hazel et al., 2012; Sinclair et al., 2021; Wright et al., 2012).

Though a lack of program follow up may be due to program deficiencies, such as understaffing or poor leadership, the voluntary nature of many aftercare/resettlement programs can also pose a challenge to long-term engagement (Citizens' Committee for Children of New York, NY, 2000; Dum & Fader, 2013; Iutcovich & Pratt, 1998; Lindquist et al., 2014). For example, a common problem appears when clients “no-show”; that is, they do not show up to meetings with aftercare/resettlement staff, or simply refuse to engage in voluntary programming (Citizens' Committee for Children of New York, NY, 2000; Dum & Fader, 2013; Lindquist et al., 2014). In the case of the London Youth Reducing Reoffending Program, there were “instances where young people were removed from the Programme as a result of their lack of willingness to participate and engage” (Ipsos MORI, 2012, p. 5). Similarly, aftercare workers in the Wisconsin Targeted Re-Entry program felt “frustrated that the youths they served were often, as a group, unmotivated and lacked ambition … they would not hold onto jobs, were reluctant to work hard, and dropped out of training and educational assistance programs” (2008, p. 51). Further, aftercare staff often had no authority to impose consequences or ensure compliance with parole or program conditions, resulting in difficulties with service provision over time (Barton et al., 2008; Dum & Fader, 2013; Iutcovich & Pratt, 1998).

Despite noted challenges, two studies found that follow-up engagement was successful. Sinclair et al. note that “…the youth reported that the transition specialist helped them stay engaged in school, by checking in and keeping them on track to accomplish their goals, and providing transportation if necessary (e.g., to get to a job interview)” (2021, p. 11). Another study found that, due to positive relationships between clients and aftercare workers, contact lasted for long periods of time post-release (Ipsos MORI, 2012). Further, another study noted that the voluntary nature of the re-entry services provided allowed aftercare/resettlement workers to develop a positive rapport with their clients (Hazel et al., 2008).

Relationships with justice personnel in the community

Theme 15: Power imbalances and resentment sometimes lead to inefficient relationships between community-based criminal justice workers and aftercare/resettlement staff.

Relationships with justice personnel continued once youth were released into the community, usually with parole officers (POs) who were not directly employed by the aftercare/resettlement programs. Though aftercare/resettlement workers and parole officers would likely have many similar interests or goals for their clients, a number of studies noted poor working relationships between POs or police and program staff that were fraught with tension or resentment (Barton et al., 2008; Dum & Fader, 2013; Hazel et al., 2012; Lindquist et al., 2014). For example, Dum and Fader explained that:

“The relationships between aftercare workers at Powelton and [parole officers] were tenuous at best. Aftercare workers often felt that [parole officers] did not buy into the value of Powelton's services and failed to enforce the mandate for their clients to attend the program. They shared a common belief that POs were lazy and relied too heavily on the aftercare workers to get information about their clients in order to complete paperwork” (2013, p. 794).

Further, in the Wisconsin site of the Targeted Re-Entry program, relationships between “[parole agents and] TR staff was not always as strong as it had been,” and when meetings between these agents were not being set up by the parole officers, “[Targeted Re-Entry] staff felt that they could not initiate these” (Barton et al., 2008, p. 47).

In addition to tense relationships, major differences in objectives between enforcement/parole workers and aftercare/resettlement teams seem to hamper the re-entry process. More specifically, several studies noted that aftercare/resettlement staff felt that POs were too quick to file technical violations or breaches/violations of parole conditions against young people in the community (Barton et al., 2008; Hazel et al., 2012). For example, in the North West Resettlement Consortium, one practitioner noted that:

“…every time [young people] bounce back [into custody] they lose all the provision that we've put in place and it gets harder and harder for us to get it – especially accommodation and education, as you can imagine … so we really need the police on board to understand what our objectives are” (Hazel et al., 2012, p. 35).

Clearly, as housing placements, employment, and other services are already difficult to put in place for youth, it becomes frustrating to aftercare/resettlement staff when young custody-leavers are quickly sent back to custody for violating conditions of release.