The effectiveness of abstinence-based and harm reduction-based interventions in reducing problematic substance use in adults who are experiencing homelessness in high income countries: A systematic review and meta-analysis: A systematic review

Types of studies

We included impact evaluations that employed study designs which included some form of comparison, which included all studies categorised as either ‘Randomised Controlled Trials’ or ‘non-experimental designs with a comparison group’ from the studies in the EGM. Because of time and resource limitations, we only included studies published in English.

Types of participants

There are a number of definitions of homelessness available, reflecting differences between countries and over time. There are also different forms of homelessness, taking into account the length of time someone has been experiencing homelessness, distinctions between living on the street or in their vehicles, or having a temporary place to stay. We used the definition of homelessness used in a recently published Campbell Collaboration protocol. This definition is:

‘Homelessness is defined as those individuals who are sleeping rough’ (sometimes defined as street homeless), those in temporary accommodation (such as shelters and hostels), those in insecure accommodation (such as those facing eviction or in abusive or unsafe environments) and those in inadequate accommodation (environments which are unhygienic and/or overcrowded). (Keenan, 2020, p. 13)

Our focus was on adults (men and women aged 18 years and over), undertaken in any high-income country. Studies of families or children were excluded from the review. These were excluded because in many countries (particularly the UK), there are different legal frameworks that apply to homeless families and children, and thereby access to different types of services, and different outcomes expected. We did not include studies that included people below age 18, even if some of the population was equal to or older than age 18.

Types of interventions

The review synthesised findings about the impact of a range of interventions aimed at reducing, stopping, or preventing problematic substance use or harms associated with problematic substance use. In developing the protocol for the review and drawing on the extant effectiveness literature, we created a typology of such interventions. This typology is set out in Table 1. The typology was discussed and validated with an expert panel of academics, policy makers, experts by experience, and practitioners involved in problematic substance use treatments targeted at people experiencing homelessness, held in November 2020. Interventions listed in the left-hand column of the typology in Table 1 are abstinence-based; interventions in the right-hand column are based on harm reduction approaches:

Abstinence-based – interventions that require participants to abstain from substance use, or whose goal is to achieve abstinence from substance use.

Interventions listed in the centre column are often practiced using both abstinence-based and harm reduction approaches. These interventions are relevant to addressing the first objective of this review (how effective are interventions designed to reduce substance use in adults who are experiencing homelessness), but were excluded from analysis intended to address the second objective around the effectiveness of harm reduction versus abstinence based interventions.

The evidence synthesised in this review relates to adults experiencing homelessness. Many of the interventions covered by this review are used more generally and with other populations, and as such there may be evidence which is not specific to adults experiencing homelessness. It is therefore possible that there is evidence that specific interventions work in general, but no evidence that they work for adults experiencing homelessness.

We provide an overview of the level of evidence in the included studies for each of the interventions listed in this typology in Table 5 in the results section.

The definitions of each of the interventions included in the typology can be found in Supporting Information: Appendix 1.

Interventions aimed at reducing problematic substance abuse can also be categorised descriptively by type. The review team identified three such categorisations, namely psychosocial, treatment through medication (pharmacological), and non-medical interventions. These descriptive categories are not material to the analysis underatken or to addressing the review objectives, but were nevertheless useful.

We also extracted information on the experience of the comparison group in each individual study (e.g., no intervention, treatment as usual, wait-list, other treatment).

Types of outcome measures

We included impact evaluations that included a substance use outcome. Substance use is measured in a number of ways in primary studies, for example:

Number of days per month substances are used

We identified a range of continuous and binary outcomes in the reviewed studies. Where possible, we converted these into the same metric (e.g., Hedges' g) for the purpose of meta-analysis (Borenstein, 2010). Where effect sizes were converted from a binary to continuous measure, we undertook a sensitivity analysis to investigate the effect of the inclusion of studies with a converted effect size in the meta-analysis (described further below).

Our primary focus of this review is on the effectiveness of substance use interventions in relation to substance use outcomes. In particular, we are interested in the relative effectiveness of abstinence based and harm reduction-based interventions.

While the primary objective of abstinence-based interventions is to reduce, stop, or prevent problematic substance use, this is not the primary objective of harm reduction interventions. These interventions do not place an abstinence requirement on individuals engaging in treatment, but can and do empower service users to make personal abstinence decisions. Nevertheless, many intervention studies that cover harm reduction interventions do measure substance use outcomes, often involving a direct comparison between harm reduction interventions of interest and abstinence-based comparisons.

Duration of follow-up

We excluded studies which measured outcomes beyond 36 months.

Types of settings

We included studies that took place in any type of setting that employed abstinence-based or harm reduction-based interventions. We included studies that took place in high-income countries only, recognising the substantive differences between high- and low-income countries in terms of access to resources and services, and drivers of homelessness (Magwood, 2020). We used the same definition of high-income country as White's (2018) EGM, that is, high-income countries according to the World Bank classification (World Bank).

Electronic searches

White et al. (2021) describe the process for identifying and searching for the studies included in the EGMs and we do not repeat this here.

The review team shortlisted studies for this review by identifying primary studies and systematic reviews from the EGM (2020 and 2022 editions) (by filtering using the previously coded classifications) which appeared to meet the inclusion criteria for this review. Systematic reviews from the EGM were unpacked, and their constituent studies added to the shortlist for screening. We also identified another systematic review, Moledina et al. (2021), during the searching process.

Searching other resources

In November 2020 the review team issued a call for grey evidence (with a deadline of 8th January 2021) which was disseminated through the review team's and review funder's social media channels, inviting people with lived experience, researchers, commissioners, service providers and wider stakeholders to submit relevant grey evidence for consideration in the review. Specifically, the call was for evidence that is:

(1)

empirical, based on research that:

We received eight studies in response to the call, none of which were impact evaluations or systematic reviews, and hence were not eligible for inclusion in the review.

The reviewers hand searched key journals, using similar search terms and date ranges as Singh and White (2022). While some may have already been searched as part of the evidence and gap map, this targeted journal search and more substance use and treatment focused search aimed to further ensure the capture of all existing literature and evidence. The hand searched journals included:

Drug Alcohol Dependency

Psychiatric Services Journal

American Journal of Public Health

BMC Medicine

Journal of Substance Abuse Treatment

European Journal of Homelessness

Housing Studies

Social Policy and Administration

Addiction Research and Theory

Drugs Education Prevention and Policy

International Journal of Drug Policy

Addiction

Description of methods used in primary research

We included studies that used a design rated at levels 3, 4 and 5 of the Maryland Scientific Methods scale, that is, experimental (randomised) studies and non-experimental studies that included a baseline measure of the outcome. The studies measured the effectiveness of an intervention designed to reduce problematic substance use against another intervention or a control group (no intervention, treatment as usual, wait-list).

Data extraction and management

Two reviewers extracted data from eligible studies. This included details of the study, quantitative data required for meta-analysis, and confidence in the study's findings (using the Campbell Collaboration's critical appraisal tool for primary studies – White et al., 2020). Coding disagreements were discussed and passed to the lead reviewer for resolution. We then extracted data for the following, where it was not already present in the EGM spreadsheet:

Publication details (e.g., authors, year, source, study location)

In addition, one reviewer examined each study to assess whether they did the following:

(1)

Present a clear theory of change linking the intervention(s) being tested to substance use outcomes, and if so, what that was.

We defined a clear theory of change as presenting information on the following as a minimum, drawing on the Better Evaluation guidance (Betterevaluation.org, ND):

Present an action theory: the authors present an explanation of intervention activities that need to be undertaken and level of success needed to produce the intended impact on substance use. There should be an explanation of why these activities have been designed as such and why the particular level of success is needed in order to see change and the final intended impact.

Assessment of risk of bias in included studies

Studies selected for this review were assessed for confidence of findings using a critical appraisal tool for primary studies developed by the Campbell Collaboration for use with maps and reviews. We used this tool over the Cochrane risk of bias as this was the tool used by White et al. (2019) to rate the confidence findings in their EGM, the primary source of studies for our review. Details of these tools and the assessment process are set out in White et al. (2019), and the tool itself can be found in Supporting Information: Appendix 2. The tool for primary studies has seven items which relate to (1) study design, (2) blinding, (3) power calculations, (4) attrition, (5) description of the intervention, (6) outcome definition and (7) baseline balance. Each of these seven items is rated as implying high, medium or low confidence in study findings. Overall quality is assessed using the ‘weakest link in the chain’ principle, so that confidence in study findings can only be as high as the lowest rating given to any of the critical items.

For studies that came from the EGM, we used the critical appraisal results undertaken by the EGM authors, with the exception of two cases where we disagreed with the original rating. For those cases, we agreed with the review funder that we would present our rating in this review instead of the original rating. However, we also identified a number of studies from our additional searching that needed critical appraisal. The team undertook independent critical appraisals of 25% of the newly included studies in order to make sure that everyone was interpreting the criteria in the tool in the same way. This process demonstrated that the team were interpreting the criteria in the same way. For the remaining studies, each study was coded by one person with checking of results by another team member.

Measures of treatment effect

The included studies reported on continuous, ordinal, dichotomous and count-based outcome measures. We calculated most standardised effect sizes using the David-Wilson Practical Meta-Analysis Effect Size Calculator (Wilson, 2019). Included studies presented the following types of statistical information that we extracted to calculate standardised effect sizes:

(1)

To calculate a SMD for continuous data: means and SDs, means and SEs, unstandardised regression coefficients and SDs/SEs or 95% confidence intervals (CIs), results from f-tests, results from t-tests.

Three of the included papers, the authors did not present the rates of substance use in the treatment and control group, only their own rate ratios. We are unaware of a formula to convert a rate ratio to other types of effect size and conceptually we do not consider these comparable. Therefore, we present these studies individually and do not include them in any meta-analysis. Therefore, for these studies, we have reported author-calculated rate ratios and associated 95% CIs only, that is, the rate of an event in one group divided by the other. These papers were all associated with the At Home/Chez Soi RCT study in Canada.

One study only presented medians and interquartile ranges (IQR) (Schumacher, 2000). In order to calculate effect sizes for these studies, we used the formula in Wan (2014) to calculate an appropriate mean and SD that was then used to calculate Hedge's g. This study was not included in any meta-analysis. The formulas can be found below, where m is the median, is the approximate mean, q1 is the first quartile of the IQR and q3 is the third quartile of the IQR: (Figure 1).

A few studies provided author-calculated odds ratios with associated 95% CIs. In order to approximate the variance of the log(OR), needed to include the studies in a meta-analysis, we used the following formula where UB and LB refer to the upper and lower bounds of the CIs of the log odds ratio: (Figure 2).

Finally, one of the studies reported results from a Poisson regression, a type of regression developed to model counts or frequencies. In this case, we used the Poisson Regression Effect size calculator to calculate both a Cohen's SMD and Rate Ratio.

In all cases, we compared the calculated effect sizes to the authors summaries of results to check that there were no significant differences between our effect size estimates and authors' conclusions. However, we found that it was common practice in this literature to consider only statistical significance of the results rather than magnitude of the effect with a discussion of statistical uncertainty that doesn't focus on a binary cut-off. Therefore, in some cases we may report a moderate or large effect size with CIs crossing zero, despite the original authors reporting no effect on substance use.

The majority of papers reported on outcomes based on a continuous scale and so the main effect size metric that was used for the purposes of the meta-analyses was the SMD, with its 95% CI. Within this, Hedges' g was used to correct for any small sample bias. However, a number of studies reported on a binary outcome for which we were able to calculate a log odds ratio. We considered on a case-by-case basis whether it was sensible to convert the odds ratio to a SMD for the purpose of combining in one meta-analysis, considering whether the binary outcome had been dichotomised from a continuous scale, for example, blood alcohol level. Where we did convert an odds ratio to a SMD, we relied on the assumptions described in the Cochrane handbook (Higgins, 2022) and used the following formula: (Figure 3).

Where we did not convert the Odds Ratio to a SMD, we present these results narratively in the results section.

As described in the Cochrane handbook, the standardisation of continuous outcomes into SMD does not correct for differences in the direction of the scale between studies. The majority of the included outcome scales used in the included studies in our review indicated a higher score or number for increased substance use, for example, days of drinking in the past month or score on the Addiction Severity Index. However, for others a decrease in substance use was indicated by a higher number, for example, number of days abstinent. We therefore multiplied the SMD and associated CI by –1 in these latter cases to ensure that all the scales point in the same direction.

Unit of analysis issues

We assessed for unit of analysis errors in the included studies in the review. However, all but one of the studies in review randomised and analysed data at the individual level. Smelson (2018) was the only cluster randomised study, that randomised case workers and the group of people they worked with to a programme group or to an implementation as usual group. Their analytical approach accounted for clustering of observations within individuals and case managers and therefore we did not need to make any adjustments to the estimated standard errors in the included studies.

Criteria for determination of independent findings

Dependent effects can occur when a study reports results for multiple measures of the same outcome construct for the same sample, the same outcome measure at multiple time points, when a study has multiple treatment arms compared to a common control group, or multiple studies evaluate the same programme and report on the same outcome. This is problematic as estimating an average effect using standard meta-analytic models rely on the statistical assumption of independence of each included effect size (Gleser, 2007). Extracting data for all results can also significantly increase the amount of work.

Once we identified our pool of included studies, we mapped the programmes being evaluated, the outcome measures used in each study and the follow-up time(s) to identify possible dependent effects. We implemented the following strategies to address dependent effects, drawing on (Piggot, 2020), with an aim to balance capturing as much relevant information from each study as possible with the limited timeframe for the review:

Multiple follow-up points: If a study reported results for the same outcome at multiple follow-up points, we extracted data for a maximum of two time points that also meet the inclusion criteria of following up at 36 months or less and calculated standardised effect sizes for both. Reporting of multiple follow-up points was very common in our pool of included studies (described in more detail in the results section). We extracted a short-term follow-up (the closest to the 6-month follow-up point) and a longer-term follow-up point. We extracted the closest result to a 6-month follow-up as this is a key point of interest for policymakers for these sorts of interventions. Given that multiple included studies reported more than two post-6-month follow-up points, we mapped the longer-term follow-ups across all studies in the review and then extracted the longer-term follow-up point that was closest to the most common across each study (24 months).

Dealing with missing data

There were six papers that did not report sufficient statistical information to calculate standardised effect sizes for at least some of the outcomes of interest for this review. In all of these cases, we contacted the authors to obtain additional information. However, for all but one of the six (Essock, 2006) we did not receive the necessary data and therefore some of the effects from these studies are excluded from statistical analyses. We were able to calculate some effect sizes for some of the relevant outcomes for Braucht (1995) and Upshur (2015). We present the characteristics of these studies and results of the critical appraisal in the overview tables below. The six papers are:

Burnam (1995) – unable to calculate effect sizes. Study included narratively in section of report on residential rehabilitation.

Braucht (1995) – study states that it collected substance use outcome data, but only service use data reported. Study included narratively in section of report on ACT/ICM.

Clifasefi (2020) – unable to calculate effect sizes. Study included narratively in section of report on group work.

Essock (2006) – authors provided data and study included in meta analysis for all interventions, and for ACT/ICM.

Stahler (1995) – unable to calculate effect sizes. Study included narratively in the section of the report on residential rehabilitation.

Upshur (2015) – some missing outcome data, but able to calculate effect sizes for other outcomes. Study included in meta analysis for ACT/ICM.

Assessment of heterogeneity

We report standardised effect sizes and CIs for each individual study (where possible) and for each meta-analysis we were able to undertake, we manually inspected the forest plots for heterogeneity. Given that our datasets included multiple effect sizes clustered into studies, it was likely that effects belonging to the same study would be more similar to each other than effects for different studies. We therefore calculated I ² using the two variance components from the meta-analytic outputs, for the between-cluster heterogeneity and the within-cluster heterogeneity. The I ² values indicate how much of the total variance can be attributed to the total amount of heterogeneity, or the sum of between- and within-cluster heterogeneity. We also present the proportion of heterogeneity that we estimate can be attributed to between and within cluster heterogeneity separately. Finally, we also present the absolute estimates for the two variance components, for the between-cluster heterogeneity () and the within-cluster heterogeneity ().

Assessment of reporting biases

Our search strategy included searches for grey literature, which should help to mitigate any publication bias which might be observed if we were to only include published studies (as published studies are likely to report larger than average effects; Borenstein, 2010). However, we also undertook additional analysis to assess whether publication bias was likely to be a factor in our findings. This included a funnel plot to determine whether the summary effects in the meta-analysis are subject to publication bias, and if this appears to be the case, further tests (e.g., Duval and Tweedie' Trim and Fill) to determine a ‘best estimate of the unbiased effect size’ (Borenstein, 2009, p. 286).

Data synthesis

Our final set of studies and effect sizes were characterised by a range of types of dependencies, meaning a complex data set containing correlated effect sizes and sampling errors. Many studies reported effects for multiple measures of substance use and reported effects across multiple follow-up times, or both. Several studies included multiple treatment arms compared to a common control group. Several studies, most notably the Chez Soi study, which was conducted across five cities in Canada, were reported on in numerous papers that referred to subsets of the population or individual city level results in the case of Chez Soi. Because of this variation of types of outcomes within studies, there was reason to expect that there would be within-study heterogeneity in effect sizes.

For this reason, we used the Correlated and Hierarchical Effects (CHE) model (Pustejovsky, 2022) in combination with Robust Variance Estimation, using the metafor (Viechtbauer, 2010) and clubSandwich (Pustejovsky, 2022) packages in R. This model assumes that there is a constant correlation among groups of effect sizes from the same study, and also allows for between-study and within-study heterogeneity in true effect sizes. We did not have access to information on the likely correlation among effect sizes in our papers, and so we used an assumption of rho = 0.6 in all meta-analyses, and tested the sensitivity of the results to higher and lower values of rho.

We present the results of the meta-analysis on forest plots. In the cases where we include a large number of effect sizes within the same analysis that are dependent, we present pooled estimates that are aggregated to the study level for the forest plots, as the full set of individual effect sizes were not legible.

Subgroup analysis and investigation of heterogeneity

We did not have a sufficient number of studies and effect sizes to include all our moderators of interest in the same analysis, which we pre-specified as our preferred approach in the protocol. We therefore estimated single variable meta-regression models for some of our moderator of interest and interpret each of these with caution. Before doing this, we explored the relationship between our moderating variables of interest in our pool of included studies to understand if any were correlated.

We used a no-intercept specification so that coefficients represent average effect sizes for the corresponding category, as well as testing the difference between moderating variable values using an intercept model.

Sensitivity analysis

Where possible, we undertook sensitivity analysis to determine the effect on our overall findings of:

Non-randomised studies

We undertook this sensitivity analyses by repeating the meta-analysis, omitting in turn each of the groups of studies described above, to determine their effect on overall findings.

Treatment of qualitative research

We did not include qualitative research in the review.

[1] https://www.betterevaluation.org/en/managers_guide/step_2/describe_theory_of_change

[1] Level descriptions taken from https://whatworksgrowth.org/resources/the-scientific-maryland-scale/

Results of the search

We identified 328 papers out of a total 562 papers that were relevant from the EGM. In addition, we identified a total of 130 papers from supplementary searches of relevant journals and unpacking of systematic reviews and from our call for evidence. We excluded 259 papers at this title and abstract screening stage, meaning that we progressed 199 papers for full-text screening. We attempted to identify full texts for all of these, however we were unable to identify the full-text PDFs of six papers.

We excluded 145 of the 193 papers screened at full-text. We recorded the first reason that we identified for exclusion. The most common reasons for exclusion were that the papers did not evaluate an intervention in our typology (n = 49), they did not target our population group of interest (n = 12), they did not use an RCT or quasi-experimental design (n = 33) or they did not measure a substance use outcome (n = 45). In addition, we identified five papers that reported on ongoing studies and therefore were not eligible for inclusion, one paper that was published before 1990.

OPEN IN VIEWER

Figure 1

 

OPEN IN VIEWER

Figure 2

 

OPEN IN VIEWER

Figure 3

 

This left us with 48 included papers in the review. These papers correspond to 34 unique studies. The overall process and outcome of the search and screening process is set out in Figure 4.

OPEN IN VIEWER

Figure 4

The PRISMA diagram.

Included studies

Forty-eight papers, covering 34 separate studies, were included in this review. Table 2 provides a descriptive account of the 48 included papers.

Linked papers

Several studies were covered by more than one paper. Most notable of these is the Canadian At Home/Chez Soi Housing First with ACT/ICM, which is covered by nine separate papers. We have included one of these papers (Aubry, 2016) in our analysis, and the other papers are presented narratively in the findings section on research question 5 with respect to ACT/ICM.

Four papers examine a contingency management and day therapy intervention in Birmingham, Alabama. These four papers – Milby (2000, 2003); Schumacher (2000, 2003) – differ in terms of the outcomes and follow up periods covered. The four are included in analysis set out in the findings section on research question 5 with respect to contingency management interventions. Three papers are included in the meta analysis, and one Schumacher (2000) is presented narratively.

Two papers cover a study on the HUD-VASH funded by the Department of Veteran Affairs in the United States. These two are Rosenheck (2003) and O'Connell (2012). Rosenheck (2003) is included in the meta analysis for the overall effectiveness of substance use interventions (research question 1) and also for research question 5 with respect to ACT/ICM, in which O'Connell (2012) is presented narratively.

Finally, three papers covered a study of variations of ACT in St. Louis, Missouri. These three papers – Morse (2006, 2008); Fletcher (2008) – differ in terms of follow up period, and Morse et al. (2008) includes a different treatment group. The two Morse papers are included in the meta analysis for research question 1; Morse (2006) is included in the meta analysis for research question 5 with respect to ACT/ICM, in which Morse (2008) and Fletcher (2008) are presented narratively.

Characteristics of the evidence base

We included 48 papers in our systematic review, that report on 34 unique studies. In this section, we describe the characteristics of these papers and studies.

Excluded studies

Of the 193 papers subjected to full review, 145 were excluded from the review. Most (n = 49) were excluded from the study because they did not evaluate an intervention of interest. In many cases, the intervention was not adequately described in the paper abstract, and it was not possible to determine whether the paper covered an intervention of interest without reviewing the article in full.

Forty-five papers were excluded because they did not measure relevant outcomes, or did not measure outcomes specifically for adults experiencing homelessness. Again, the review team found that outcomes measured were often not included in the title and abstracts reviewed. Thirty-three papers were excluded at full review because of research design issues, including lack of comparator for the intervention of interest. Nine papers were excluded because study participants were not aged 18 or over. A number of papers in the effectiveness EGM use terms such as ‘youth’, ‘young adults’, or ‘adolescents’ without being explicit in the abstract about the age range of participants. As there is no single agreed age range associated with these terms, full review was necessary to determine whether the paper met our age inclusion criteria. A summary of the reasons for exclusion is provided in Figure 4.

Research question 1: How effective are interventions designed to reduce substance use in adults who are experiencing homelessness compared to TAU?

To answer our first research question, we looked at the subset of studies that compared any intervention in our typology to a TAU control group. There were a possible 34 included papers, corresponding to 24 studies, that were relevant for this analysis. From these, we were able to combine 45 effect sizes from 15 studies reported in 16 papers, using a CHE meta-analysis with cluster robust estimation. There were a number of papers and studies that we did not include in the meta-analysis, for the reasons below:

We were unable to calculate an SMD for Burnam (1995), Stahler (1995) and Clifasefi (2020) and these three studies therefore are not included in this analysis (the full list of studies is presented in the Missing data section in the methods section. We did not have a measure of variance for Smelson (2018) so could not include in the analysis).

Finally, we did not think it was appropriate to convert odds ratio to SMD for Nyamathi (2016) or Thompson (2020) and so these two studies are not included in the meta-analysis. These results are presented narratively in the sections on ICM and motivational interviewing respectively.

Figure 8 below shows the meta-analysis results to answer research question 1. Please note that we have averaged the individual effect size from each study for the purposes of displaying on the forest plot as there were too many individual effects to display clearly. Estimates refers to the number of effect sizes included for each study in the meta-analysis. The mean number of effect sizes per study was 3 with the number of effects ranging from 1 to 6. Multiple effect sizes from single studies resulted from multiple follow-up periods, multiple treatment arms against a common control and multiple measures of the same construct (substance use). All of the included studies were from the USA, with the exception of the two studies from Canada (Aubry, 2016 – the at Home/Chez Soi programme; Cherner (2017), a combined housing first and ICM programme in Ottawa) and study from France (Tinland, 2020). They tested a range of types of interventions from our typology, covering ACT, CM, Residential Rehabilitation, Motivational Interviewing techniques combined with a range of other interventions and ICM. The sample size column indicates the total number of people involved in the study across intervention and control group at baseline.

OPEN IN VIEWER

Figure 8

Average effect of homelessness interventions on substance use compared to TAU service provision.

We calculated an average effect of –0.11 SD (95% confidence intervals [CI], −0.27, 0.05), indicating on average a reduction in substance use for people experiencing homelessness that participate in the included programmes compared to people experiencing TAU service provision. However; there is substantial heterogeneity across studies, which can be observed by looking at the forest plot. We calculated prediction intervals, that is, the expected range of true effects in similar studies in future settings (IntHout et al., 2016). We estimated these to be –0.11 SD (−0.69, 0.46), which contain a much larger range of treatment effects than the CIs, with a range of values on both sides of the line of no effect (indicating possible reduced or increased substance use as a result of being involved in these types of programmes). This heterogeneity is to be expected given the range of interventions being tested, the complex diversity of needs of the people involved across different studies and the mix of follow-up periods (explored further below). Unfortunately, due to a limited number of included effect sizes and studies, we were unable to include these possible explanatory variables as moderators in the same analysis.

The I ² value in this context indicates how much of the total variance can be attributed to the total amount of heterogeneity, which is the sum of between- and within-cluster heterogeneity. We calculated an I ² value of 71%, with 61% due to between-study heterogeneity and 10% from within-study heterogeneity (and the remaining 29% from sampling variation).

Sensitivity analysis

We tested the sensitivity of the results of this meta-analysis to:

(1)

Different values of rho (the correlation among groups of effect sizes)

We found that the point estimate and CIs were not sensitive to a range of values of rho (from a very low value of rho of 0.05 up to a very high correlation of 0.80).

However; the results were sensitive to the removal of low confidence studies (−0.21 SD, 95% [−0.59, 0.17] − 6 studies, 17 effect sizes), the removal of quasi-experimental studies (−0.14 SD, 95% CI [−0.30, 0.02] − 14 studies, 41 effect sizes) and the removal of studies where an effect size had been converted from a binary to a continuous outcome (−0.08 SD, 95% CI [−0.31, 0.15] − 10 studies, 31 effect sizes). This suggests that the findings are sensitive to the inclusion of lower quality studies, although unusually the average effect increases when we remove the lowest confidence studies, as well as the choices we made on how to calculate effect sizes.

As explored in the next section however, we also found that the papers that were rated as low confidence were also more likely to take a harm reduction intervention approach. We therefore cannot be sure whether the intervention approach, the strength of the study approach, or indeed another factor, is contributing to variation in outcomes.

Research questions 2 and 3: What is the effect of abstinence-based interventions on substance use outcomes in adults who are experiencing homelessness? What is the effect of harm-based interventions on substance use outcomes in adults who are experiencing homelessness?

Research questions 2 and 3: What is the effect of abstinence-based interventions on substance use outcomes in adults who are experiencing homelessness? What is the effect of harm-based interventions on substance use outcomes in adults who are experiencing homelessness?

To answer our second and third research questions, we used the same set of studies that we used to answer RQ1, that is, those studies that compared an intervention in our typology to a TAU control group. All but two of these studies compared an intervention to a TAU condition that we determined to be primarily an abstinence-based offer of typical services for people experiencing homelessness who were also engaging in problematic substance use. In two cases, we were unable to categorise the control condition as harm reduction or abstinence based because of insufficient information (Essock, 2006; Tucker, 2017).

We were able to combine 45 effect sizes from 15 studies reported in 16 papers, using a CHE meta-regression with cluster robust estimation. We included the categorisation of the intervention as either harm reduction based or abstinence-based in the meta-analysis as a moderating variable. We estimate a no intercept model, so that both harm and abstinence are included in the model as predictor variables. The studies excluded from this analysis are reported in the previous section addressing research question 1.

We caution that these estimates are based on a small number of effect sizes and studies. The studies are also diverse within both the abstinence and harm reduction categories, in terms of intervention and populations. As stated above, most of the harm reduction studies were rated as being of low confidence and the results from these studies are therefore to be treated with more caution. In addition, the most common intervention tested in the harm reduction category was ACT and ICM (six of the nine included studies). Therefore, we interpret the meta-analysis results that answer research question 3 on the effect of harm reduction interventions compared to treatment as usual with some caution, examined in more detail in the discussion.

The average effect for the abstinence-based interventions included in this study compared to TAU service provision is –0.28 SD (95% −0.65, 0.09) (6 studies, 15 effect sizes), while the average effect for the harm reduction interventions included in this study compared to a TAU service provision is close to 0 at 0.03 SD (95% CI, −0.08, 0.14) (9 studies, 30 effect sizes). The CIs for both estimates are wide and crossing zero. For both of these analyses, the TAU comparison groups in the included studies are for the most part primarily abstinence-based, with the exception of two studies where the comparison group condition was unclear (Figures 9 and 10).

OPEN IN VIEWER

Figure 9

Average effect of abstinence-based interventions on substance use compared to TAU service provision.

OPEN IN VIEWER

Figure 10

Average effect of harm reduction-based interventions on substance use compared to TAU service provision.

Sensitivity analysis

We intended to do the following types of sensitivity analysis:

(1)

Different values of rho (the correlation among groups of effect sizes)

As we found with the previous analysis, we found that the point estimates and CIs were not sensitive to a range of values of rho (from a very low value of rho of 0.05 up to a very high correlation of 0.80). We found little difference in both point estimates when we removed studies that had been converted from a binary outcome to a continuous outcome (10 studies, 31 effect sizes in the analysis), but found wider CIs (abstinence-based interventions vs. TAU: −0.30 SD [95% CI: −0.96, 0.36]; harm-reduction based interventions vs. TAU: 0.05 SD [95% CI: −0.13, 0.23]).

Because of the large number of harm reduction studies that were rated as low confidence we are unable to undertake sensitivity analysis by confidence rating for this set of papers. The overview of included studies in this analysis by confidence rating is shown in Table 4 below. The point estimate for abstinence-based interventions was not sensitive to the removal of low confidence studies, however the CIs became wider (−0.28 SD [95% CI, −0.96, 0.40]).

The only two quasi-experimental papers in the analysis were harm reduction-based (Cherner, 2017; Morse, 2008); that is, all abstinence-based studies were RCTs. The harm reduction point estimate changed by a very small amount following the removal of the quasi-experimental estimates from the meta-analysis (−0.02, 95% CI [−0.14, 0.10]).

Research question 4: Are abstinence-based interventions more or less effective than harm reduction-based interventions?

We had intended to answer this research question using studies that made a direct comparison between the introduction of a new abstinence-based intervention and a new harm reduction intervention. However, only four studies made a direct comparison between a new harm reduction intervention and an abstinence-based intervention; McHugo (2004), Koffarnus (2011), Tsemberis (2004) and Young (2009). Three of these studies evaluate the impact of ACT compared to another intervention that was abstinence-based, while Koffarnus (2011) evaluate several approaches to contingency management. Because of the limited number of studies, most of which are restricted to the same type of intervention, we were unable to draw conclusions about whether in general abstinence-based interventions are more or less effective than harm reduction approaches. These studies are instead discussed in the following section on the effectiveness of ACT and contingency management.

There are 18 papers that compared a harm-reduction based intervention to an abstinence-based TAU, as reported in the previous section. However, we do not consider these studies to reasonably answer the question of whether abstinence-based interventions are more or less effective than harm-reduction based interventions. Although we determined that the TAU offer was primarily abstinence-based, it is not clear from many of these papers whether the comparison group participants chose to take-up any abstinence-based services. In addition, the abstinence-based treatment as usual condition often appeared to be a mix of different services, that were not consistently reported, and that may have also included some harm reduction-based services that were not reported. While take-up of the interventions in the four studies above comparing new harm-reduction and new abstinence-based treatments may have been mixed, in general take-up of new interventions in these studies was higher. We therefore do not use the results of this set of studies to answer the question of whether abstinence-based approaches are more or less effective than harm reduction-based approaches.

Research question 5: What is the effect of individual interventions designed to reduce substance use in adults who are experiencing homelessness, compared to treatment as usual and each other?

So far, we have examined the effectiveness of all substance use interventions compared with treatment as usual, the effectiveness of abstinence based and harm reduction-based interventions. These analyses provide useful insights, particularly that interventions tend to be more effective than treatment as usual.

In this section, we set out our analysis of the effectiveness of individual interventions. Table 5 gives an overview of the findings for each of the interventions listed in the typology of interventions set out in Table 1. There were nine individual interventions for which we were able to undertake analysis. For three interventions, we were able to undertake meta analysis: for ACT and intensive case management (presented together), and for contingency management. For the other six interventions, we provide narrative synthesis. For all of the other interventions listed in our typology in Table 1, we did not identify any relevant effectiveness studies and were unable to undertake meta-analysis or narrative synthesis.

It is worth noting again two issues that are relevant when interpreting Table 5. These are: firstly, the focus of this review is the effectiveness of substance use interventions for adults experiencing homelessness. Many of the interventions listed in the typology are used more generally (i.e., with populations other than adults experiencing homelessness), and therefore there might be effectiveness evidence not included here because is not specific to adults experiencing homelessness; secondly, a lack of effectiveness evidence does not indicate that the intervention is ineffective. It simply means that there is no evidence that met the inclusion criteria for this review.

The effectiveness evidence for nine of the interventions listed in our typology is as follows:

ACT and ICM

Assertative Community Treatment was originally developed for patients with severe mental illness, providing personalised, high intensity, holistic and integrated multidisciplinary community care services. In its initial design, this intervention did not include substance use treatment; however, over the past two decades counselling and other approaches to reducing substance use have been integrated into ACT. ACT can include components that are abstinence or harm reduction based, although the latter is more usual. Because of this, we have assumed that intervention studies that cover ACT or ICM are harm reduction based, unless otherwise stated in the primary research. We also included evaluations of ICM, which are similar to ACT but tend to work with people with less severe needs. Given their similar approach, we analyse them together but also undertook moderator analysis to explore whether there is a difference between the two.

We looked at the subset of studies that assessed the impact of ACT or ICM interventions compared to a control group without these interventions offering either TAU or another new intervention. We included 27 papers reporting on 17 studies that assessed the impact of ACT or ICM. We were able to combine 36 effect sizes reported in 15 papers from 14 studies, using a CHE meta-analysis with cluster robust estimation. We do not includeBraucht 1995 in the analysis as we were only able to calculate a standardised effect size for the outcome that looked at service use, rather than substance use. We do not include Nyamathi (2016) as we were unable to convert the odds ratios to SMDs. However, we present the findings narratively for these two studies below. Fletcher (2008) and O'Connell (2012) are follow-up papers of studies included in the meta-analysis (Morse, 2006; Rosenheck, 2003 respectively) and are also presented narratively below.

We included 9 papers that report on the At Home/Chez Soi trial in Canada, an RCT of Housing First combined with ACT or ICM, depending on the needs of participants. It took place across Montreal, Moncton, Toronto, Vancouver and Winnipeg, with delivery in each city including a focus on a different sub-population of people experiencing homelessness with severe or moderate mental health needs. ‘Housing First’ involves provision of immediate access to subsidised housing without conditions for those experiencing homelessness, alongside accompanying support, in this case either ACT or ICM. We included the results of only one paper reporting on the At Home/Chez soi trial in Canada in the meta-analysis, Aubry (2016), as this paper presents combined results across all five included cities. However, we also present the results of the other included papers narratively below.

Most of the studies included in the meta-analysis took place in the USA, with the exception of Tinland (2020), which took place in France, and Aubry (2016) and Cherner (2017) which took place in Canada. Follow-up ranged from 6 months up to 36 months.

We calculated an average effect of 0.04 SD (95% CI, [−0.07, 0.14]). This indicates that the average effect of ACT or ICM programmes on substance use is close to 0. However, there is some heterogeneity both within and across studies, which can be observed by looking at the forest plots. We calculated prediction intervals of 0.04 SD (−0.23, 0.31), which contain a larger range of treatment effects than the CIs, with a range of values on both sides of the line of no effect (indicating possible reduced or increased substance use as a result of being involved in ACT or ICM programmes). Previous effectiveness research has identified issues with fidelity with some programmes described as ACT, and that greater fidelity is associated with better outcomes (see e.g., a systematic review of the effectiveness of ACT for substance use disorders by Penzenstadler, 2019). It may be that differences in fidelity between studies explain some of the heterogeneity found here. The review team did not include intervention fidelity as an area of interest in this research.

We calculated an I ² value of 34%, with 23% due to between-study heterogeneity and 11% from within-study heterogeneity (and the remaining 66% from sampling variation) (Figure 11).

OPEN IN VIEWER

Figure 11

Average effect of ACT and ICM programmes on substance use.

When we undertake a moderator analysis to explore the individual effects of ACT or ICM, we see a very similar effect size to the combined average effect for the two types of programmes (ACT = 0.03 SD, 95% CI [−0.07, 0.13], 9 studies, 24 effect sizes; ICM = 0.05 SD, 95% CI [−0.28, 0.39], 5 studies, 12 effect estimates).

Sensitivity analysis – ACT and ICM

We undertook the following types of sensitivity analysis for the meta-analysis of ICM and ACT studies:

(1)

Different values of rho (the correlation among groups of effect sizes)

As we found with the previous analysis, we found that the point estimates and CIs were not sensitive to a range of values of rho (from a very low value of rho of 0.05 up to a very high correlation of 0.80). We found that the results were not sensitive to the removal of studies that had been converted from a binary outcome to a continuous outcome (13 studies, 34 effect sizes included for the analysis).

When we removed low confidence papers from the analysis (Aubry, 2016; Cherner, 2017; Morse, 2006, 2008; Rosenheck, 2003; Young, 2009), again the point estimate remains similar although the CIs become wider (0.00 SD, 95% CI [−0.17, 0.16], 9 studies, 21 effect sizes). Almost all the included studies were RCTs, with the exception of Cherner (2017); Morse (2008); Young (2009). The results were not sensitive to the removal of these three studies (0.03 SD, 95% CI [−0.07, 0.13], 12 studies, 30 effect sizes).

Narrative synthesis – ACT and ICM

We included Braucht 1995 in the review, for which we were only able to calculate a standardised effect size for the outcome measure, number of services that the clients in the ICM group received. They tested an ICM approach which was offered to residents in a rehabilitation facility called Arapahoe House in the Denver area in the USA. Both intervention and control group were offered access to the full array of services offered by Arapahoe House, including detoxification, residential and outpatient services, and substance abuse counselling. The average age of participants was 35.4 years and 85% of the people involved in the study were male. The authors looked at the impact on service use as an outcome at 6 months after the start of the intervention, as well as stating they looked a range of substance use outcomes. The sample size at the beginning of the study was 323 people, with 163 people in the intervention group and 160 in the control group.

While the authors expected that case managers involved in the ICM programme would increase the number of services that the clients in the ICM group received, they did not find this to be the case, with a Hedge's g of 0.03 SD (95% CI, –0.20, 0.27). However, we rated this study as low confidence, rated down primarily on attrition.

As described earlier, we include nine papers in the systematic review that report on the At Home – Chez Soi RCT in Canada. We present the results of the papers not included in the meta-analysis below. The pattern of results across these studies is a lack of a significant impact on substance use at any of the follow-up points for which we extracted data. There are some effect sizes that indicate a small positive or negative impact on substance use for the intervention group compared to control but almost all have wide CIs that span the line of no effect. Substance use is measured in all papers using the Global Appraisal of Individual Needs (GAIN) Substance Problem Scale (SPS), with the exception of Somers (2015). This lack of impact is consistent with the combined results presented in Aubry (2016), which is included in the meta-analysis above. None of these studies were rated as being of high confidence.

Aubry (2019) presents results for Moncton only for ACT participants only at a 21–24-month follow-up period. They found a Hedge's g effect size of 0.01 SD (95% CI, –0.30, 0.31) on the GAIN-SPS.

Fletcher (2008) is a longitudinal follow-up of the study originally reported in Morse (2006). The authors report on an RCT testing two versions of ACT compared to a TAU comparison group for people experiencing homelessness with both severe mental health and substance use disorders. The authors test a typical version of ACT compared to a version called integrated ACT which offered a team with a substance abuse specialist to provide counselling, treatment groups and outreach and a team trained in integrated treatment principles. At the start of the study, 191 participants were randomised between the three groups. They follow-up with the study participants at 30 months after the start of the study on a number of outcomes, including two five-point scales assessing alcohol and drug use abuse severity. The study was rated as low confidence given the high rates of attrition by the end of the study.

The integrated ACT arm had an effect size of 0.23 SD (95% CI, −0.16, 0.63) on the substance abuse rating scale while the ACT had an effect size of 0.12 SD (95% CI, −0.28, 0.52), where a positive number indicates worse substance use severity. However, both effect sizes have large CIs and were not statistically significant and therefore the authors report that the programmes had no group effects on substance use.

Nyamathi (2016) undertook an RCT to evaluate the impact of two interventions compared to usual care, an intensive peer coaching programme and the intensive peer coaching programme combined with intensive nurse case management. Although not described as such by the authors, the intervention appears to be ICM and has been treated as such for this review. The nurse was trained in culturally-competent case management and spent about 20 min each week with participants on topics including completion of drug treatment, as well as health promotion such as vaccination compliance, and reduction of risky drug and sexual behaviours. The usual care condition involved some limited peer coaching on health promotion and brief nurse counselling, primarily on vaccine uptake. The study took place in California in the USA and worked with 600 male parolees experiencing homelessness. For both treatment arms, the authors were primarily interested in justice related outcomes such as rearrest and reincarceration but also evaluated the impact of the programmes on use of marijuana, stimulants or heroin over the 12-month observation period.

The authors report that no significant differences were detected on any of the reported drug use outcomes among the two programme groups compared to usual care. For the intensive case management plus peer coaching group, we find odds ratios of 1.16 (95% CI, 0.76, 1.76) for effect on marijuana use, 0.83 (95% CI, 0.54, 1.26) on stimulants use and 0.52 (0.25, 1.09) on heroin use, where an odds ratio of less than 1 indicates a reduction in substance use. For the intensive peer coaching group, we find odds ratios of 1.10 (95% CI, 0.72, 1.66) for the impact on marijuana use, 0.79 (95% CI, 0.52, 1.19) on stimulants use and 0.96 (95% CI, 0.51, 1.78) on heroin use.

O'Connell (2012) is a follow-up paper to the study originally reported in Rosenheck (2003). The RCT evaluated the U.S. Department of Housing and Urban Development–Veterans Affairs Supported Housing (HUD-VASH) programme, which offers ICM plus housing subsidy vouchers for veterans with substance abuse issues experiencing homelessness in the USA. Two different versions of the treatments were compared against the TAU offer, ICM plus rent subsidy vouchers and ICM only. At the start of the study, 460 individuals were randomised into the three different groups. This follow-up study explored how the impacts of these two interventions on the Addiction Severity Index (ASI) for alcohol and drugs varied by sub-group of veterans, including African American compared to White veterans. The study was rated as low confidence due to high rates of attrition. Comparing ICM and TAU, the authors found an standardised effect size of 0.58 (Hedge's g) (95% CI, 0.23, 0.93) in relation to alcohol and 0.12 (Hedge's g) (95% CI, −0.23, 0.46) in relation to drugs.

The authors found a significant interaction between race and treatment condition for drug use, specifically that ICM plus rent subsidy vouchers had a larger impact on reducing drug use among African American veterans compared to White veterans. They did not find an interaction between race and treatment condition for severity of alcohol use.

CM

CM is an approach to treatment that maintains that the form or frequency of behaviour can be altered through a planned and organised system of positive and negative consequences. This could include, for example, regular testing and requirements for treatment engagement. CM is an abstinence-based intervention.

We looked at the subset of studies in our systematic review that assessed the impact of a CM intervention compared to a control group, which is 9 papers reporting on 6 studies. We were able to combine 20 effect sizes from 6 studies reported in 8 papers, using a CHE meta-analysis with cluster robust estimation. All of these studies took place in the USA. None of the included studies followed-up over a period of longer than 12 months since the intervention started, with follow-up ranging between 2 months and 12 months. We do not include Schumacher (2000) in the analysis as they only looked at service use, rather than substance use. However, we present the findings below narratively.

Although we categorised 6 studies as CM, they still evaluated a diverse range of CM interventions that worked with different populations. Koffarnus (2011) randomly assigned adults experiencing homelessness and alcohol-dependency to 3 groups: either a group requiring abstinence from alcohol to engage in paid job skills training, a group that offered paid job skills training with no abstinence contingencies or a group offering unpaid job skill training with no abstinence contingencies[1]. Kashner (2002) is an evaluation of a US federal compensated work therapy programme that provides financial rewards to participants, homeless, substance-dependent veterans, who demonstrate continued abstinence. Reback (2010) evaluate a community-based HIV prevention programme working with homeless men that targeted reduced substance use and increased health-promoting behaviours. Participants in the CM condition earned points for drug/alcohol abstinence as well as for undertaking health-promoting behaviours. Milby (1996) combined group-orientated, abstinence-based day treatment with CM. After 2 months of the day treatment, participants went on to participate in 4-months of work therapy which was contingent on drug-free urine toxicologies. Milby (2000) (also reported in Milby, 2003; Schumacher, 2000, 2003) also test the impact of abstinence-based day treatment combined with abstinence contingent housing and work therapy. Finally, Smith (1998) combine the provision of the drug Disulfiram and a community-reinforcement approach with abstinent-contingent housing.

We calculated an average effect of –0.47 SD (95% CI, [−0.72, −0.21]), indicating on average a large reduction in substance use for people experiencing homelessness that participate in the included CM programmes compared to a control group not receiving CM. There is some heterogeneity across studies, which can be observed by looking at the forest plot. We calculated prediction intervals, that is, the expected range of true effects in similar studies in future settings (IntHout et al., 2016). We estimated these to be –0.47 SD (−1.09, 0.16), which contain a much larger range of treatment effects than the CIs, although most of the interval falls on the side of the line of no effect that indicates a reduction in substance use as a result of being involved in these types of programmes. We calculated an I ² value of 50%, with 12% due to between-study heterogeneity and 38% from within-study heterogeneity (and the remaining 50% from sampling variation) (Figure 12).

OPEN IN VIEWER

Figure 12

Average effect of contingency management programmes on substance use by adults experiencing homelessness.

Sensitivity analysis – CM

We intended to do the following types of sensitivity analysis:

(1)

Different values of rho (the correlation among groups of effect sizes)

As we found with the previous analysis, we found that the point estimates and CIs were not sensitive to a range of values of rho (from a very low value of rho of 0.05 up to a very high correlation of 0.80). All of the included studies in the contingency management meta-analysis were RCTs, and therefore we did not need to undertake sensitivity analysis along this dimension.

The results were sensitive however to inclusion of low confidence studies and inclusion of effect sizes converted from a binary to a continuous outcome:

After removing the studies rated as low confidence (Koffarnus, 2011; Milby, 1996; Schumacher, 2003), the average effect was −0.54 SD (95% CI −0.94, −0.14) – 4 clusters, 9 effect sizes.

Narrative synthesis – CM

Schumacher (2000) reports on the impact of an abstinence-based day treatment combined with abstinence contingent housing and work therapy on the number of scheduled sessions of treatment attended (also reported in Milby, 1996, 2000; Schumacher, 2003). People in the treatment group had access to rent-free housing, if they provided four consecutive drug-free urine test results during the first 2 months of treatment. Alongside this, they had access to abstinence-based day treatment. During the next 4 months, they were offered housing for a modest price, again contingent on abstinence from drugs and were also eligible for supervised, paid work therapy. The control group offer was for the abstinent day treatment only. The participants were crack-cocaine dependent homeless people. The average age of participants was 37.7 years and 72% of the people involved in the study were male. Eighty-three percent of the sample were African American. The sample size at the beginning of the study was 141 people, with 72 people in the intervention group and 69 in the control group.

The authors found that after 2 months, the intervention group receiving abstinent contingent housing attended a large number of scheduled sessions that the control group, with a Hedge's g of 0.92 SD (95% CI, 0.57, 1.26). Although not a focus of this systematic review, the authors found that abstinence was a function of treatment attendance, that is, the more sessions attended, the higher the likelihood of abstinence.

Harm reduction psychotherapy

Harm reduction psychotherapy draws on the concept of harm reduction within a psychotherapeutic approach that integrates cognitive and behavioural interventions with a psychodynamic understanding of substance use as personally meaningful. This approach emerged in the 1990s, challenging the idea that problematic substance use is a disease (Milet, 2021), and with an assumption that individuals might use substances to alleviate psychological, social, and economic distress (Heather, 2018). This intervention assumes that individuals have autonomy, and is non-judgemental in approach. As its name implies, it is a harm reduction based intervention.

To date, the HaRT-A programme (Harm Reduction Treatment for Alcohol) is the only harm reduction psychotherapy intervention to be tested in a RCT (Milet, 2021). Developed in Seattle, Washington, using a co-production approach that involved people experiencing homelessness and problematic use of alcohol – HaRT-A involves four treatment sessions, using motivational interviewing and therapy, with participants developing their own harm reduction goals (Collins, 2019).

Two papers about this intervention are included in this systematic review, and are synthesised here. Both studies report a reduction in use compared to treatment as usual.

Collins (2019) is the first of the papers, about the effectiveness of this intervention. A total of n = 169 participants were recruited between October 2015 and February 2017, of whom n = 125 were included at analysis (n = 65 receiving the intervention, n = 60 not receiving the intervention). The average age of participants was 47.86 years (SD = 9.56), and participants were overwhelmingly male (n = 40, 24%, were women). Fifty eight percent of the sample identified as Black/African American, with White/European American representing 22% of the sample. This study examined a number of different outcomes of interest, including two around alcohol use. ‘Peak alcohol use’ was a self-reported measure of the number of drinks consumed on the heaviest drinking day in the previous 2 weeks. The second measure, and the one included in this systematic review, was self-reported number of days intoxicated in the previous 2 weeks. This is a measure of frequency from the 5th edition of the Addiction Severity Index (Mclellan et al., 1992), which was turned into a dichotomous rating of the number of days intoxicated.

The researchers reported a difference in means between the treatment and control group at post test and 3 months, with a small difference at 1 month follow up. The researchers report that the treatment effect for this measure was significant, but that time × treatment interaction effect was not significant. We calculated the odds ratio to be 1.77 (95% CI between 0.87 and 3.61), and the Cohen's d effect size as 0.31 (i.e., participants in the treatment group were more likely than those in the control group to see a reduction in their alcohol consumption).

The second paper on HaRT-A was published in 2021 (Collins, 2021). This examined a variation of HaRT-A, which involves combining it with a pharmacological treatment (naltrexone). Naltrexone is a medication that is used to manage alcohol or opioid disorder by reducing the cravings or feelings of euphoria associated with their use. The study compared this intervention with HaRT-A + a placebo, HaRT-A only, and treatment as usual. Between October 2013 and November 2017, n = 308 individuals were recruited and randomly assigned to one of the treatment groups, of whom n = 190 were included in the analysis. This included n = 48 in the HaRT-A + naltrexone group (n = 74 at baseline), n = 48 in the HaRT-A + placebo group (n = 78 at baseline), n = 54 in the HaRT-A only group (n = 79 at baseline), and n = 40 in the TAU control group (n = 77 at baseline). The treatment lasted 12 weeks, with follow ups until 36 weeks.

For the n = 309 individuals at the start of the trial, the average age ranged between 46.55 years (SD = 10.46) for the HaRT-A + placebo group, through to 49.27 (SD = 9.11) for the HaRT-A + naltrexone group. All four groups were mainly men (79% in the TAU group to 87% for the HaRT + placebo group). Black/African America and White/European America were the two largest ethnic groups in each of the treatment groups.

The study examined a number of outcomes. Primary outcomes used self-reported measures, including peak alcohol quantity, alcohol frequency, alcohol related harm, and measures of physical and mental ill health. Secondary outcomes included treatment adherence, and objective measure of alcohol frequency via blood test. It is this latter measure that we include in this systematic review, measured at two points: 3 months and 9 months. We use the author calculated odds ratio for both of these, as well as our own SMD, for the alcohol frequency via blood test outcome, where an odds ratio of more than 1 and a positive Hedge's g indicate a reduction in substance use.

For the main treatment group of interest, HaRT-A + naltrexone, at 3 months the odds ratio were 2.77 (95% CI, 1.01, 7.58) (Hedge's g = 0.56 SD) and at 9 months, odds ratio of 1.60 (95% CI, 0.57 to 4.49) (Hedge's g = 0.26 SD). In relation to HaRT-A + placebo, the effects were 0.91 at 3 months (95% CI, 0.28 to 2.95) (Hedge's g = −0.05 SD) and 1.09 (95% CI, 0.36 to 3.33) (Hedge's g = 0.05 SD) at 9 months respectively. For the final treatment group of interest, HaRT-A only, at 3 months the odds ratio was 4.02 (95% CI, 1.46 to 11.11) and the Hedge's effect size at 0.76 SD. At 9 months, the odds ratio was 1.16 (95% CI, 0.40 to 3.34) (Hedge's g = 0.08 SD). While these appear to show that those in the treatment groups were more likely than individuals in the control group to see a reduction in their alcohol consumption at 9 months compared to baseline, the difference was not statistically significant.

The researchers report that during the 12-week period of treatment, the three treatment groups did better than the control, but that after treatment ceased, the three HaRT-A groups saw outcomes plateau, whereas the treatment as usual group continued to see improvements.

Motivational interviewing

Motivational interviewing is designed to strengthen the personal motivation of the participant, to develop and commit to a specific goal by exploring the individual's own reasons for change. It is non-judgemental and collaborative in approach, and usually adopts a harm reduction approach. In the analysis presented in this review, we have assumed primary studies of the effectiveness of motivational interviewing interventions are harm reduction-based, unless otherwise stated in the primary research. Motivational interviewing is used as both a stand-alone intervention, but also as part of multi-component interventions, where it is combined with other approaches. (This is the case for HaRT-A, a multi-component intervention, i.e., primarily described as harm reduction psychotherapy, but includes motivational interviewing.) It can be done one-to-one or in group settings. We identified two studies that assessed the impact of a motivational interviewing intervention.

One of the included studies examines the effectiveness of motivational interviewing in addressing problematic substance use. The AWARE programme is a group motivational intervention aimed at young people aged 18 to 25 experiencing homelessness in Los Angeles County, California, with the aim of reducing alcohol consumption and risky sexual behaviours (Tucker, 2017). The intervention involves four, 45-minute-long group sessions, delivered through two drop-in centres in Hollywood and Venice in Los Angeles.

A sample of n = 200 participants were enroled in this RCT, equally split between the treatment and control group (treatment as usual, able to access all services at the drop-in centre). The sample was mostly men, with just over a quarter (27%) being women. The largest ethnic group was white (31%), followed by African American (24%) and Latino (21%). The mean age was 21.81 (SD = 1.87), with an average age of 16.01 (SD = 3.23 years) when they first left home. The authors state (p5) that participants' most recent period of homelessness had lasted 20.62 months (SD = 29.99 months) (NB as reported by the authors). The researchers report that there were no significant differences between the treatment and control group.

The evaluation used a number of outcome measures, including alcohol and drug use, and sexual behaviours, all of which were measured at baseline and 3 months. We extracted data for three frequency of consumption measures for alcohol, marijuana, and other drugs. Each of these measures was self-reported. In relation to frequency of alcohol consumption, the authors report standard deviation of means (post intervention) at 2.33 for the intervention group compared to 2.03 for the control group (pooled SD at 2.204). For marijuana consumption, 2.98 for the intervention group compared to 3.02 for the control group (pooled SD at 3.00); and for other drugs 0.59 compared to 0.94 (pooled SD at 0.785). The authors stated that there were positive changes in alcohol consumption (i.e., reduced alcohol consumption), but that there were no significant treatment effects in terms of the two drug consumption outcomes.

The second study by Thompson (2020) evaluated the impact of a brief motivational interviewing intervention combined with the provision of a smartphone application called OnTrack for participants to self-monitor their substance use and sexual risk behaviours. The intervention was targeted at 18- to 21-year-olds who were experiencing homelessness. The participants in the intervention group were compared to a TAU comparison group, who were offered usual substance use referral or services at the shelter where the study took place, as well as HIV testing. At the beginning of the study, 60 young people were randomised into the intervention and control group and followed up after 2 weeks, 4 weeks and 6 weeks for the number of times they had drank alcohol each day in the past 2 weeks and how many times per day in the past 2 weeks they smoked marijuana.

The authors found that young people in the OnTrack and motivational interviewing group had significantly lower odds than those receiving TAU services for drinking alcohol (OR of 0.14, 95% CI [0.03, 0.64]). They found an odds ratio of 0.39 (95% CI, 0.065, 2.33) for smoking marijuana.

Residential rehabilitation

Residential rehabilitation is a drugs and alcohol treatment programme delivered in a communal residential, non-hospital setting, where ‘live in’ treatment is provided. These programmes may include, but usually follow, detoxification. They differ in terms of length of stay, and are usually abstinence-based. Treatment programmes can draw on different approaches, but usually involve a structured programme including individual and group therapy. Programmes may employ therapeutic communities, 12 step, or cognitive behavioural therapy approaches.

Four of the included studies evaluated residential rehabilitation programmes, three of which were published in the 1990s and one was published in 2011. Two of the studies focused on veterans experiencing homelessness, of which one was about women. Overall, these studies suggest no real difference between residential rehabilitation and treatment as usual in terms of substance use outcomes.

Burnam (1995) evaluated two different interventions compared to treatment as usual for people experiencing homelessness, problematic substance abuse, and mental ill health. One of the treatment interventions was a residential rehabilitation programme, based on a social model recovery approach. This model focuses on peer to peer interactions and individual recovery plans, and is often seen as an alternative to clinically-orientated approaches. 12 step programmes such as Alcoholics Anonymous are also based on the social model recovery approach. The second treatment intervention used a similar programme, but without a residential component. The evaluation compared these two treatments with treatment as usual, which the authors describe as access to other available community services, such as shelters, clinics, and 12 step programmes. Both treatment interventions involved 3 months of intense treatment (residential or non-residential), followed by a 3 month non-residential programme of activities.

Two hundred and seventy six (n = 276) met the study's eligibility criteria and were randomly assigned to the residential rehabilitation programme (n = 67), non-residential programme (n = 144), or the treatment as usual group (n = 65). The average age of the sample was 37 years (no standard deviation provided), and mostly men (84% in total, no standard deviation provided). Fifty eight percent are reported as being White, and 28% Black. Participants had been homeless for just under 5 years on average, and had spent 49 of the previous 60 days experiencing street homelessness.

The researchers measured a number of outcomes, including frequency and severity of drug and alcohol use, a number of symptoms of mental ill health, and several housing stability measures. We were unable to calculate effect sizes from the published data, as the authors did not provide standard deviations or standard errors. The authors report findings from two sets of analysis: treatment versus control (where data from the residential and non-residential programmes were combined for analysis), and residential versus non-residential treatments.

The authors report that there were no significant differences between treatment and control groups in terms of substance use outcomes, although they do report that both groups saw improvements in this area. The authors also report no significant differences between the residential and non-residential treatment groups in terms of substance use outcomes. The paper makes clear that a significant proportion of those assigned to each treatment group failed to attend either programme, with 25% of participants completing the residential programme and 8% completing the non-residential programme. The research team noted significant exposure effects at 3 months, concluding that longer exposure may results in greater improvements. However, they also noted that this exposure effect was not seen at six or 9 month follow ups.

Kendon Conrad and colleagues (Conrad, 1998) evaluated a 6-month duration residential rehabilitation programme which included follow up and case management services. Overall, the intervention was intended to last for a year. The intervention was targeted at veterans experiencing homelessness and problematic substance use. The intervention was based in a 30 bed unit, located on hospital grounds. Those participating in the programme undertook inpatient detoxification at the hospital, followed by 3 to 6 months in the residential rehabilitation unit, followed by a period of follow up and case management. The evaluation lasted for 5 years, with a sample of n = 358 participants (n = 178 in the treatment, and n = 180 in the control group). The control group was treatment as usual, which involved inpatient detoxification followed by referral to half-way house or community based non-residential support services. All of the sample were men, as only men were eligible for inclusion in the study. The average age of the sample was 40 years, ranging from 25 to 70 years (standard deviation not provided). Most of the sample were African Americans (75%). All of the sample experienced problematic substance use, with two thirds had problems with two or more substances.

The research team utilised a number of outcome measures, including several around frequency and severity of use of alcohol and drugs. We have calculated the odds ratios for the reported composite of six alcohol measures at 0.43 (95% CI between x and y) (Cohen's d = −0.476), and the odds ratio for the reported composite of six drug measures at 0.74 (95% CI between x and y) (Cohen's d = −0.169). The researchers reported that both the treatment and control groups improved over the intervention duration, with the treatment group doing better early on, but the effects decreasing over time. In terms of drug use, again both groups saw improvements compared to baseline, with statistically significant differences between the treatment and control groups at 3, 6 and 9 months. As with the alcohol measure, these effects diminishing over time. The research team stated that they expected the 3 month differences between treatment and control would be greatest at 3 months, because the treatment group would still be in the residential accommodation at this point. They also concluded that the small effects at 12 months were puzzling.

The final residential rehabilitation study from the 1990s included in this systematic review is by Gerald Stahler and colleagues (1995), which compares a residential rehabilitation intervention, a shelter-based intensive case management intervention, and treatment as usual, based in Philadelphia, Pennsylvania. Both the case management and treatment as usual groups were shelter based. The study lasted for 18 months, and recruited n = 722 participants, randomly assigned to the residential rehabilitation treatment (n = 222), intensive case management (n = 200), and control group (n = 302). The study sample only included men. The average age was 32.6 years, 23% were veterans, and 92% are recorded as being Black. On average, the individuals in the sample spent 52 of the previous 60 days experiencing homelessness.

The researchers examined a number of outcomes, including housing stability, drug and alcohol use, employment, and what are described as ‘psychological troubles’. Of interest here are outcome measures around alcohol use and cocaine use, and the study also reported days abstinent over previous 30 days from alcohol and cocaine.

We were unable to calculate effect sizes from the published data, as the authors did not provide means or standard deviations. The research team report stated their assumption that the residential rehabilitation treatment would be more effective than the intensive case management intervention, and both would fare better than the treatment as usual. They report that there were improvements across all outcomes for all three groups, but no significant differences between the two treatment groups, or between the treatment groups and the control group. The authors state that those in the two treatment groups reported higher levels of satisfaction about the services they received.

More recently, in 2011, Ilan Harpaz-Rotem and colleagues (2011) examined the effectiveness of residential treatment services for women veterans. This study focused on eleven Homeless Women Veteran Programmes, which were established in Veteran Affairs Medical Centres. The research team compared participants recruited between 2000 and 2005 who received at least 30 days of residential treatment with those who did not (i.e., the comparison group included women who had received less than 30 days residential treatment and no residential treatment at all). Women were not randomly assigned to the treatment group in this study. Outcomes were measured at baseline and at 1 year. Outcomes included employment, housing status, and health status. Health outcomes included alcohol and drug composite scales from the Addiction Severity Index, using self-reported measures of use and days of use over the past 30 days. The authors acknowledge that drug and alcohol use was not objectively observed, and go on to state that this ‘may be particularly problematic because in most cases the client's clinician was the person conducting the research interview’ (p. 897).

The study participants included 217 women in residential treatment, and 234 women in the comparison group. At 12 months follow up, 96 women remained in the treatment group and 119 in the comparator. The average age of the treatment group at baseline was 43.5 (SD 8.9 years) compared to 43.9 years (SD 6.7 years). The authors report the lifetime homelessness of both treatment and comparator groups at 28 months (SD 44 months) and 26 months (SD 31.7 months) respectively. Literal homelessness was also measured, with 16.1% of the treatment group and 22.4% of the comparator group being literally homeless at baseline.

The authors report that on the days of alcohol use measure, the treatment group had higher use on average, and both treatment and comparator groups had significantly decreasing days of use over time. The difference in change of use between the two groups was not significant (p = 0.03), controlling for baseline. The treatment group also had significantly higher scores on average for the ASI Drug scale (p < 0.001), controlling for baseline, particularly at 3 months (p = 0.01) and 6 months (p = 0.001). It is also reported that groups also showed significantly decreasing days of drug use over time, but the treatment group did not have a significantly different change in use (p = 0.07), controlling for baseline.

The authors make clear that there were differences between the 11 different centres in terms of clinical services and case management approaches. They commented that the differences ranged from ‘from very small professional staffs with primarily peer-led groups to large professional staffs, and from Alcoholics Anonymous group models to intensive cognitive-behavioral treatment approaches’. The authors speculate that the provision of stable accommodation during an significant point of transition might be the important ingredient that makes the difference to outcomes, and suggest that more research is needed.

Talking therapies (including cognitive behaviour therapy)

Talking therapies involved the individual or group talking with a trained professional about their thoughts, feelings, or behaviour. Talking therapies might include or be described as counselling, psychological therapies, psychosocial therapies, and including interventions such as cognitive behavioural therapy. (A number of specific therapy interventions, including harm reduction psychotherapy, motivational interviewing, and therapeutic communities, are dealt with separately.) These interventions typically adopt a harm reduction approach.

Two included studies examine the effectiveness of talking therapies, both as part of multi-component interventions. One considers an intervention that includes dialectic behavioural therapy, and one that examines an intervention called MISSION-Vet. These findings from these two studies are mixed overall. The first study says that it was effective for drugs only or alcohol only, but not for drugs and alcohol. The second study suggests the intervention was not effective. Taken together, these studies provide no clear insight into whether the interventions are effective.

Dialectic behavioural therapy is based on cognitive behavioural therapy, adapted for individuals who feel emotions intensely. Its main goals are to teach people how to live in the moment, develop healthy ways to cope with stress, regulate their emotions, and improve their relationships with others. An intervention that combined dialectic behavioural therapy and case management, targeting women who were on probation/parole and experiencing homelessness, was evaluated between February 2015 and November 2016 across four sites in California by Adeline Nyamathi and colleagues (Nyamathi, 2017).

This RCT compared this intervention, which was delivered over 6 weekly group-based sessions and 6 weekly one to one sessions, with a health promotion intervention. The evaluation involved n = 130 women aged between 18 and 65 years, with a history of problematic drug use and who were homeless when released from prison. Participants were equally split between the intervention and comparator, with an average age in the treatment group of 38.6 years (SD 11.3) and 39.1 years in the control group (SD 11.5). The authors report that the two largest ethnic groups in both the treatment and control were Black (44.6% and 36.9% respectively) and Latino (40% in each group). They also report that 10.8% of the treatment group and 16.8% of the control group were White. The primary outcome of interest was drug abstinence, with alcohol abstinence also being of interest. Outcomes were measured using self-reported measures, but also objectively with a urine test.

The authors report that there were improvements in drug abstinence at the 6-month follow up for both treatment group and control group participants, with a greater increase seen in the treatment group. They also report that participants in the treatment group were more likely to become or remain abstinent from alcohol than those in the control group. In discussing drug and alcohol abstinence, the authors state that ‘the differences in increased odds of substance abstinence (abstinent for both drugs and alcohol) were not significant (i.e., the interaction term was nonsignificant; OR = 2.39, 95% CI [0.92, 6.23], p = 0.07)’ (p. 437).

The second paper included in this systematic review that examines talking therapies is by a team of researchers led by David Smelson (2018), which evaluated an intervention called Maintaining Independence and Sobriety Through Systems Integration, Outreach and Networking – Veterans Edition, known as MISSION-Vet. This intervention was an adapted form of the HUD-VASH programmes which provides subsidised housing and case management service. The adaptation evaluated in this study included ‘certain enhancements like integrated dual disorders treatment, peer support, supported employment and trauma informed care’ (p. 2) and includes a Critical Time Intervention and Dual Recovery Therapy (p. 3). The study focused on a number of outcomes, including treatment engagement, service utilisation, housing, and drug and alcohol use. It is these last two that are relevant to this systematic review.

A total of n = 168 individuals participated in the evaluation, of which n = 81 were in the treatment group. The paper provides some demographic information about participants, stating that treatment group participants were significantly older, had been in housing longer, were less likely to need treatment for a medical problem than those in the control group. No information on age, sex, or ethnicity of participants is provided.

The researchers provide results of mixed effects models for drug and alcohol use, with adjusted odds ratios. They do not provide CIs, standard errors, or p-values. The authors report treatment versus control for changes in alcohol and drug use, stating that ‘neither the main effect of membership in the MISSION GTO group, nor the interaction with time were statistically significant predictors in the other outcome models’ (p. 5). This suggests that the intervention was not effective. The authors provide odds ratios of 0.84 (p = 0.19) for drug use, and 0.83 (p = 0.11) for alcohol use, for treatment × time versus control.

Therapeutic communities

Therapeutic communities are structured, participatory, group-based, residential interventions for long term mental ill health, personality disorders, and problematic substance use. Treatment focuses on drug abstinence, coupled with social and psychological change that requires a multidimensional effort involving intensive mutual self-help typically in a residential setting. Therapeutic communities are often a component in an overall intervention, and there is a body of research evidence around these types of interventions.

None of the included studies used the term therapeutic communities to describe the intervention of interest. Several papers cover interventions that include components or approaches that are consistent with the definition of therapeutic communities, but are not identified as such. In most of these cases, the intervention is multi-component, where the primary component is one that is included in our typology of interventions, and these are included elsewhere in this systematic review.

One paper, by Gerald Stahler and colleagues (2005), evaluates an intervention that appears to be a therapeutic community. The paper covers an intervention called ‘Bridges to the Community’. This programme was developed to work within a residential rehabilitation service, and involved peer mentors called community anchor persons, and group activities including workshops, training sessions, and cultural and recreational activities. The programme was targeted at African American women with children at risk of homelessness, lasted between 6 and 9 months in a residential programme called Hutchinson Place in Philadelphia, Pennsylvania. The programme was compared with standard treatment, which involve residential rehabilitation described by the researchers as being ‘largely 12-step orientated’ (p. 174).

The authors make clear that this was a preliminary demonstration project, and that participants were not randomly assigned to either the treatment or control group. N = 47 women participated in the treatment group, with an average age of 34.4 years (SD 4.8 years), of whom 95.8% were African American. The researchers report that the treatment group were older than the control group, but that this was not considered to be material. The treatment group reported lower drug and alcohol use in the previous 30 days, which the researchers took into account in their analysis.

The paper includes three outcome measures of interest: (1) average days use of cocaine in the past 30 days; (2) average days use of alcohol in the past 30 days; (3) average days use of more than one substance in the past 30 days. The researchers report the means and standard deviations for the treatment and control group, as well as the F-value, p-value, and effect size. The authors state that ‘at follow-up, 100% of participants in the Bridges group reported that they had abstained from cocaine in the last 30 days compared to the standard treatment group, which reported an average of about 2 days of use’ (p. 176) and that both groups saw reductions in use. Women in the treatment group were also more likely to remain in treatment.

Group work

Group work in the context of substance use treatment can involve a variety of group treatment models to meet client needs during the multiphase process of recovery. This may include skills development groups, cognitive–behavioural/problem solving groups, or support groups.

One of the included studies examined group work as one element of an overall intervention called LEAP (Life Enhancing Alcohol-management Programme) and published by Seema Clifasefi and Susan Collins (Clifasefi, 2020). The paper includes details on how the intervention was developed through a coproduction approach called community-based participatory research (CBPR), as well as details on the intervention and how it was tested. All participants were Housing First residents, who were non-randomly assigned to the treatment group and control group. A total of n = 116 individuals participated in the research, of whom n = 66 were in the treatment group (a single-site Housing First programme that offered the intervention) and n = 50 in the control (across two single-site Housing First programmes that did not offer the intervention). All three Housing First sites were in Seattle, Washington.

The authors report that there were not significant differences between the treatment and control groups for alcohol quantity or alcohol-related harm (p's > 0.06); but there was a significant difference in terms of non-drinking days (Wald χ ²(4, N = 105) = 23.01; p < 0.001) The authors conclude that ‘there was a significant main effect for the intervention group, which indicated lower odds of nondrinking days (OR = 0.25; robust SE = 0.11; p = 0.002) at baseline for LEAP versus control participants. The time × group interaction, however, was not significant (p = 0.71)’ (p. 769).

Research Question 6: How do participant and study characteristics moderate the effect of interventions designed to reduce substance use in adults who are experiencing homelessness?

Implications from the review for policy, commissioning, and practice

This systematic review aimed to understand the effectiveness of substance use interventions in reducing, stopping, or preventing problematic substance use by adults experiencing homelessness, and to provide insight for policy makers, commissioners, and service providers on which interventions are more effective. Our review points to the potential benefits of individual interventions designed to reduce substance use. Nevertheless, high rates of attrition in many studies and small average effect size make it difficult to have high confidence in the interventions. The evidence points to the need for commissioners and service providers to be realistic and expect high attrition rates given the often complex and chaotic nature of this population.

Using incentives can be beneficial

Although our analysis of harm reduction versus treatment as usual, abstinence versus treatment as usual, and harm reduction versus abstinence suggests that these different approaches make little real difference to the outcomes achieved, the findings suggest that some individual interventions are more effective than others. The meta-analysis of CM, an abstinence-based group of interventions, which used positive or negative consequences (typically vouchers or exclusion from housing or services) to alter the form and frequency of an individual's behaviour, was found to be most effective in reducing, stopping, or preventing problematic substance use by adults experiencing homelessness. On average, our analysis identified a large reduction in substance use for people experiencing homelessness receiving a contingency management intervention compared to treatment as usual or a non-contingency management intervention.

Case management does not significantly impact on substance use

Our analysis of case management interventions – ACT and ICM – was less positive. These approaches typically involve outreach, assessment, planning, onward referral to relevant services, and advocacy. Despite these interventions gaining popularity in recent years in policies to tackle homelessness such as Housing First and permanent supportive housing, we found that these interventions lacked any significant impact on substance use. Hence, while they are supported by the evidence base in terms of positive housing and mental health outcomes, they appear less successful in reducing substance use.

Evidence based policy gaps

It is vital that policy makers, commissioners, and service providers have access to high quality evidence about which substance use interventions work – and which do not work – for this population. However, our review found that the existing evidence base available for policy makers, commissioners and services to inform service development is lacking in several areas. There is a clear need for a more robust evidence based to be developed beyond North America. This evidence needs to include the often-lacking voices of people with lived experience of homelessness in the research design and should also focus specific attention on the effectiveness of substance use interventions for women.

We found that almost all of the interventions were generally undertheorised and lacked explanations such as the expected casual mechanisms through which interventions were expected to work to reduce, stop, or prevent problematic substance use by people experiencing homelessness. More consideration should be given by policy makers, commissioners and service providers to developing the theoretical justification for the chosen approach around what works.

The key policy focus of this review related to the effectiveness of abstinence requirements and harm reduction approaches in substance use interventions for adults experiencing homelessness. Yet no studies that met the inclusion criteria involved a comparison between an abstinence-based treatment with a harm reduction control. A particular evidence gap relates to harm reduction focused interventions. Many of the harm reduction focused interventions lack evaluated outcome measures in adults who are experiencing homelessness. For example, we found no studies that focused on overdose prevention centres.