Outreach programs to improve life circumstances and prevent further adverse developmental trajectories of at-risk youth in OECD countries: A systematic review

Evidence on the effectiveness of outreach programmes for at-risk youth in OECD countries is inconclusive

The evidence on outreach programmes to improve life circumstances and prevent further adverse developmental trajectories of at-risk youth in OECD countries is inconclusive.

In this review, we aimed to find evidence of the effectiveness of outreach programmes on improving at-risk youth's life circumstances. However, the evidence is inconclusive because of the small number of studies.

What is this review about?

At-risk youth are defined as a diverse group of young people in unstable life circumstances, who are currently experiencing, or at risk of developing, one or more serious problems. At-risk youth are often very unlikely to seek out help for themselves within the established facilities, as their adverse developmental trajectories have installed a lack of trust in authorities.

A number of outreach programmes have been established seeking to help these young people on an ad hoc basis, meaning that the interventions are designed to fit the individual needs of each young person rather than as a one-size-fits-all treatment model.

What is the aim of this review?

This Campbell systematic review examines the effects of outreach programmes on problem/high-risk behaviour of young people between eight and 25 year old, living in OECD countries. The review summarises evidence from five studies undertaken in the USA and Canada that involved 578 participants in total.

What studies are included?

Included studies had to examine the impact of targeted outreach programmes on at-risk youth. Studies had to have a comparison group.

Sixteen studies analysing 17 different interventions were identified. Of these, only five studies could be used in the data synthesis. The studies were from the USA and Canada. There were four randomised controlled trials (RCTs) and 12 non-randomised studies. The studies contained data for 578 participants.

What are the main findings of this review?

The evidence was inconclusive. At most, the results from two studies could be pooled in a single meta-analysis. The outcomes drug (other than marijuana) use, marijuana use and alcohol use each at two different time points (one and three months follow up) were meta-analysed. In addition, a number of other outcomes were reported in a single study only.

What do the findings of the review mean?

The current landscape of research on outreach programmes targeting at-risk youth in the OECD countries shows that it has yet to be evaluated thoroughly. The evidence was inconclusive because too few studies reported results on the same type of outcome.

Furthermore, all the available evidence used in the data synthesis was from the USA and Canada, and so the findings may not be generalisable to other settings and systems outside Northern America.

None of the studies used in the meta-analyses reported on long term impacts.

These considerations point to the need for more rigorously-conducted studies reporting a larger number of outcomes.

How up-to-date is this review?

The review authors searched for studies published up to November 2021.

Description of the condition

At-risk youth may be defined as a diverse group of young people in unstable life circumstances, who are currently experiencing or are at risk of developing one or more serious problems such as school failure or drop-out, mental health disorders, substance and/or alcohol abuse, unemployment, long-term poverty, delinquency and more serious criminal behaviour (Arbreton et al., 2005; Quinn, 1999). At-risk youth typically have a multitude of social and psychological problems and typically also come from families considered at-risk (Treskon, 2016). They may occasionally or permanently be homeless and spend time in the streets.

No readily available statistics on the numbers of at-risk youth exist but statistics on the numbers experiencing the adverse outcomes can be found. For example, according to the National Conference of State Legislatures (NCSL) on any given night, approximately 41,000 unaccompanied youth ages 13-25 experience homelessness in the US (NCSL, 2019). It is estimated that 4.2 million youth and young adults experience homelessness each year, and that 10% of young adults ages 18–25, and at least one in 30 adolescents ages 13–17, experience some form of homelessness over the course of a year (NCSL, 2019). A substantial part of them report having a number of other problems too; for example having substance misuse problems (29%), mental health problems (69%) or been in the juvenile justice system, in jail or detention (50%), Further, school drop-out and no high school diploma or General Equivalency Diploma (GED) is the number one correlate for elevated risk of youth homelessness (NCSL, 2019). In Denmark the numbers are much lower. The estimated number of homeless youth, less than 25 years of age, was 1036 in 2019 (Benjaminsen, 2019) which amounts to less than 1% of those aged 13–24 years; but in line with the evidence from the US a large part of them have other problems (e.g., substance misuse and mental health problems) as well and the majority in the age group 18–24 are NEET, that is, neither employed nor in education or training (Benjaminsen et al., 2020). Numbers of homeless youth across OECD countries are hard to locate and definitions of homelessness vary across countries (OECD, 2020a) but most likely, there is as great variation as in other indicators of at-risk youth. For example, the rates of school drop-out, those that do not reach a basic minimum level of skills, is on average 19% across OECD countries and range from 2% in Korea to 58% in Turkey for the 25–34 years-old (OECD, 2012). Also, the NEET rates vary a lot across OECD countries; from less than 7% of the 15–29 year old in Iceland and the Netherlands to more than 37% in South Africa with an OECD average of 13% (OECD, 2020b).

At-risk youth are often very unlikely to seek out help for themselves within the established venues, as their adverse developmental trajectories have installed a lack of trust in authorities such as child protection agencies and social workers (Ronel, 2006). To help this population, a number of outreach programmes have been established seeking to help the young people on an ad hoc basis, meaning that the interventions are designed to fit the individual needs of each young person rather than as a one-size-fits-all treatment model (Korf et al., 1999; Svensson, 2003). The programmes are often multicomponent interventions and often rely on volunteers as outreach workers, as these are proposed to offer the young people a unique possibility for forming trusting relationships due to the fact that help is offered as an act of altruism (Ronel, 2006). The programmes may offer basic necessities such as food or shelter, and they may offer counselling, mentoring and medical assistance. What define the outreach programmes is that they are targeted at helping the young people away from the streets and their current adverse developmental paths towards more stable living situations and developmental prospects.

Due to the very nature of the programmes, the effects are difficult to determine. First, randomisation is difficult when there is no system of referral, and the uniquely tailored interventions, which each young person receives raises the question if one can even describe the intervention as uniform even within the same programme. Second, the aims of the programmes are typically to change the long-term developmental paths of the participants, but longitudinal studies are often not feasible, and the establishment of long-term preventive effects is difficult. However, even if the obstacles are many, it is still important to explore the efficacy of outreach programmes, as the stakes are extremely high. If left alone, the target population of at-risk youth are likely to develop serious long-term problems, which are not just detrimental to the individual but also very costly to societies.

Description of the intervention

The intervention in this review is targeted outreach work which may be (but does not have to be) multicomponent programmes in which outreach may be combined with other services. There are different meanings of the concept outreach work throughout Europe and a wide variety of outreach initiatives with different arrangements where outreach may work in one or many ways (Svensson, 2003).

The term outreach work as we will use it in this review is commonly known throughout Scandinavia and is corresponding with detached youth work in England (similar to street work or fieldwork, Korf et al., 1999). Detached outreach work is executed outside any agency setting, is taking place in the community where groups of marginalised youth are known to meet, with the aim of engaging young people who lack any kind of belonging by directing young people to treatment or care services when necessary. It may be based on voluntary efforts, peer groups or professionals, social workers, social pedagogical workers and health workers but the common nature is to meet the young people on their own terms. Outreach work is based on voluntary participation and is an important approach for intervening with hard to reach populations, and identifying their needs in a flexible and responsive manner with no manual-based restrictions.

However, an outreach programme may be associated with a specific service or combination of services offered by one or more organisations targeting a specific population. The services combined with the outreach component could be case management or participation in community programmes or even a continuum of comprehensive services including education, employment, and intensive supervision.

Outreach efforts with services only focusing on nutritional and medical care (e.g., testing for HIV) was excluded.

The comparison population were young people at-risk who are not contacted by the outreach workers and are not encouraged to attend any services.

How the intervention might work

The primary mechanism of change in outreach work with at-risk youth is to facilitate positive change by gradually building up a sense of trust between the young person and the outreach worker(s) (Svensson, 2003). Characteristically, the aim of the outreach youth worker is to find solutions to young people's problems in their own environment, rather than deciding while sitting behind a desk what they consider best for the person concerned. The goal is always to prevent further marginalisation and encourage social integration (Svensson, 2003).

Theoretically, outreach work may be understood through an empowerment lens. Empowerment theory is both a value orientation for working in the community and a theoretical model for understanding the processes whereby individuals gain access to resources and acquire skills and knowledge enabling them to take advantage of opportunities within the community and to exert control and influence over decisions that affect their lives (Zimmerman, 2002). As a value orientation empowerment theory proposes that many social problems exist because of unequal distribution of, and access to, resources within the community. The theory further suggests that many individuals are best served by mutual help, helping others or working for their rights rather than having their needs fulfilled by a benevolent professional (Perkins & Zimmerman, 1995; Zimmerman, 2002). What this means is that outreach work is aimed at enabling the at-risk young person to function more autonomously and adaptively within their community rather than just providing a quick fix for their current problems. Empowerment theory proposes that by identifying strengths rather than pointing out and cataloguing risk factors, at-risk youth may become motivated to actively engage in their own positive change. Outreach work may thus also be understood as aimed at promoting resiliency by enabling the young person to make better use of their personal and social resources. Theoretically a number of protective factors may serve to buffer the adversity a young person might be exposed to. Protective factors at the personal level may include being physically healthy, having a good self-esteem and adaptive coping skills. At the family level protective factors may include having a supportive network of family or friends and at the societal level protective factors may include living in a community with access to support. Thus, outreach work may be seen as drawing on resiliency theory when working to assist the young person in identifying protective factors (Zimmerman et al., 2013). As proposed by Rappaport (1985) social change based on empowerment is proposed to be brought on by a change of both language and conceptions. Instead of perceiving the outreach workers and at-risk young people as ‘professionals’ and ‘clients’, empowerment thinking proposes a bidirectional relationship between helpers and participants. In outreach work this means that the outreach workers aim to meet the at-risk youth with a non-judgemental approach characterised by genuine empathy rather than prejudice and victim blaming (Svensson, 2003; Zimmerman, 2002). In addition to meeting the youth with empathy outreach workers strive to become ‘culturally competent’ which may be defined as the willingness to understand young people from different cultural and social backgrounds and the ability to put oneself in their situation. It also includes the ability and readiness to sympathise with young people subjected to prejudice, social exclusion and stigmatisation, and to approach each young person with respect, open-mindedness and commitment (Svensson, 2003).

As stated in the introduction at-risk youth often come from socio-economically less advantaged and dysfunctional families (Treskon, 2016). At risk youth have often experienced at number of adverse events such as poverty, emotional or physical abuse and neglect, out-of-home placement, living with mentally ill or substance abusing parents and unstable housing situations leading to a lack of continuity in their education. Thus, at-risk youth often lack stable attachment figures and suitable adult role models, which leads to a lack of adaptive life skills and compromises their ability to seek appropriate help within established venues. Early adverse experiences may also lead to a deeply installed mistrust of authorities and thus at-risk youth are often unlikely to seek out help for themselves. In line with empowerment thinking, outreach programmes seek to meet the young person at their own terms offering them the specific help they need here and now and thus slowly building up a trusting relationship which may be used for future motivational work (Svensson, 2003). Outreach workers aim at offering the young person a positive adult role model and thus provide the young person with the kind of socio emotional support which they often lack. Sometimes outreach workers may teach the young person basic life skills, such as personal hygiene, offer assistance with homework or writing job applications, paying bills, getting help for substance or alcohol abuse problems and being on time for work or school, or they may accompany the young person to meetings with authority figures, which are fear-inducing in the young person due to their negative past experiences. Furthermore, outreach work may include tutoring programmes, or offer assistance with baby-sitting and housing for socially disadvantaged teenage mothers. What characterises all efforts is that they seek to support and instal a sense of empowerment within the young person which may enable them to master similar challenges in the future in a more adaptive way and to motivate the young person to behaviour changes which may facilitate further social re-integration (Perkins & Zimmerman, 1995; Svensson, 2003; Zimmerman, 2002).

In sum, empowerment theory provides a framework for understanding the mechanisms of change within youth outreach work. The goal of outreach work with at-risk youth is to facilitate positive long-term social change by motivating the young person to become actively engaged. Based on Svensson (2003) the theoretical approach to youth outreach work is based on the following principles: –

Distribution of services where youth, subcultural groups, young people at risk and young drug users are present in their own environment.

Why it is important to do this review

We have located one systematic review on outreach programmes for youth; however, it only included programmes for street-involved youth, a term used by the authors instead of homeless youth (Connolly & Joly, 2012). The participant population was young people aged 12–25, who did not have a permanent place of residence. Furthermore, it only included articles published in peer-review journals between 1990 and 2010 and had no restrictions on how the studies measured an impact (i.e., studies without comparison groups were included). The only impact result reported is on later participation rates in the offered service.

Further, we have located five systematic reviews on street-connected and/or homeless youth.

The systematic review by Coren et al. (2016); focused on street-connected children and young people (i.e., living on, or closely connected to, the street), from birth to 24 years, and included studies of harm reduction or reintegration interventions that used a comparison group study design. The searches were performed up to April 2015. The primary outcomes of the review were inclusion and reintegration. The secondary outcomes were measures of health, well-being and educational and occupational achievement. Thirteen studies were included and most of them compared therapy-based services versus usual shelter and drop-in services, or versus other therapeutic/health interventions.

Another systematic review on homeless youth (between the ages of 12–24 years) focused solely on HIV/acquired immunodeficiency syndrome (AIDS) prevention programmes (Naranbhai et al., 2011). The searches were performed up to December 2010 and only randomised controlled trials were included.

In the systematic review by Altena et al. (2010), studies published up to 2008 were included if they empirically examined the effectiveness of an intervention for homeless youth. Randomised as well as non-randomised studies and studies without a control group, that is, before-after studies were included. No meta-analysis was performed, only a narrative analysis describing each study and results.

The systematic review by Slesnick et al. (2009), included runaway, shelter, street or drop-in centre recruited youth between the ages of 12–24. In addition to intervention studies, the review also included studies assessing youth outcomes after shelter or drop-in utilisation (i.e., service evaluations) and qualitative studies. No meta-analysis was performed, only a narrative analysis describing each study and results. When the searches were performed is not reported.

In Xiang (2013), studies that examined the effectiveness of interventions to improve substance abuse problems amongst homeless youth between the ages of 12 and 24 were included. Searches were performed up to April 2012. Only studies that reported data on substance use outcomes were included. Randomised as well as non-randomised studies and studies without a control group, that is, before-after studies were included. No meta-analysis was performed, only a narrative analysis describing each study and results.

Three systematic reviews were found, focusing explicitly on mentoring interventions for youth.

Tolan et al. (2008) performed a systematic review on mentoring intervention with the aim of affecting juvenile delinquency and associated problems for youth, defined as persons under age 18. The review was limited to studies conducted within the United States or another predominately English-speaking country reported between 1970 and 2005. Eligible outcomes were measures of juvenile delinquency, aggression or high levels of externalising problems, drug abuse and academic achievement/school failure.

DuBois et al. (2002) searched for studies from 1970 through 1998 reporting on the effectiveness of one-on-one mentoring programmes for youth. The eligible age of youth is not reported but the average age of the youth participants in the study population had to be less than 19. The review included before-after studies, and excluded studies were the adult mentors were mental health professionals (e.g., social workers). Studies of peer tutoring or mentoring programmes were also excluded. It is unclear what the eligible outcomes were, all outcomes was analysed in one meta-analysis; however a moderator analysis distinguishes between the outcome types: emotional/psychological, problem/high-risk behaviour, social competence, academic/educational and career/employment.

DuBois et al. (2011) is a follow-up to DuBois et al. (2002) with some modifications. Before-after studies were no longer eligible, participants was required to be less than 18 years, studies of peer tutoring or mentoring programmes were now eligible and mentoring was not required to be one-on-one. The review included studies published between 1999 and 2010. Eligible outcomes were attitudinal/motivational, social/relational, psychological/emotional, conduct problems, academic/school, physical health, and career/employment.

Besides being up-to-date, a major difference between these nine systematic reviews and the review we have performed is, that we focused on programmes with a targeted outreach component for youth aged 8–25. Participants need not be homeless (but were eligible if they were), and we only included studies with a control group. All relevant outcome areas were analysed separately in meta-analyses taking into consideration the dependencies between effect sizes.

Criteria for considering studies for this review

Primary outcomes

The primary focus was on measures of problem/high-risk behaviour, such as delinquency/criminal behaviour, drug and alcohol use, high levels of externalising problems, school failure, sexual risk taking, gang involvement/membership, poverty, unemployment, runaway and homelessness.

Secondary outcomes

A secondary focus was on measures of social and emotional outcomes, such as internalising symptoms (anxiety, depression), self-identity, interpersonal relations and social awareness.

Search methods for identification of studies

We implemented a wide range of search methods and strategies to maximise coverage of relevant references, while simultaneously attempting to reduce different types of bias related to publication and dissemination systems. The different strategies and methods will be presented below.

Data collection and analysis

Selection of studies

Under the supervision of the review authors, two review team assistants first independently screened titles and abstracts to exclude studies that were clearly irrelevant. Studies considered eligible by at least one assistant or studies with insufficient information in the title and abstract to judge eligibility, were retrieved in full text. The full texts were then screened independently by two review team assistants under the supervision of the review authors. Any disagreement of eligibility was resolved by the review authors. Exclusion reasons for studies that otherwise might be expected to be eligible were documented and presented in section Excluded studies. The study inclusion criteria were piloted by the review authors (see Supporting Information: Appendix 3). The overall search and screening process is illustrated in Figure 1. None of the review authors were blind to the authors, institutions, or the journals responsible for the publication of the articles.

OPEN IN VIEWER

Figure 1

Flow diagram

Description of studies

Risk of bias in included studies

The risk of bias coding for each of the 16 studies and their outcomes is shown in a supplementary document.

Four studies reported on randomised trials, all individually randomised trials.

Table 6 shows a summary of the risk of bias associated with the randomised studies.

Three studies reported an appropriate method of randomisation and to some extent showed or discussed baseline imbalances on the pre-specified confounders. We rated all three studies Some concerns on the Randomisation Process item as they all had some issues with the balance on the pre-specified confounders. One study did not report the randomisation method but most likely it was concealed and there were no imbalances on the pre-specified confounders. This study was also rated Some concerns on the Randomisation Process item. On the Deviations from intervention item, all four studies were rated some concerns, mainly due to lack of blinding. Concerning missing outcome data, one study had no issues, and we rated it Low risk of bias, three studies were rated Some concerns. All four studies were rated Some concerns on the Measurement of Outcome item, mainly due to lack of blinding. One study was rated Low risk of bias on the Selection of Reported Results item, the remaining were rated Some concerns as there was no a priori analysis plan and an insufficient reporting of outcomes. Overall, none of the studies were rated Low risk of bias, they were all rated some concerns overall.

Unfortunately the study Herrera et al. (2013) did not report data in a form that permitted calculation of an effect size and standard error (see the supplementary document Numerical outcome for an explanation).

The assessment of one study (Walker et al., 2019) was rated using the ROBINS-I tool, as even though participants were randomised (although not using an appropriate method), youth assigned to the comparison group from a previous evaluation (approximately 1 year before the current study) was included. This compromised the randomisation to an extent that it was most appropriate to assess the risk of bias using the ROBINS-I tool.

The remaining 11 studies used non-randomised designs, 1 study (Arbreton & McClanahan, 2002) reported on 2 different interventions including different individuals so in total 13 interventions were rated using the ROBINS-I tool. Table 7 shows a summary of the risk of bias associated with the non-randomised studies. As stated in the protocol, we stopped the assessment of a non-randomised study outcome when it was rated ‘Critical’ on any of the items, therefore not all studies are rated on all domains. One study (McClanahan et al., 2012) stated that a detailed description of the analysis was summarised in the Technical Appendix, which unfortunately is not available. Thus, we could not assess risk of bias for this study, as there was very little (close to nothing) description in the main text. We contacted the author, but unfortunately she does not have the technical appendix and the publishing institution (P/PV) closed in 2012. The study could therefore not be rated.

Eight of the non-randomised studies were rated Critical risk of bias on the Overall judgement item corresponding to a risk of bias so high that the findings should not be considered in the data synthesis. The overall Critical risk of bias rating was mainly due to issues on the Confounding bias item; four were rated Critical risk of bias on this item; that is, they failed to establish a comparison group that was balanced on important confounders and further either did not control for any confounders or controlled for confounders but included only those participants who had an event (arrest) in either the pre-programme period or in the programme period in the analysis of that particular outcome (arrests). The remaining four studies rated Critical risk of bias on the Overall judgement item, were rated Serious risk of bias on several other items (all four on the Confounding bias and Selection bias items and one study in addition on the Reporting item and two studies in addition on the Measurement item) which lead to an Overall judgement rating of Critical risk of bias.

Three studies (reporting on four interventions) were rated Serious risk of bias overall. Unfortunately, the study (Arbreton & McClanahan, 2002) reporting on two interventions did not report data that permitted calculation of an effect size and standard error. We contacted the authors but unfortunately, the data is no longer available as the publishing institution (P/PV) closed in 2012. This left only two non-randomised studies to be used in the data synthesis.

Of the four interventions not rated Critical risk over bias overall, all had serious issues on this item. On the Selection bias item three were rated Moderate risk of bias and one was rated Serious risk of bias. All were rated Low risk of bias on the Classification item; three were rated Low risk of bias on the Deviation item and one was rated Moderate. On the missing data item two were rated Low risk of bias, one was rated Moderate and one was rated Serious risk of bias. On the measurement item, one was rated Low risk of bias and three were rated Moderate risk of bias. All four were rated Moderate risk of bias on the Selection of Reported Results mainly because there was no a priori analysis plan.

Effects of interventions

Synthesis of results

Five studies were not rated Critical risk of bias and reported data that permitted calculation of an effects size and standard error and could thus be used in the meta-analysis.

A large variety of different outcomes were reported in the studies (e.g., drug use, abstinence, housing, mental and physical health).

To carry out a meta-analysis, every study must have a comparable effect size. We synthesise effects separately by type of outcome (conceptual outcomes as outlined in section ‘Types of outcomes measures’) and time point (end of intervention and follow up). Unfortunately each type of outcome was only reported in a small subset of studies (in many cases in only one single study). Thus, each meta analysis contains a very small number of effect sizes, at most two. The studies included in the meta-analyses contribute only a single effect size to each analysis.

All continuous outcomes (effect sizes measured as Hedges g) were coded such that a larger effect size indicated better outcomes for the treated group. All binary outcomes (reported as odds ratio) were likewise coded such that a larger effect size indicated better outcomes for the treated group.

Primary outcomes

Two studies analysed the effect of outreach on three different substance uses in last 30 days: drug (other than marijuana), marijuana and alcohol. Both studies reported on outcomes at two time points; 1 month post-baseline and 3 months post-baseline.

Drug, other than marijuana

The random effects weighted standardised mean difference at 1 month post-baseline was 0.0 (95% confidence interval [CI]: −0.29 to 0.29) and not statistically significant. The forest plot is displayed in Figure 2. There was a very small amount of heterogeneity between the studies; the estimated τ ² was 0.01, Q = 1.27, df = 1 and I ² was 21% as displayed in Figure 2.

OPEN IN VIEWER

Figure 2

Analysis 1.1—1 month follow up

The random effects weighted standardised mean difference at 3 months post-baseline was 0.07 (95% CI: −0.18 to 0.33) and not statistically significant. The forest plot is displayed in Figure 3. There was no heterogeneity between the studies; the estimated τ ² was 0.00, Q = 0.41, df = 1 and I ² was 0% as displayed in Figure 3.

OPEN IN VIEWER

Figure 3

Analysis 1.2—3 months follow up

Marijuana

The random effects weighted standardised mean difference at 1 month post-baseline was 0.04 (95% CI: −0.21 to 0.29) and not statistically significant. The forest plot is displayed in Figure 4. There was no heterogeneity between the studies; the estimated τ ² was 0.00, Q = 0.39, df = 1 and I ² was 0% as displayed in Figure 4.

OPEN IN VIEWER

Figure 4

Analysis 2.1—1 month follow up

The random effects weighted standardised mean difference at 3 months post-baseline was −0.03 (95% CI: −0.29 to 0.22) and not statistically significant. The forest plot is displayed in Figure 5. There was no heterogeneity between the studies; the estimated τ ² was 0.00, Q = 0.37, df = 1 and I ² was 0% as displayed in Figure 5.

OPEN IN VIEWER

Figure 5

Analysis 2.2—3 months follow up

Alcohol

The random effects weighted standardised mean difference at 1 month post-baseline was 0.05 (95% CI: −0.21 to 0.30) and not statistically significant. The forest plot is displayed in Figure 6. There was no heterogeneity between the studies; the estimated τ ² was 0.00, Q = 0.23, df = 1 and I ² was 0% as displayed in Figure 6.

OPEN IN VIEWER

Figure 6

Analysis 3.1—1 month follow up

The random effects weighted standardised mean difference at 3 months post-baseline was −0.17 (95% CI: −0.43 to 0.09) and not statistically significant. The forest plot is displayed in Figure 7. There was no heterogeneity between the studies; the estimated τ ² was 0.00, Q = 0.28, df = 1 and I ² was 0% as displayed in Figure 7.

OPEN IN VIEWER

Figure 7

Analysis 3.2—3 months follow up

Other primary outcomes

In addition, a number of outcomes were reported in a single study only. The outcomes were measures on housing situation, NEET status, gang membership, externalising problems and delinquency/criminal behaviour. The effect sizes and 95% CIs are reported in Table 8.

OPEN IN VIEWER

Table 8

Additional primary outcomes

Author (year)	Measure	Outcome	ES [95% CI]
			SMD [95% CI]
Kidd (2020)	Housing Security Scale	Housing security	0.02 [−0.47, 0.51]
Kidd (2020)	Housing Security Subscale	Subjective housing security (housing satisfaction and perception of housing stability)	0.19 [−0.30, 0.68]
Kidd (2020)	GAIN Short Screener	Externalising	0.05 [−0.44, 0.54]
Kidd (2020)	GAIN Short Screener	Substance Use (note not actual use)	0.10 [−0.39, 0.59]
Walker et al. (2019)	Court records	Warrant ordered in Case setting	−0.35 [−0.53, −0.17]
Walker et al. (2019)	Court records	Warrant ordered in Arraignment	0.19 [0.01, 0.37]
Baer et al. (2007)	Self-reported, 1 month FU	Abstinence in last 30 days.	−0.34 [−0.77, 0.09]
Baer et al. (2007)	Self-reported, 3 month FU	Abstinence in last 30 days.	−0.40 [−0.83, 0.03]
			OR [95% CI]
Kidd (2020)	Self-reported	Housing	2.01 [0.31, 12.94]
Kidd (2020)	Self-reported	NEET	2.30 [0.66, 8.06]
Thompson and Jason (1988)	Comparing names with gang rosters provided by gang members involved with BUILD's remediation program.	Gang member	7.49 [0.81, 69.32]

Secondary outcomes

A number of secondary outcomes were reported in a single study only. The outcomes were Quality of Life, internalising symptoms, mental health, resilience, hope, mindfulness, interpersonal and community relations and support. The effect sizes and 95% CIs are reported in Table 9.

OPEN IN VIEWER

Table 9

Additional secondary outcomes

Author (year)	Measure	Outcome	ES [95% CI]
			SMD [95% CI]
Kidd (2020)	Mental Health Continuum Short Form	Mental health	−0.15 [−0.64, 0.34]
Kidd (2020)	Cognitive and Affective Mindfulness Scale	Mindfulness	−0.34 [−0.85, 0.17]
Kidd (2020)	Community Integration Scale		0.13 [−0.36, 0.62]
Kidd (2020)	Community Integration Measure		−0.07 [−0.56, 0.42]
Kidd (2020)	Connor-Davidson Resilience Scale	Resilience	−0.28 [−0.79, 0.23]
Kidd (2020)	Adult Hope Scale	Hope	−0.18 [−0.67, 0.31]
Kidd (2020)	GAIN Short Screener	Internalising	0.04 [−0.45, 0.53]
Kidd (2020)	Medical Outcomes Study	Emotional Supports	0.08 [−0.41, 0.57]
Kidd (2020)	Medical Outcomes Study	Tangible Supports	−0.08 [−0.57, 0.41]
Kidd (2020)	Medical Outcomes Study	Affectionate Supports	0.07 [−0.42, 0.56]
Kidd (2020)	WHO Quality of Life	Physical Health	0.06 [−0.43, 0.55]
Kidd (2020)	WHO Quality of Life	Psychological	0.05 [−0.44, 0.54]
Kidd (2020)	WHO Quality of Life	Social	−0.29 [−0.80, 0.22]
Kidd (2020)	WHO Quality of Life	Environment	−0.11 [−0.60, 0.38]

Summary of main results

Overall, there were too few studies included in any of the meta-analyses in order for us to draw any conclusion concerning the effectiveness of outreach. At most, the results from two studies could be pooled in a single meta-analysis. It was only possible to pool the outcomes drug (other than marijuana) use, marijuana use and alcohol use each at two different time points (1 and 3 months follow up). At 1 month follow up the weighted averages varied between zero and 0.05 and at three months follow up between −0.17 and 0.07. None of them were statistically significant.

Overall completeness and applicability of evidence

We included in total five studies in the data synthesis and of these, a maximum of two studies reported the same outcome and could be used in a specific meta-analysis. This number is lower than the number of studies (16) meeting the inclusion criteria. The reduction was caused by three different factors.

Eight studies were judged to have a Critical risk of bias and, in accordance with the protocol, we excluded these from the data synthesis on the basis that they would be more likely to mislead than inform. One study provided very little (close to nothing) information on the method of analysis in the main text and referred to a technical appendix for this information. We contacted the author, but unfortunately she does not have the technical appendix and the publishing institution (P/PV) closed in 2012. The study could therefore not be rated and could not be used in the data synthesis. Finally, two studies (reporting on three different interventions) did not report effect estimates or provide data that would allow the calculation of an effect size.

If all the included studies had provided an effect estimate with lower risk of bias, the final list of useable studies in the data synthesis would have been larger, which again would have provided a more robust literature on which to base conclusions.

All studies used in the data synthesis were from the USA and Canada. This narrow geographical coverage is a clear limitation of the review.

Long term follow-up analyses were not possible. This is also a clear limitation of the review.

It was not possible to examine the impact of the moderators nor sensitivity analyses for each outcome to check whether the obtained results were robust across study design and methodological quality.

Quality of the evidence

The majority of studies (12) used non-randomised designs, and four were randomised trials. Overall the risk of bias in the included studies was high. Among the non-randomised studies only three studies (reporting on four interventions) were not rated Critical risk of bias (in addition, one study provided too little information to be rated). The level ‘Critical’ means: the study (outcome) is too problematic in this domain to provide any useful evidence on the effects of intervention, and it is excluded from the data synthesis.

None of the randomised trials were overall rated low risk of bias, they were all assessed to have some concerns overall.

We examined the risk of bias using Cochrane's revised risk of bias tool, ROB 2 (Higgins et al., 2019) for the randomised studies and using the model ROBINS–I, developed by members of the Cochrane Bias Methods Group and the Cochrane Non-Randomised Studies Methods Group (Sterne et al., 2016) for the non-randomised studies.

The quality of the evidence in this review was enhanced by excluding studies assessed to be at critical risk of bias using the ROBINS–I tool from the data synthesis. We believe this process excluded those studies that are more likely to mislead than inform.

With two studies contributing effect sizes for one outcome (although reported at two different time points) it is of little use to discuss overall consistency in the direction and magnitude of effects and heterogeneity between studies.

Potential biases in the review process

We performed a comprehensive electronic database search, combined with grey literature searching, and hand searching of key journals. All citations were screened in teams by two independent screeners from the review team (TPC, MCTM., FSB., and FLWS), and one review author (TF) assessed all included studies against inclusion criteria.

We believe that all the publicly available studies on the effect of outreach on young people's problem/high-risk behaviour and social and emotional outcomes up to the censor date were identified during the review process.

However, six references were not obtained in full text.

We were unable to comment on the possibility of publication bias as at most two studies was included in the same meta-analysis. Thus, we cannot rule out that there are still some missing studies, which were not published or made public.

We believe that there are no other potential biases in the review process as two members of the review team (MCTM, FLWS) independently coded the included studies. Any disagreements were resolved by discussion. Further, decisions about inclusion of studies were made by the two teams of each two members of the review team (TPC, MCTM, FSB, FLWS) and one review author (TF). Assessment of study quality and numeric data extraction was made by one review author (TF) and each study was checked by another review author (NTD).

Agreements and disagreements with other studies or reviews

One systematic review on outreach programmes for youth; only including programmes for street-involved youth, a term used by the authors instead of homeless youth, was found (Connolly & Joly, 2012). The eligible participants were young people aged 12–25, who did not have a permanent place of residence. Eligible studies were articles published in peer-review journals between 1990 and 2010 and there were no restrictions on how the studies measured an impact (i.e., studies without comparison groups were included). The only impact result reported is on later participation rates in the offered service. It is reported that outreach programmes successfully engage 63% of the street-involved youth they connect with. It is not possible to compare this conclusion to the conclusions of the current review.

Further, four systematic reviews on street-connected and/or homeless youth were located.

Coren et al. (2016), focused on street-connected children and young people (i.e., living on, or closely connected to, the street), from birth to 24 years, and studies of harm reduction or reintegration interventions that used a comparison group study design were eligible. The searches were performed up to April 2015. The primary outcomes were inclusion and reintegration and the secondary outcomes were measures of health, well-being and educational and occupational achievement. Thirteen studies were included and most of them compared therapy-based services versus usual shelter and drop-in services, or versus other therapeutic/health interventions. The authors conclude that no consistent results on a range of relevant outcomes within domains of psychosocial health, substance misuse and sexually risky behaviours were found and there was considerable heterogeneity between studies. Only two of the included studies compared an outreach approach to no treatment (the two studies comparing brief, motivational interventions to no treatment also included in the current review, Baer et al., 2007; and Peterson et al., 2006).

In the systematic review by Altena et al. (2010), studies published up to 2008 were included if they empirically examined the effectiveness of an intervention for homeless youth. There were no restrictions on study design, that is, before-after studies were included. No meta-analysis was performed, only a narrative analysis describing each study and results. The overall conclusion was ‘no compelling evidence of the effectiveness of interventions for homeless youth can be presented on the basis of the results (and the effect sizes) of these studies’. (p. 643). The conclusion concerning outreach approaches is based solely on two studies comparing brief, motivational interventions to no treatment also included in the current review.

In Slesnick et al. (2009), runaway, shelter, street or drop-in centre recruited youth between the ages of 12–24 were included. The review also included studies assessing youth outcomes after shelter or drop-in utilisation (i.e., service evaluations) and qualitative studies in addition to intervention studies. No meta-analysis was performed, only a narrative analysis describing each study and results. When the searches were performed is not reported. The conclusion concerning outreach approaches is also based solely on the two studies comparing brief, motivational interventions to no treatment also included in the current review.

Studies that examined the effectiveness of interventions to improve substance abuse problems among homeless youth between the ages of 12 and 24 were included in Xiang (2013). Searches were performed up to April 2012. Only studies that reported data on substance use outcomes were included. All study designs, with or without a control group, that is, before-after studies were included. No meta-analysis was performed, only a narrative analysis describing each study and results. The conclusion concerning outreach approaches is based solely on the two studies comparing brief, motivational interventions to no treatment included in the current review.

In the current review, we focused on programmes with a targeted outreach component for youth aged 8–25. Participants needed not be homeless (but were eligible if they were), and we only included studies with a control group. However, as very few studies of a sufficiently low risk of bias and who further reported data that permitted calculation of an effects size and standard error were located, the conclusions in the current review do not disagree with the conclusions concerning outreach in the four systematic reviews described above.

Implications for practice

Public as well as private after-school programmes and youth clubs that provide healthy alternatives for youth may serve as important resources for reducing school failure and youth crime. However, it is questionable whether the youth who would benefit most are those who are attracted to and attend such programmes (Arbreton & McClanahan, 2002). Outreach work represents an important preventive working approach with the aim of attracting and serving the youth who are very unlikely to participate on their own and who probably need the help most.

Outreach programmes targeting at-risk youth are designed to reach the youth who need help to prevent high-school dropout, crime, drug abuse, and other forms of delinquency. Unfortunately the evidence on outreach programmes to improve life circumstances for at-risk youth in OECD countries was inconclusive because too few studies could be used in the data synthesis (see section Overall completeness and applicability of evidence). Further, the few studies included in the data synthesis did not report results on the same type of outcome, leaving very few observations to base a conclusion. Only two studies reported on the same outcomes and could be pooled in meta-analyses of drug and alcohol use. Furthermore, the two studies used in these meta-analyses both had a very brief intervention called ‘Brief Motivational Intervention’ consisting of motivational interviewing (Baer et al., 2007; Peterson et al., 2006), in which participants were offered feedback on alcohol, marijuana, and other drug use frequency and perceived norms. Participants were provided with a ‘colourful booklet’ from which they could choose topics for discussion and the average session was 30 min (ranging from 10 to 70 min). In one study (Peterson et al., 2006) additional feedback sessions were offered to youths, but few were scheduled and none were completed. Given the multitude and severity of problems experienced by the participants in the two studies (e.g., homelessness, alcohol and illicit drug use) it does not seem surprising that the Brief Motivation Intervention offered was not comprehensive enough to engage the youth or make a real difference in the lives of the participants. Thus, the nature of the interventions used in the meta-analysis also constitutes a severe limitation to the generalisability of findings, as it is possible that given the multitude and magnitude of problems experienced by at-risk youth, there is a need for comprehensive multi-component programmes in order for interventions to be effective.

Finally, all the available evidence used in the data synthesis was from either Canada or the USA, countries with a less developed welfare state and social security system (i.e., liberal regime countries) than for example the Scandinavian countries with comprehensive welfare state institutions (Esping-Andersen, 1990). Thus, the findings may not be generalisable to other settings and systems outside North America.

Implications for research

Further research is required to fully address the effects of outreach on young people's problem/high-risk behaviour. Few studies have investigated this issue using appropriate comparison groups, and none have investigated the long term effects. It should be acknowledged that research in this field is problematic for a number of practical and methodological reasons. Locating an appropriate control group is difficult as outreach programmes targeting at-risk youth is a preventive working approach with the aim of attracting and serving the youth who are very unlikely to participate on their own and who probably need the help most (Arbreton & McClanahan, 2002). Further, as the aims of the programmes are typically to change the long-term developmental paths of the participants, longitudinal studies are preferable.

We found only few randomised controlled trials and the risk of bias in most of the included non-randomised studies was very high leaving only two non-randomised studies to be used in the data synthesis.

These considerations point to the need for more rigorously conducted studies. Obtaining balance on important confounding factors may be difficult when participants are not randomised, which adds to the importance of statistically controlling for relevant factors.

Some studies reported only descriptive results even though data had been gathered on important confounding factors, such as age, ethnicity, risk factors and household characteristics The risk of bias due to confounding would be judged to be of less concern had the primary study authors controlled for these factors. As the data already are gathered it is recommended that analyses controlling for important confounding factors are carried out using these data. Unfortunately two of the included studies (one randomised and one non-randomised) did not provide data that permitted the calculation of an effect size and standard error (and attempt to achieve them was fruitless) and could therefore not be used in the data synthesis.

Given the limited number of rigorous studies available from countries other than the USA and Canada, it would be natural to consider conducting randomised controlled trials even though it might be argued that it is difficult for the population of interest. However, depending on the specific target population, an appropriate control group could be obtained in several ways. If the target population is homeless youth, treatment and control youth may be recruited from drop-in centres, shelters or other agencies serving homeless youth and to include an even broader range of youth street sampling methods could be used. All these techniques were for example combined in the randomised controlled study (Peterson et al. (2006)). If the target population is school children, schools could be the unit of randomisation or, as in the study Herrera et al. (2013) evaluating mentoring through the Big Brothers Big Sisters programme, In this study they reached at-risk youth in a number of ways, including collaborations with schools, partnerships with social service agencies, participation in community activities and events, word of mouth, and other media/communications strategies. Interested youth were then randomised to treatment and a wait list control group.

Such adapted trials in other countries than the USA and Canada would have the potential of making useful contributions to the outreach effectiveness literature if due consideration is made to the strengths and weaknesses of the studies found in this review. Thus, besides specific attention would have to be paid to stringency in terms of conducting a well-designed randomised trial with low risk of bias as well as ensuring that the sample sizes are large enough to enable sufficient power, the trials should also pay attention to reporting relevant outcomes with sufficient details for them to be included in an inverse variance weighted meta-analysis of standardised effect sizes. Further, trials performed in countries with access to administrative data about the participant's school, housing, employment and health outcomes (e.g., Denmark) would enable the investigator to report on long-term effects of the intervention.