Abstract
The objective of this Campbell systematic review was to assess the effectiveness of independent living programmes (ILPs) for young people leaving the care system.
The following electronic databases were searched: Cochrane Register of Controlled Trials (CENTRAL) (Issue 3, 2005); MEDLINE (1966 to June 2005); EMBASE (1980 to June 2005); CINAHL (1982 to June 2005); Psyc INFO (1887 to June 2005); Sociological Abstracts (1952 – June 2005); Applied Social Science Index and Abstracts (ASSIA) (1987– June 2005) and Dissertation Abstracts (to June 2005). All bibliographies were cross-referenced, and experts were contacted for unpublished or ongoing studies. Studies were included if they were randomised or quasi-randomised controlled trials comparing ILPs to standard care, another intervention, no intervention, or a wait-list control, for young people leaving care systems at their country's statutory age of discharge.
No study was found that met the inclusion criteria of the review. Eighteen studies using non-randomised or noncomparative designs were found, which generally reported favourable outcomes for ILP participants; however, reliable inferences cannot be drawn from these studies due to their use of weak methodology.
Results from randomised controlled trials show no evidence of the effectiveness of ILPs in improving or impairing outcomes for young people discharged from care. Further research into ILPs using randomised controlled designs is needed.
Synopsis
This review aimed to evaluate the effectiveness of independent living programmes (ILPs), a widespread and varied group of programmes intended to improve outcomes for foster care (looked-after) youth leaving the care system. There is evidence that of the many thousands of young people who are discharged from care each year, a sizeable number lack the life skills or resources necessary to succeed independently. Compared to the general population, these young adults face much higher rates of homelessness, unemployment, dependency on public assistance, physical and mental health problems, and involvement with the criminal justice system. ILPs, which incorporate independent living skills training, personal development, and educational and vocational support, are one strategy to improve these outcomes.
So as to incorporate only the highest-quality evidence, this review was limited to randomised and quasi-randomised controlled trials that assessed the effectiveness of ILPs for young people leaving the care system. Outcomes of interest included educational attainment, employment, health, housing, and other relevant life skills outcomes.
After an exhaustive search, no study was found that met our criteria. The primary reason for excluding studies was the lack of a randomised or quasi-randomised design. Eighteen studies utilising nonrandom comparisons, one-group longitudinal designs, or qualitative methodology were identified and are detailed in the table of excluded studies. The results of these studies generally favoured ILP participants for the outcomes of interest; however, their weak methodology makes it difficult to draw any firm or reliable inferences for policy and practice. On the whole, adverse effects were rarely observed.
This review is severely limited by the methodological quality of the evidence base for ILPs. Further research incorporating randomised designs is both feasible and necessary.
Abstract
Background
Independent living programmes (ILPs) are designed to provide young people leaving care with skills that will limit their disadvantage and aid in their successful transition into adulthood. Programmes focus on personal development, independent living skills, education, and vocational support.
Objectives
To assess the effectiveness of independent living programmes for young people leaving the care system.
Search strategy
The following electronic databases were searched: Cochrane Register of Controlled Trials (CENTRAL) (Issue 3, 2005); MEDLINE (1966 to June 2005); EMBASE (1980 to June 2005); CINAHL (1982 to June 2005); PsycINFO (1887 to June 2005); Sociological Abstracts (1952 - June 2005); Applied Social Science Index and Abstracts (ASSIA) (1987- June 2005) and Dissertation Abstracts (to June 2005). All bibliographies were cross-referenced, and experts were contacted for unpublished or ongoing studies.
Selection criteria
Randomised or quasi-randomised controlled trials comparing ILPs to standard care, another intervention, no intervention, or a wait-list control, for young people leaving care systems at their country's statutory age of discharge.
Data collection & analysis
2196 citations were identified and screened independently by two reviewers. Full text versions were obtained for 54 papers. None met the review's inclusion criteria.
Main results
No study was found that met the inclusion criteria of the review. Eighteen studies using nonrandomised or noncomparative designs were found, which generally reported favourable outcomes for ILP participants; however, reliable inferences cannot be drawn from these studies due to their use of weak methodology.
Reviewers' conclusions
Results from randomised controlled trials show no evidence of the effectiveness of ILPs in improving or impairing outcomes for young people discharged from care. Further research into ILPs using randomised controlled designs is needed.
Linked article:
Background
The Child Welfare System
It is widely agreed that good parenting consists of providing a safe, secure and stable environment in which children can develop to their full potential (
Prevalence and Experiences of Children in Public Care
Each year a large number of children enter public care systems around the world. There were 523,000 children in public care in the United States in 2003 (
Children in public care systems come from diverse backgrounds with different cultures, ethnicities, needs, abilities and pre-care histories (
Young People Leaving Care
Every year about 20,000 American and 6,000-8,000 English young people leave their respective public care systems (
Studies indicate that a significant proportion of young people leaving care do not possess the life skills or resources necessary to succeed independently. Upon leaving care they are more likely than youth in the general population to be homeless, unemployed, and/or dependent on public assistance; they are also more likely to experience physical and mental health problems, engage in risky health behaviours, and become involved with the criminal justice system (
In recognition of the difficulties facing young people leaving care, policies have been enacted to help prepare them for adulthood. These include the John H. Chafee Foster Care Independence Program of 1999 in the US (
Independent Living Programmes
Independent living programmes (ILPs) are designed to provide young people leaving care with skills that will limit their disadvantage and aid in their successful transition into adulthood. ILPs recognize that leaving care is a process, not an event, and that it requires social support and life skills preparation. These programmes are not intended to replace the supportive role played by a family, but instead aim to provide care leavers with skills that will help them succeed despite the absence of family support. In the main, ILPs utilise social skills training techniques, which incorporate instruction, modelling, roleplays and feedback. These training techniques have been used effectively to teach skills acquisition and improve youth performance in both clinical and non-clinical settings (
ILPs focus on both personal development skills and independent living skills. Personal development skills may include communication, decision making, conflict resolution, and anger management. Independent living skills include career exploration, job and interview skills, money management, household management, accessing housing, seeking legal assistance, and utilising community resources (
ILPs are frequently conducted in group formats with individual support (i.e., mentoring) provided on a one-to-one basis (
Notwithstanding the wide use of independent living programmes, their effectiveness is unknown (
This review aims systematically to determine the effectiveness of these independent living programmes in increasing the life chances of young people leaving care. Knowing the effectiveness of such programmes is important given the numerous challenges associated with living in and leaving public care.
Objectives
To assess the effectiveness of independent living programmes for young people leaving the care system.
Criteria for considering studies for this review
Types of studies
Randomised and quasi-randomised controlled studies (i.e., where allocation is by date of birth, alternate numbers, case number, day of the week, or month of the year) were eligible for inclusion. Since this review aimed to synthesise the evidence from study designs least prone to bias, quasi-experimental studies were not included. However, all studies evaluating ILPs which were identified by the search were described in the Table of Excluded Studies, regardless of study design.
No study was found that met the review's inclusion criteria, so meta-analysis was not possible in this version of the review. Should relevant studies be identified in the future, study design will be included in the data extraction and explored as a possible source of heterogeneity.
Included studies had to compare an independent living programme to a control group. The control group could be a ‘standard/usual care’, another intervention (e.g. mentoring alone), no intervention, or a waiting list.
Types of participants
Young people leaving the care system at their respective country's statutory ages of discharge from the care system.
Types of interventions
Independent living programmes (as described above), containing the provision of training and/or support in the acquisition of personal development.
Programmes specifically targeted at young people with special needs such as those with physical or learning disabilities, teenage parents, young offenders, and those in psychiatric institutions were excluded.
Types of outcome measures
Studies were only included if they were explicitly targeted at improving at least one of the following:
Educational attainment (example, high school diploma, national vocational diploma, higher education)
Employment (example, full time employment, unemployment rates, income levels) Health status (example, teenage pregnancy/fatherhood rates, drug use, mental health) Housing (example, homeless, own accommodation, or living with family) Life skills including behaviour outcomes (examples: coping skills; financial skills and knowledge; knowledge of state benefits systems; accessing community resources; dependence on public assistance; involvement with the criminal justice system)
No study was found that met the review's inclusion criteria, so meta-analysis was not possible in this version of the review. Should relevant studies be identified in the future, these outcomes will be treated as entirely separate constructs in all analyses. For example, if a study reports on educational attainment and health status, these two outcomes will be entered into separate analyses. If a study reports two separate measures for the same outcome (e.g., percentage experiencing homelessness and percentage living in their own accommodation), each of the outcomes will be analysed separately (e.g., all studies reporting on homelessness will be grouped for one analysis, and all studies reporting on the percentage of participants living in their own accommodation will be grouped in a separate analysis).
Some possible outcomes of ILPs such as housing and employment can be assessed immediately after intervention. Other outcomes such as higher education attainment, health status, holding on to employment and housing, and behaviour outcomes need to be assessed over longer time periods. Should relevant studies be identified in the future, outcomes will therefore be assessed as short term (immediately after intervention) and long-term (12 months after intervention) to determine whether immediate outcomes can be sustained.
The data sources used to assess outcomes included agency records and self reports using psychometrically sound and validated scales of assessment. Should relevant studies be identified in the future, we will investigate the method of outcome assessment as a source of heterogeneity and possible bias. Political influences such as government targets and the high mobility of care leavers may affect the reliability of agency records as a source of outcome measurement.
Search strategy for identification of studies
The following electronic databases were searched:
Cochrane Register of Controlled Trials (CENTRAL) (Issue 3, 2005)
MEDLINE (1966 to June 2005)
EMBASE (1980 to June 2005)
CINAHL (1982 to June 2005)
PsycINFO (1887 to June 2005)
Sociological Abstracts (1952 - June 2005)
Applied Social Science Index and Abstracts (ASSIA) (1987- June 2005)
Dissertation Abstracts (to June 2005)
Further identification of studies was attempted through cross-referencing bibliographies of all relevant studies and reviews discovered in the search. Experts and authors identified by the search were contacted for information on unknown published and unpublished studies, as well as ongoing studies and other suggested contacts. The following journals were hand-searched for relevant articles: Children and Youth Services Review, Research in Social Work Practice. Forward searches were also conducted. The following search terms were used in finding the relevant studies for inclusion in the review. These terms were adjusted as necessary to suit the indices of individual databases.
FOSTER HOME CARE OR
foster* OR
(care home*) OR
(institution* near care*) OR
(social near care) OR
(children* near home*) OR
((child* near home*) near care) OR
(substitute near parent*) OR
(substitute near care) OR
(home near placement*) OR
(residential near care) OR
(child* near care) OR
(home care) OR
(welfare care) OR
AND
ADOLESCENT OR
(child* or adolescen* or youth* or teen*) OR
((young next person) or (young next people))
AND
AFTERCARE OR
Leaving OR
(after* near care) OR
(look* near after*) OR
support* OR
aftercare* OR
(independent living)OR
((independent near live*) or (independent near living))
No language restrictions or geographical restrictions were applied.
Updated versions of this review will incorporate the term “independence training unit” into the search strategy. Additional searches will be run in Social Work Abstracts (which overlaps with the sources covered in the existing strategy), and hand-searching of Child Welfare and Social Work Research will be conducted.
Methods of the review
No study was identified that met the inclusion criteria for this review. The following paragraphs document the methods used for selecting trials and the proposed analytical approach that will be used if relevant studies are identified in future updates.
Selection of trials
Titles and abstracts of studies yielded by the searches were checked by CD and PM independently (i.e., without conferencing) to determine their eligibility for inclusion in the review. If either reviewer considered a study to be potentially relevant, a full copy of the text was obtained by CD. Once retrieved, the studies’ methodological quality and eligibility for the review was assessed by CD and PM independently. Where there was uncertainty or disagreement between the two reviewers regarding the eligibility of a study, this was resolved by discussion. Where discussions were inconclusive, the review's editorial base was contacted to resolve the dispute. To avoid the possibility of investigator bias, effect sizes were not computed or considered until after the eligibility of a study had been established.
Quality assessment
No study was found that met the review's inclusion criteria, so quality assessment was not conducted for this version of the review. Should relevant studies be identified in the future, two authors working independently will critically assess the methodological quality of studies against a set of criteria that considers their degree of allocation concealment, follow up, intention-to-treat, and blinding of assessors. Quality categories will be assigned to each criterion. For example, allocation of concealment will be assessed, as illustrated in the Cochrane Collaboration Handbook ( Indicates adequate allocation concealment; e.g. by telephone randomisation or sealed envelopes. Indicates uncertainty about the adequacy of allocation concealment; e.g. where method of concealment is not reported Indicates allocation was inadequately concealed; e.g. open random number lists or quasi-randomisation such as alternation, day of the week, case number.
Since studies using quasi-randomisation methods (e.g., assignment by coin flip, case record number, date of birth) will be included, evidence of baseline differences and attempts made to control for them will be examined. Evidence of baseline differences will not necessarily lead to exclusion. If a quasi-randomised study does not control for baseline differences, authors will be contacted for additional data regarding the allocation sequence and the possible effects of baseline differences. The review group's editorial base will be contacted where reviewers are uncertain whether to include such studies.
Given the nature of the intervention, it is unlikely that providers and participants in the intervention can be blinded; hence this will not be used as a quality criterion. Should relevant studies be identified in the future, information about blinding will be coded and investigated as a possible source of heterogeneity and bias.
Additional information regarding methodological quality will be sought from primary study authors as necessary. Uncertainty and disagreements will be discussed among the review authors. If no consensus can be reached, disagreements regarding methodological quality will be brought to the review's editorial base.
Data management
No study was found that met the review's inclusion criteria, so meta-analysis was not possible in this version of the review. Should relevant studies be identified in the future, data extraction will be done independently by CD and PM with the aid of a pilot tested extraction form. Differences in coding will be resolved by discussion and referral to the review group's editorial base. Information will be extracted on the following: participants’ characteristics at baseline (including ethnicity, age, geographical location, gender, and pre-care experiences), study design and methods, specific details of the intervention delivered (features and duration), outcomes, outcome measurement (e.g., agency records, self-report), implementation fidelity, cost-effectiveness, and participant satisfaction. The extracted data will be shown in a Table of Included Studies. Information about how effect sizes are extracted from the primary studies will be coded. We plan to calculate effect sizes from means and standard deviations reported in the studies; however, where this is impossible, we will seek statistical guidance from the review's editorial base and code the statistical methods used.
Incomplete data and attrition
Missing data may consist of statistical data (e.g., standard deviations for means), or raw follow-up data for participants who dropped out of a study. No study was found that met the review's inclusion criteria, so meta-analysis was not possible in this version of the review. Should relevant studies be identified in the future, the study authors will be contacted in cases of missing data. Attrition will be explored as a possible source of heterogeneity and bias.
Measures of treatment effect
No study was found that met the review's inclusion criteria, so meta-analysis was not possible in this version of the review. Should relevant studies be identified in the future, any meta-analysis will be conducted according to the following methods. For dichotomous outcome data, log odds ratios with 95% confidence intervals will be calculated. Continuous data will be analysed if means and standard deviations are available and the data are not skewed. For continuous data that must have values greater than 0 (e.g., number of arrests), we will define skewed data as that for which the mean is less than the sum of two standard deviations (
Assessment of heterogeneity
No study was found that met the review's inclusion criteria, so meta-analysis was not possible in this version of the review. Should relevant studies be identified in the future, any meta-analysis will be conducted according to the following methods. Heterogeneity will be assessed using the chi square test of heterogeneity, visual inspection of the graph, and the I2 statistic (
Data syntheses
No study was found that met the review's inclusion criteria, so meta-analysis was not possible in this version of the review. Should relevant studies be identified in the future, any meta-analysis will be conducted according to the following methods. Both fixed effects and random effects models will be considered in conducting the analyses. The random effects model will be used where there is indication of heterogeneity and the source of such heterogeneity cannot be explained. The random effects model will also be used for analyses incorporating small numbers of studies, for which tests of heterogeneity may be underpowered. Where there is no source of heterogeneity beyond differences in the observed covariates, we will conduct both fixed effects and random effects analyses and investigate differences between the two procedures. The value of meta-analysis will be strongly considered if there is substantial clinical or methodological heterogeneity.
Sensitivity analyses
No study was found that met the review's inclusion criteria, so meta-analysis was not possible in this version of the review. Should relevant studies be identified in the future, any meta-analysis will be conducted according to the following methods. Sensitivity analyses will be conducted to assess the impact of the quality of included studies on the outcome of the review. The quality criteria used in the analyses will be the method of allocation concealment and intention-to-treat.
Subgroup analyses
No study was found that met the review's inclusion criteria, so meta-analysis was not possible in this version of the review. Should relevant studies be identified in the future, subgroup analyses will be conducted according to the following methods. Regardless of heterogeneity tests, subgroup analyses will be performed to explore the differential impact of covariates such as gender, ethnicity, and care placement history (i.e., foster care vs. residential care). These covariates are often associated with differential outcomes for young people leaving care (
Boys vs. girls Majority vs. minority ethnicities Foster care vs. residential placement histories
If the literature search suggests that there are strong theoretical reasons to search for moderators, additional subgroup analyses may also be appropriate regardless of heterogeneity tests.
Assessment of bias
No study was found that met the review's inclusion criteria, so meta-analysis was not possible in this version of the review. Should relevant studies be identified in the future, funnel plots (effect size against standard error) will be drawn if a sufficient number of studies are found. Additional analyses to detect bias will include the trim and fill technique (
Description of studies
In all, 2196 citations were identified. After a thorough screening process, 54 articles were retrieved in full for scrutiny, but none met the inclusion criteria for the review. Studies were excluded from the review mainly because they were not randomised or quasi-randomised controlled trials.
Methodological quality of included studies
No study was found that met the inclusion criteria of the review.
Results
No study was found that met the inclusion criteria of the review. The search yielded eighteen studies that used nonrandomised, one-group longitudinal, and qualitative designs to evaluate ILPs. These studies are cited in the table of excluded studies; where the primary references were unavailable after extensive searching and repeated attempts to contact the authors, study data were extracted from secondary sources. Besides a lack of randomisation, these studies faced a number of methodological limitations, such as the use of small sample sizes, the presence of baseline differences, substantial variation in ILP design, inadequate information regarding effect sizes and confidence intervals, and inadequate reporting on implementation fidelity. Collectively, these studies appeared to show that some ILPs may improve educational, employment-related, and housing-related outcomes for young people leaving the care system. The strength of this evidence, however, is insufficient to draw conclusions for policy or practice.
Discussion
Considering that the review intended to assess the effectiveness of ILPs, studies that used a randomised controlled design would have provided the most reliable evidence. Yet, no randomised or quasi-randomised controlled study was found, meaning that no study could be included in this review.
In the absence of randomised or quasi-randomised controlled studies, studies utilising other designs were found. Where control groups were used, these evaluations generally reported better outcomes for participants who took part in an ILP prior to discharge than for controls. This trend was consistent across most of the outcomes of interest to this review, including educational attainment, housing, and employment. However, reliable inferences cannot be drawn from nonrandomised studies due to their use of weak methodology.
Reviewers’ conclusions
Implications for practice
Results from this review show no firm evidence from randomised controlled studies regarding the effectiveness of ILPs in improving outcomes for young people discharged from care. Given the methodological weakness of available studies, it is difficult to make definite conclusions for practice at this time.
Implications for research
There is the need for further research into ILPs using randomised controlled designs. Studies that randomise participants among intervention conditions can investigate questions of effectiveness and harm most thoroughly. It is important to acknowledge that randomisation may be difficult in settings where policies mandate ILP services for care leavers, such as the UK context after the Children (Leaving Care) Act of 2000; however, in settings where randomisation is possible, RCTs will provide the clearest evidence of effectiveness. Future research should also take into account the weaknesses identified in the available evidence and incorporate larger sample sizes, report more details regarding implementation fidelity, and measure outcomes over longer follow up periods. Additional studies or moderator analyses should address the effectiveness of ILPs among care leavers with different care placements, such as family placements or group homes. The theoretical assumptions of ILPs also require investigation, since it is unclear whether (and how) independent living skills can compensate for a relative lack of family support. The results of new studies can then be compared to the current evidence to establish a more accurate appraisal of effectiveness. Without more rigorous research, the evidence base cannot provide reliable answers to practitioners and policymakers regarding the role of independent living programmes for youth leaving care.
Acknowledgements
Jo Abbott (Trial Search Coordinator) from the Cochrane Developmental, Psychosocial and Learning Problems Review Group assisted us in developing the protocol and carrying out the search strategy. Various primary study authors, including Mark Courtney, aided in locating unpublished and ongoing studies. Many thanks to the Danish National Institute of Social Research and to Laila Espersen for her help with Nordic studies and her translation of this review.
Potential conflict of interest
None known.
Characteristics of excluded studies
Characteristics of ongoing studies
Internal sources of support
University of Oxford, UK
External sources of support
Socialforskningsinstituttet/The Danish National Institute of Social Research, DENMARK
This review is co-registered with the Cochrane Collaboration. The Methods section of this review has been revised in accordance with feedback from Campbell Collaboration peer reviewers.
Kristen Underhill joined as a third reviewer in November 2005.
References to studies
References to excluded studies
References to ongoing studies
Other references
Additional references
Contact details for co-reviewers
Mr Charles Donkoh
The Centre for Evidence-Based Social Work
University of Oxford
Barnett House
32 Wellington Square
Oxford
UK
OX1 2ER
Telephone 1: +44 1865 280 325
E-mail:
Ms Kristen Underhill
Centre for Evidence-Based Intervention
Barnett House
32 Wellington Square
Oxford
UK
OX1 2ER
Telephone 1: +44 186 528 4373
E-mail:
