PROTOCOL: Independent living programmes for improving outcomes for young people leaving the care system

The Child Welfare System

It is widely agreed that good parenting consists of providing a safe, secure and stable environment in which children can develop to their full potential ( Smith 2001 ). However, not all parents are able to offer such secure and stable environments for their children, and the state may intervene for these children through the legal system. Sometimes this entails removing children from their parents’ care and placing them in public care, with the state assuming overall responsibility for their upkeep. As “corporate parent,” the state aims to provide for these children's education, health, social interaction, safety, and other needs that are traditionally fulfilled by the family. While in public care, children are usually placed in foster care or residential care.

Prevalence and Experiences of Children in Public Care

Each year a large number of children enter public care systems around the world. There were 523,000 children in public care in the United States in 2003 ( CTD 2005 ), 61,100 children in public care in England in 2004 ( DfES 2004 ), over 4,500 children in public care in Wales in 2004 ( NAW 2005 ), over 12,000 children in public care in Scotland in 2005 ( SENS 2005 ), and 21,735 children in out-of-home care in Australia in 2004 ( AIHW 2005 ). In the United States, public (foster) care placement types include homes of nonfamily members, homes of family members, group homes or institutions, pre-adoptive homes, and “other” placements: in 2003, the distribution of youth among these placements was 46%, 23%, 19%, 5%, and 7% respectively (NCCANI 2005). In England, public care placement types include foster family assignments, children homes, placement with parents, adoption, and “other”: in 2005, the distribution of youth among these placements was 68%, 11%, 9%, 5%, and 5% respectively (DfES 2005).

Children in public care systems come from diverse backgrounds with different cultures, ethnicities, needs, abilities and pre-care histories ( Biehal 1995 ). Most enter public care because they have suffered or are likely to have suffered maltreatment ( DOH 1991 ); they are also more likely to come from multiply disadvantaged backgrounds ( Bebbington 1989 ). Once children are in public care, their deleterious experiences do not necessarily end; many have to deal with the long-term effects of the abuse and neglect they experienced before entering care ( Meltzer 2003 ), and some may have further negative experiences whilst in public care. Children in public care are more likely to be diagnosed with emotional or behavioural problems ( Meltzer 2003 ), to experience unstable care placements ( Biehal 1992 ), and to face accompanying difficulties at school ( Jackson 1998 , OFSTED 1995 ) than children living in private households. A 2003 survey by England's Office for National Statistics reported that 45 percent of children aged 5-17 years in public care had been diagnosed with a mental disorder ( Meltzer 2003 ). Several studies have also catalogued the poor educational attainment of young people living in and leaving the care system ( Barth 1990 , Cheung 1994 , Courtney 1998 , Festinger 1983 , Garnett 1992 , Jackson 1994 ). The difficulties that these children experience at school not only lessen their educational attainments but, more importantly, also deprive them of potentially protective factors necessary to counter the adverse effects of unpleasant experiences during childhood ( Rutter 1990 ).

Young People Leaving Care

Every year about 20,000 American and 6,000-8,000 English young people leave their respective public care systems ( CLA 2006 , DfES 2002 , DfES 2005 , USGAO 1999 ). A significant number of young people leaving public care are disadvantaged and ill-prepared for adult life ( Barth 1990 , Cheung 1994 , Courtney 1998 , Festinger 1983 , Garnett 1992 ). The families of young people leaving care are often unable to offer the sustained and substantial support that would benefit the youths’ transition from adolescence to adulthood. As a result, many young people leave care with little social, emotional or financial support from their families as compared to their peers in the general population; they also typically make the transition to independence earlier ( Cashmore 1996 , Courtney 1996 , Morrow 1996 ). Notably, approximately 50% of the youth leaving care annually in England are aged 16-17 years, while youth in the general population are estimated to leave home at age 22 ( DOH 1999 ). However, the age of young people leaving care varies worldwide.

Studies indicate that a significant proportion of young people leaving care do not possess the life skills or resources necessary to succeed independently. Upon leaving care they are more likely than youth in the general population to be homeless, unemployed, and/or dependent on public assistance; they are also more likely to experience physical and mental health problems, engage in risky health behaviours, and become involved with the criminal justice system ( Barth 1990 , Cook 1994 , Courtney 2001 , Courtney 2005 , Festinger 1983 , Fowler 1996 , Maunders 1999 ).

In recognition of the difficulties facing young people leaving care, policies have been enacted to help prepare them for adulthood. These include the John H. Chafee Foster Care Independence Program of 1999 in the US ( NRCYD 2004 ; Barth 2004 ) and the Children (Leaving Care) Act of 2000 in the UK ( DOH 2001 ).

Independent Living Programmes

Independent living programmes (ILPs) are designed to provide young people leaving care with skills that will limit their disadvantage and aid in their successful transition into adulthood. ILPs recognize that leaving care is a process, not an event, and that it requires social support and life skills preparation. These programmes are not intended to replace the supportive role played by a family, but instead aim to provide care leavers with skills that will help them succeed despite the absence of family support. In the main, ILPs utilise social skills training techniques, which incorporate instruction, modelling, roleplays and feedback. These training techniques have been used effectively to teach skills acquisition and improve youth performance in both clinical and non-clinical settings ( Spence 1995 ).

ILPs focus on both personal development skills and independent living skills. Personal development skills may include communication, decision making, conflict resolution, and anger management. Independent living skills include career exploration, job and interview skills, money management, household management, accessing housing, seeking legal assistance, and utilising community resources ( Cook 1994 , USGAO 1999 ). ILPs can also provide educational and vocational support. Some authors have advocated that ILPs include interpersonal and relationship training as well ( Courtney 1996 , Propp 2003 ), although these skills are not consistently incorporated into existing programmes.

ILPs are frequently conducted in group formats with individual support (i.e., mentoring) provided on a one-to-one basis ( Biehal 1995 , Meston 1988 ). Many ILPs provide supervised living conditions under which young people can practise the skills they have learnt ( Mauzerall 1983 ), and they occur in diverse settings such as community centres, group homes, transition placements, and supervised practice placements ( Biehal 1995 , Meston 1988 ). ILPs may also be delivered to young people living in independent tenancies. The content, setting, and delivery of ILPs may vary depending on a country's culture, legislation, or policy context, as well as the age at which youth leave care.

Notwithstanding the wide use of independent living programmes, their effectiveness is unknown ( USGAO 1999 ) and the extent to which the acquisition of independent living skills by young people leaving care is associated with easier transition to independent and self sufficient living remains uncertain. Some evidence suggests that such programmes may be successful in improving outcomes such as education, employment, housing, health and life skills for young people leaving care ( Loman 2000 , Mallon 1998 , Scannapieco 1995 , Biehal 1995 ), but this evidence is based on narrative reviews, non-systematic searches, and non-experimental studies.

This review aims systematically to determine the effectiveness of these independent living programmes in increasing the life chances of young people leaving care. Knowing the effectiveness of such programmes is important given the numerous challenges associated with living in and leaving public care.

Types of studies

Randomised and quasi-randomised controlled studies (i.e. where allocation is by date of birth, alternate numbers, case number, day of the week, or month of the year) will be eligible for inclusion. Since this review aims to synthesise the evidence from study designs least prone to bias, quasi-experimental studies will not be included. However, all studies evaluating ILPs which are identified by the search will be described in the Table of Excluded Studies, regardless of study design.

For randomised and quasi-randomised studies that meet all inclusion criteria, study design will be included in the data extraction and explored as a possible source of heterogeneity.

Included studies will compare an independent living programme to a control group. The control group can be a ‘standard/usual care’, another intervention (e.g. mentoring alone), no intervention, or a waiting list.

Types of participants

Young people leaving the care system at their respective country's statutory ages of discharge from the care system.

Types of interventions

Independent living programmes (as described above), containing the provision of training and/or support in the acquisition of personal development.

Programmes specifically targeted at young people with special needs such as those with physical or learning disabilities, teenage parents, young offenders, and those in psychiatric institutions will be excluded.

Types of outcome measures

Studies will only be included if they are explicitly targeted at improving at least one of the following:

Educational attainment (example, high school diploma, national vocational diploma, higher education)

Employment (example, full time employment, unemployment rates, income levels) Health status (example, teenage pregnancy/fatherhood rates, drug use, mental health) Housing (example, homeless, own accommodation, or living with family)

Life skills including behaviour outcomes (examples: coping skills; financial skills and knowledge; knowledge of state benefits systems; accessing community resources; dependence on public assistance; involvement with the criminal justice system)

These outcomes will be treated as entirely separate constructs in all analyses. For example, if a study reports on educational attainment and health status, these two outcomes will be entered into separate analyses. If a study reports two separate measures for the same outcome (e.g., percentage experiencing homelessness and percentage living in their own accommodation), each of the outcomes will be analysed separately (e.g., all studies reporting on homelessness will be grouped for one analysis, and all studies reporting on the percentage of participants living in their own accommodation will be grouped in a separate analysis).

Some possible outcomes of ILPs such as housing and employment can be assessed immediately after intervention. Other outcomes such as higher education attainment, health status, holding on to employment and housing, and behaviour outcomes need to be assessed over longer time periods. Outcomes will therefore be assessed as short term (immediately after intervention) and long-term (12 months after intervention) to determine whether immediate outcomes can be sustained.

The data sources used to assess outcomes will include agency records and self reports using psychometrically sound and validated scales of assessment. We will investigate the method of outcome assessment as a source of heterogeneity and possible bias. Political influences such as government targets and the high mobility of care leavers may affect the reliability of agency records as a source of outcome measurement.

Selection of trials

Titles and abstracts of studies yielded by the searches will be checked by CD and PM independently (i.e., without conferencing) to determine their eligibility for inclusion in the review. If either reviewer considers a study to be potentially relevant, a full copy of the text will be obtained by CD. Once retrieved, the studies’ methodological quality and eligibility for the review will be assessed by CD and PM independently. Where there is uncertainty or disagreement between the two reviewers regarding the eligibility of a study, this will be resolved by discussion. Where discussions are inconclusive, the review's editorial base will be contacted to resolve the dispute. To avoid the possibility of investigator bias, effect sizes will not be computed or considered until after the eligibility of a study has been established.

Quality assessment

CD and PM will critically assess the methodological quality of studies against a set of criteria that considers their degree of allocation concealment, follow up, intention-to-treat, and blinding of assessors. Quality categories will be assigned to each criterion. For example, allocation of concealment will be assessed, as illustrated in the Cochrane Collaboration Handbook (Higgins 2005) as follows: (A)

Indicates adequate allocation concealment; e.g. by telephone randomisation or sealed envelopes.

Since studies using quasi-randomisation methods (e.g., assignment by coin flip, case record number, date of birth) will be included, evidence of baseline differences and attempts made to control for them will be examined. Evidence of baseline differences will not necessarily lead to exclusion. If a quasi-randomised study does not control for baseline differences, authors will be contacted for additional data regarding the allocation sequence and the possible effects of baseline differences. The review group's editorial base will be contacted where reviewers are uncertain whether to include such studies.

Given the nature of the intervention, it is unlikely that providers and participants in the intervention can be blinded; hence this will not be used as a quality criterion. Information about blinding will be coded and investigated as a possible source of heterogeneity and bias.

Additional information regarding methodological quality will be sought from authors as necessary. Uncertainty and disagreements will be discussed between the authors. If no consensus can be reached, disagreements regarding methodological quality will be brought to the review's editorial base.

Data management

Data extraction will be done independently by both authors with the aid of a pilot tested extraction form. Differences in coding will be resolved by discussion and referral to the review group's editorial base. Information will be extracted on the following: participants’ characteristics at baseline (including ethnicity, age, geographical location, gender, and pre-care experiences), study design and methods, specific details of the intervention delivered (features and duration), outcomes, outcome measurement (e.g., agency records, self-report), implementation fidelity, cost-effectiveness, and participant satisfaction. The extracted data will be shown in a Table of Included Studies. Information about how effect sizes are extracted from the primary studies will be coded. We plan to calculate effect sizes from means and standard deviations reported in the studies; however, where this is impossible, we will seek statistical guidance from the review's editorial base and code the statistical methods used.

Incomplete data and attrition

Missing data may consist of statistical data (e.g., standard deviations for means), or raw follow-up data for participants who dropped out of a study. In cases where data are missing the study authors will be contacted. Attrition will be explored as a possible source of heterogeneity and bias.

Measures of treatment effect

For dichotomous outcome data, log odds ratios with 95% confidence intervals will be calculated. Continuous data will be analysed if means and standard deviations are available and the data are not skewed. For continuous data that must have values greater than 0 (e.g., number of arrests), we will define skewed data as that for which the mean is less than the sum of two standard deviations ( Altman 1996 , Alderson 2005 ). Where they are reported in the primary studies, we will also inspect histograms, scatterplots, and summary statistics for evidence of skew. If any test or inspection suggests that data are likely to be skewed, authors will be contacted for more information, log transformed data, or the raw data. Where the same outcomes are measured in different ways, standardised mean differences will be calculated and compared across studies. Where outcomes are measured in the same way, weighted mean differences will be calculated.

Assessment of heterogeneity

Heterogeneity will be assessed using the chi square test of heterogeneity, visual inspection of the graph, and the I2 statistic ( Higgins 2002 ). The I2 statistic will determine the percentage of variability that is due to heterogeneity rather than sampling error, where a value greater than 50% suggests moderate heterogeneity. If any of these methods indicates heterogeneity, we will investigate possible explanations, including clinical and methodological characteristics. Even when tests for heterogeneity are non-significant, we plan to conduct subgroup analyses and explore other potential moderators.

Data syntheses

Both fixed effects and random effects models will be considered in conducting the analyses. The random effects model will be used where there is indication of heterogeneity and the source of such heterogeneity cannot be explained. The random effects model will also be used for analyses incorporating small numbers of studies, for which tests of heterogeneity may be underpowered. Where there is no source of heterogeneity beyond differences in the observed covariates, we will conduct both fixed effects and random effects analyses and investigate differences between the two procedures. The value of meta-analysis will be strongly considered if there is substantial clinical or methodological heterogeneity.

Sensitivity analyses

Sensitivity analyses will be conducted to assess the impact of the quality of included studies on the outcome of the review. The quality criteria used in the analyses will be the method of allocation concealment and intention-to-treat.

Subgroup analyses

Outcomes of ILPs may vary depending on covariates such as gender, ethnicity, and care placement history (i.e. foster care vs. residential care). Regardless of heterogeneity tests, subgroup analyses will be performed to explore the differential impact of the above covariates, which are often associated with differential outcomes for young people leaving care (Barn 2005 , Biehal 1995 , Courtney 2005 ).

o

Boys vs. girls

If the literature search suggests that there are strong theoretical reasons to search for moderators, additional subgroup analyses may also be appropriate regardless of heterogeneity tests.

Assessment of bias

Funnel plots (effect size against standard error) will be drawn if a sufficient number of studies are found. Additional analyses to detect bias will include the trim and fill technique ( Duval 2000 ) and the planned Egger regression approach with a weight-function model. Asymmetry can be due to publication bias, but it can also be due to clinical and methodological heterogeneity. In the event that a relationship is found, these sources of heterogeneity will also be examined as possible explanations ( Egger 1997 ).