Treatment Foster Care for Improving Outcomes in Children and Young People

A. Looked-after child outcomes:

Behavioural outcomes, covering: i. behavioural problems within the treatment foster home using measures of externalising behaviour, including carers’ report (i.e., physical and verbal aggression, self-harming, rule-breaking behaviour, defiance, truancy); ii. anti-social behaviour as measured by rates of delinquency, rates of arrest, rates of conviction, and rates of incarceration, iii. drug and substance abuse (based on self reports and biological measures); iv. prescribed medication for behavioural symptom reduction/management (as indicated by agency records and carers reports);

Note: Interpreting the use of medication, particularly in the field of mental ill health, is complex. First, where associated with maintenance or improvement of health then starting or maintaining a pharmacotherapeutic regimen could be a good outcome (e.g. in cases with schizophrenia). In some cases, however, medication might mean that participants are not receiving the psychological therapy that would be more beneficia/have less harmful side effects. Secondly, TFC foster carers may be more likely to advocate for psychiatric assessment and treatment, on the basis of children's needs, or because it is seen as an answer to difficult child management problems. Finally, it could be indicative of harm if the need for medication precipitates stress and associated mental health problems. Given the challenges of disinterring these processes we have said we will report these outcomes in the review and only synthesise and/or interpret them if the meaning of the data are unambiguous i.e. clearly beneficial, clearly indicative of appropriate advocacy, clearly an unintended harm. No relevant data were reported in the included studies in this review.

B. Treatment foster carer(s)/family outcomes:

Measures of skills (e.g., behaviour management skills, problem solving skills), knowledge and attitudes as measured by agency personnel and/or standardised scales;

C. TFC agency outcomes:

Placement stability (e.g., number of requests for removal, number of unrequested removals) and/or completion of allocated stay;

D. Costs

It is generally estimated that TFC is less expensive than residential care, although there is wide variation in costs among programmes (Curtis 2001) and more expensive than regular foster care. Where possible, data on programme costs and cost-benefit analyses will be reported.

Where possible, outcome data are reported for the short-term (up to one year post-intervention) and long-term (over one year post-intervention).

Primary outcomes are emboldened.

1. Selection of studies

Both authors (WT and GM) independently selected studies for inclusion in the review. In two cases there was initial disagreement between the authors but these were resolved through discussion (in relation to Clark 1994 and Evans 1994). See Figure 01.

2. Data extraction and management

Data were independently extracted by the authors and recorded on proforma data extraction sheets (available from the authors). There were no disagreements. All decisions were documented and, where necessary, study investigators were contacted to assist in resolving problems and secure missing data. In relation to Chamberlain 1998 the investigators provided some unpublished raw data. In relation to both this study, Clark 1994 and Chamberlain 1991 the investigators were approached for additional data but were unable to supply this due to the age of studies.

As per the protocol, data were extracted on the following where available: study design and implementation (including data on programme differentiation [Dane 1998; MRC 2000]), sample characteristics, intervention characteristics (including theoretical underpinning of services, delivery, duration, and within-intervention variability). Outcome data as listed above were also extracted. Data are summarised in a series of tables and - where appropriate - in the body of the review.

3. Assessment of methodological quality of included studies

To assess concealment of allocation to intervention groups, the authors independently assigned each selected study to the following categories described in the Cochrane Handbook (Higgins 2005):

(A) indicates adequate concealment of the allocation (for example, by telephone randomisation, or use of consecutively numbered, sealed, opaque envelopes);

(B) indicates uncertainty about whether the allocation was adequately concealed (for example, where the method of concealment is not known);

(C) indicates that the allocation was definitely not adequately concealed (for example, open random number lists or quasi-randomisation such as alternate days, odd/even date of birth, or hospital number).

The following items were also assessed as potential sources of bias: blinding of outcome assessors (where relevant); extent of drop-out or attrition. We also identified those studies that used Intention to Treat analysis. Our assessment of the methodological quality of included studies can be found in Additional Table 03 .

4. Missing data

The authors wrote to study investigators in an effort to obtain missing data. Both Chamberlain (Chamberlain 1991; Chamberlain 1992; Chamberlain 1998; Leve 2005) and Clarke (Clark 1994, Clark 2007) responded helpfully to these requests but it was only in relation to Chamberlain's recent work (Leve 2005) were able to provide data. In other cases the data were no longer available. Data were not ‘missing’ in the sense of respondents being lost to follow-up, but rather that the published analyses did not provide means and standard deviations, or the information to enable us to calculate treatment effects. See Additional Table 04 for details of how we will manage missing data due to drop out or other forms of attrition in future versions of the review.

5. Data synthesis

Random effects meta-analyses were performed using RevMan 4.3 and are illustrated within the review using standard forest plots.

Binary data. Of the studies included in this review, only Clark 1998 provided binary data but it was not suitable for meta-analysis (see below for reporting of odds ratios in the published paper). Should binary data be reported in studies identified in future updates of this review, these will be analysed as described in Table 02, ‘Additional methods for future updates.’

Continuous data. Continuous data were analysed where (i) means and standard deviations were available or calculable and ii) there was no clear evidence of skew in the distribution (mean less than 1 standard deviation from an upper or lower bound). Where means or standard deviations were not given directly effect sizes were calculated using test statistics wherever possible. Where scales were measuring the same clinical outcomes in different ways, mean differences were standardised in order to combine results across scales (Higgins 2005).

Where an outcome was assessed using a measure with sub-scales it was our intention to use the total score where appropriate, providing the full scale addressed the outcome of interest. Otherwise we would used the most relevant subscore. In this present version of the review this issue did not arise in relation to the rare occasions when it was possible to combine the results of studies. In the reporting of results from individual studies we have used the sub-scales of the CBCL where reported, alongside the total score for this measure when used.

6. Types of analyses.

All studies included in this version of the review allocated participants individually to experimental and control groups.

Data were not available for this version of the review to consider the impact of per protocol versus intention-to-treat analysis. See Table 02 .

7. Assessment of heterogeneity

The assumption that the studies were estimating a common treatment effect was assessed using the standard test for heterogeneity and the statistic I² (Higgins 2002, Higgins 2003).

8. Subgroup analyses and investigation of heterogeneity

Subgroup analyses were not undertaken because the small number of included studies provided insufficient power reliably to identify correlated predictors (see also Additional Table 02 for information about future updates of this review.

9. Assessment of bias and sensitivity analyses

Analyses were based on available data from all included studies relevant to the comparison and outcome of interest. Planned sensitivity analyses designed to examine the impact on results of the quality of data and approach to analysis were not conducted due to the paucity of data. See Additional Table 02 for further information.

10 and 11. Use of qualitative research and data on programme costs

Plans were made to draw on available qualitative data within studies to discuss programme processes and implementation issues and to summarize available data on the costs of programs within the studies under review: no such data were reported in the included studies.

Results of the search

Searches were run in January 2007. After removing duplicates a total of 634 citations were identified. Of these 31 citations appeared potentially relevant and full texts were retrieved and read. See Additional Figure 01.

Included studies

Five studies (ten citations) met the inclusion criteria (Chamberlain 1991; Chamberlain 1992; Chamberlain 1998; Clark 1994; Leve 2005). One study (two citations) is awaiting further information from the authors (Evans 1994).

Location

All included studies were conducted in the USA, four by investigators working at the Oregon Social Learning Centre (Chamberlain 1991; Chamberlain 1992; Chamberlain 1998; Leve 2005) and one in Florida (Clark 1994).

Design

All included studies were randomised controlled trials. Chamberlain 1992 randomised foster carers rather than children, but the children for whom data are reported were those living with the foster carer at the time of randomisation. All other studies randomised individual children. Clark 1994 randomised more participants to the control group in order to compensate for an anticipated higher rate of attrition amongst the control group, for example from extended runaways and voluntary dropouts. All but one study allocated participants to one of two groups, the experimental (TFC) arm and a control group that was usual foster care practice. One study allocated participants to one of three groups, two of which were experimental and comprised i) an augmented monthly allowance, and ii) an augmented monthly allowance plus training and support (Chamberlain 1992). The third group was regular foster care.

Sample sizes

Samples sizes ranged from 20 (Chamberlain 1991) to 132 (Clark 1994), the remaining three comprising 72, 85 and 81 (Chamberlain 1992; Chamberlain 1998 and Leve 2005 respectively).

Participants

Three studies focused on children and young people with emotional and behavioural problems (Chamberlain 1991; Chamberlain 1992; Clark 1994). Of these, one study focused on children & youths aged 9-18 who were in psychiatric hospital with severe emotional disorders who required out-of-home placements (Chamberlain 1991). The other two were focused on children who were in out-of-home placement for reasons of abuse or neglect, and who either had emotional or behavioural problems or were deemed at risk of developing these. The remaining studies focused on delinquent children, one on boys aged 12-17 with histories of chronic delinquency (Chamberlain 1998) and the other on chronically delinquent girls aged 13-17 (Leve 2005).

One study provided no information on the ethnicity of child participants (Chamberlain 1991). In the remaining studies the vast majority of participants were Caucasian, with percentages ranging from 62% in Clark 1994 and 74% in Leve 2005 to 85% and 86% in Chamberlain 1998 and Chamberlain 1992 respectively. See Table 01 for a profile of the ethnicity of participants in each study, where given.

Interventions

Three studies reported a multi-dimensional intervention (Chamberlain 1991; Chamberlain 1998; Leve 2005). In each case the TFC intervention included i) pre-service training of specially selected foster carers; ii) the implementation of a behaviour management programme for the child in both the home and the community; iii) an individualised treatment plan for the children and young people; iv) supervision and support of foster carers via daily telephone calls and weekly groups, and v) services were coordinated by case managers were on call 24 hours per day, seven days per week and available to biological parents as well as foster carers, where appropriate.

Of these three studies, two dealt with delinquents (Chamberlain 1998; Leve 2005). In these studies, the basic framework outlined above was further developed to include: vi) additional meetings with foster carers as and young people to deal with routine problems of difficulties in implementing the programme; vii) weekly family therapy for the biological family or other aftercare resource, focussed on parent management training, including an emphasis on supervision, encouragement, discipline and problem solving;; viii) enrollment of young people in schools underpinned by carefully planned and implemented behaviour management arrangements with backup from the programme; ix) weekly individual therapy with young people which focused on skills building in problem-solving, social perspective taking and nonaggressive methods of self-expression; x) psychiatric consultation, evaluation and medication as needed; xi) close supervision of the young peoples whereabouts and activities; xii) prohibition of contact with peers with known histories of delinquency and other peer associations closely monitored; xiii) individually tailored consequences of rule infringements that were consistently delivered. Young people were encouraged to accept consequences and start ‘each day with a clean slate’; xiv) all parole violations were reported and xv) where the plan was to return the child home, frequent visits were arranged beginning with one to two hours and increasing to overnight stays as the young people progressed through the programme. In the study with delinquent girls (Leve 2005) the investigators describe the intervention as incorporating ‘gender specific processes’ with foster carers providing additional reinforcement and response cost for girls’ avoidance of / commitment of social-relational aggression. Girls were taught skills to avoid this particular form of aggression.

Interventions in the remaining two studies were less intensive in nature, with fewer components and focused on abused/neglected children and youth. Chamberlain 1992 provided foster carers with weekly support via a two-hour meeting with other foster carers, led by a facilitator who had been a foster carer and who was trained in child behaviour management methods and group process. Foster carers were also telephoned three times a week by the group leader to review their report of the child's progress and recent problems, and to offer suggestions and support. Clark 1994 provided an interventions called the Fostering Individualised Assistance Programme (FIAP) in which standard fostering services were enhanced by i) a strengths-based approach to assessment covering 10 life domains; (ii) life domain planning designed to address the priority needs of children and the family; iii) clinical case management through which Family Specialists provided intensive, individualised services and support in the context of the child's home and community settings, to the extent possible and; iv) follow-up services designed to secure the maintenance of change and further progress.

Summary overview.

All of the included studies were conducted by programme developers. Studies examined the effectiveness of the following forms of treatment foster care with the following groups of children and youth:

multi-component TFC with delinquent boys (Chamberlain 1998)

Outcomes

In this section we profile the outcomes used in the included studies. A detailed account of these can be found in Additional Table 04 and in the Results Section below.

A. CHILDREN

1. Child Behaviour

Three measures of child behaviour were used in the included studies.

In Chamberlain 1991 the Parent Daily Report Checklist (PDR) (Chamberlain 1987) was used to measure the occurrence of problem symptoms during the prior 24-hour period. It was administered by phone on 10 separate days to ward staff (at baseline) and then to foster parents, ward staff, or other primary caretaker 3 and 7 months later. In Chamberlain 1992 it was used to assess the extent of problem behaviours within the foster home on five consecutive weekdays at baseline and again 3 months later to determine any changes over time in the foster parents’ reports of child problems. Chamberlain 1998 used a modified version of the PDR to collect information on foster care supervision and young people's association with deviant peers.

The Child Behavior Checklist (CBCL) and the Youth Self Report (YSR) are measures for describing the behaviour of children from the perspective of caregivers and youth respectively Achenbach 1991a; Achenbach 1991b). The instruments provide a total problem score, eight narrow-band factors (i.e., withdrawn, somatic complaints, anxious/depressed, social problems, thought problems, attention problems, delinquent behaviour, aggressive behaviour) and two-broad-band factors (i.e., internalising & externalising). The internalising score is based on the narrow-band factors of withdrawn, somatic complaints, and anxious/depressed, and the externalising score is based on the narrow-band factors of delinquent and aggressive behaviours. Two studies Clark 1994 and Leve 2005 used the Child Behavior Checklist (CBCL) (Achenbach 1991a). Clark 1994 also used the Youth Self Report (YSR) of the CBCL (YSR) (Achenbach 1991b).

2. Antisocial behaviour

Chamberlain 1998 and Leve 2005 also used variously all (Chamberlain 1998) of the subscales of the Elliot Behavior Checklist, a self-report measure of offending (Elliot 1983) or one of its subscales, the Delinquency Subscale (Leve 2005). In Chamberlain 1998 participants were asked how many times they had engaged in any of the criminal behaviours during a specific time frame. The General Delinquency subscale was chosen as an additional measure of overall criminal activity and the other two (Index Offenses and Felony assaults) to measure the rates of more serious and person crimes.

In addition, three studies reported data on criminal referrals and offenses (Chamberlain 1998; Clark 1994; Leve 2005).

The Diagnostic Interview Scale for Children (DISC) (Fisher 1991) is a structured interview that contains items reflecting the presence of current and past symptoms, emotions and behaviours that correspond to DSM III-R criteria. One section of the DISC is the conduct disorder interview protocol, which yields a conduct disorder diagnosis. The conduct disorder instrument was added to the interview protocol used by Clark 1994 for Waves 6, 7, and 8 of the study. The subscale scores from these waves were averaged to provide a single conduct disorder score for each young person.

Clark 1994 also calculated a Total Delinquency score. This composite score was derived from the dependent measures of the delinquency items on the CBCL and YSR, the conduct disorder score, and the official delinquency score. A total score was computed for each of the 131 participants. This score was calculated by converting each of the scale scores available for each particular individual into Z scores and averaging them to establish a total delinquency score.

3. Emotional disturbance

Chamberlain 1991 used the Child Global Assessment Scale (CGAS) (Shaffer 1983) to assess the severity of the emotional disturbance of the psychiatrically hospitalised girls in this study. The CGAS measures respondents’ level of functioning during the previous month. No other included study reported on this outcome. The study also used the Behavior Symptom Inventory (BSI), a 53-item self-report inventory in which children are asked to rate their levels of symptoms and distress (Derogatis 1982).

4. Social Functioning

Chamberlain 1991 also reported on the social competence of the girls in this study using two age-specific versions of the Social Interaction Task: the Adolescent Problem Inventory for youth aged 12 years and above (Gaffney 1981) and the Taxonomy of Problematic Social Situations for youth under 12 years (Dodge 1985). No other included study reported on this outcome.

5. Educational Outcomes

Only one study (Leve 2005) measured homework completion. Leve 2005 and Chamberlain 1998 both measured school attendance.

6. Health

No study reported outcomes for mental or physical health or psychological outcomes other than those reported above.

B. FOSTER CARER OUTCOMES

Chamberlain 1992 monitored foster carers’ skills at discipline via a Staff Impressions Measure completed by study staff members. These staff also rated the foster mothers’ (sic) levels of personal strength and their levels of social skill. They also surveyed foster carers’ perceptions of the effectiveness of the weekly training and support groups and foster carer drop-out. No other included study reported on foster carer outcomes.

C. AGENCY OUTCOMES

All studies reported outcome data in relation to placements, but all measured this outcome in different ways, including for example, institutionalisation rates, number of days living at home, and days ‘on the run’. Please see Additional Table 04 for an overview.

D. COSTS

Only one of the studies reported data on programme costs (Chamberlain 1991).

Excluded studies

Of the eight excluded studies, five were excluded because there were not randomised or quasi-randomised controlled trials (Boyd 1997; Fisher 2000a; Larson 1978; Moore 1994; Rubenstein 1978), two because their focus was not treatment foster care (Barbell 1996; Faulkner 2001). One was ‘a downward extension of treatment foster care’ used with younger children in foster care as a preventive measure for developmental delay (Fisher 2005) - see ‘Close calls’ below.

Close calls

This intervention has been developing over a number of years, during which time it has changed in title and potential content. One study (Chamberlain 1992) was described by the authors as a ‘“light” version of MTFC that could be used universally with all foster parents providing regular foster care’ (Chamberlain, personal communication). The profile of children included in the study suggested they had histories of abuse and neglect and presented with a range of behavioural problems. The intervention was not described as treatment foster care and the study was primarily designed to examine the effects of the intervention on foster carer retention. However, given the profile of the children and the categorisation of the intervention by the authors we have included it. Fisher 2005 was primarily concerned with developing an evidence base to address the merits of early intervention in the lives of pre-school foster children who, research indicates, are at increased risk of developmental delay. The intervention is described as ‘oriented toward creating optimal environmental conditions to facilitate developmental progress’ (Fisher 2005 p. 62). It also examines whether - as an early intervention -MTFC mitigated risks associated with placement history. The study uses permanent placements and placement failure as outcome measures in view of their likely impact on psychosocial outcomes. We were undecided as to whether to include this study. We decided against including it as the children did not directly meet the inclusion criterion of children placed in out of home care ‘for reasons of severe medical, social, psychological and behavioural problems’ or at risk of developing these problems. Rather, the authors describe their approach as addressing ‘the challenges of foster preschoolers are viewed from the perspective of delayed maturation, rather than as strictly behavioral and emotional problems. Consequently, it is oriented toward creating optimal environmental conditions to facilitate developmental progress’ (Fisher 2005 page 63).

In both cases we would welcome the views of others as to the appropriateness of these decisions.

Awaiting assessment

We have written to the authors of Evans 1994 for further information about the intervention.

Ongoing Studies

The intervention described in Chamberlain 1992 is currently being replicated with a large sample of foster carers in San Diego California, but data are not currently available (personal communication Chamberlain 2007b). In addition three trials are currently ongoing: one is a multi-site trial being undertaken in the UK (Biehal & Green 2004). The other two are Swedish trials that have yet to publish its findings (Hansson 2007; Hansson 2008).

Reporting of quality aspects

Quality aspects were generally poorly reported. Full details of our assessments are available in a quality assessment table (Table 03).

Method of sequence generation and allocation concealment

Clark 1994 reported method of sequence generation but not allocation concealment. In four studies (Chamberlain 1991; Chamberlain 1992; Chamberlain 1998 and Leve 2005) neither method of sequence generation nor allocation concealment were reported in published papers, however one investigator (Chamberlain 2007c; Chamberlain 2007d) informed us that coinflipping was the method used in each of these studies and was undertaken in a private office location independent of the centers where treatment was delivered. Sequence generation and allocation concealment are therefore both rated as adequate for these studies.

Outcome blinding

Many of the data on placement outcomes and, where relevant, offending, were gathered via agency records, although some were gathered through the programme staff. Leve 2005 states clearly that the staff members responsible for data collection and data entry were blind to participants’ group assignment and were not involved in delivering the intervention (Chamberlain 2007a, p. 188-189). In two studies the investigators report some limited blinding of outcome assessors to treatment condition as a reliability check: Chamberlain 1991 report that an ‘independent observer who was blind to group assignment and phase’ independently rated videotapes of the social interaction tasks, and their score compared with that of the interviewer who administered the task. The result was an inter-rater reliability of .70 at baseline and .74 at seven months follow-up (page 275). In Chamberlain 1998 the investigators reported that ‘In 10% of the cases, criminal referrals were recoded after having removed all identifying information’, resulting in a 90% level of reliability, entry by entry (page 626). Two studies do not report on this dimension of quality (Chamberlain 1992; Clark 1994).

Attrition

Attrition rates were generally very low or non-existent, depending on the study or outcome measure. Leve 2005 provides an exemplary flow-chart tracking attrition throughout the study, in line with the CONSORT statement (Moher 1999), a rare feature of reports in the field of child and adolescent mental health. Two studies reported undertaking Intention to Treat analyses (Chamberlain 1998) or analyses designed to handle missing data (latent variable analysis of covariance - Leve 2005).

Outcome reporting

Most studies appear to have reported on all the measures they set out to use (as far as one can tell from published reports), at all time points. In Leve 2005 the investigators appear to have dropped the carers’ reported Child Behavior Checklist (Achenbach 1991a) at the two year follow-up but this may be an artefact of the report (i.e., it focused on the long-term maintenance of the effects of multi-dimensional treatment foster care on delinquency) and therefore may have been uncollected rather than unreported. Largely because of the focus of the published papers, data are not always presented in ways that support secondary analysis. For early studies the authors are unable to supply raw data, though for later studies this has been possible in some cases e.g. Leve 2005.

A. CHILDREN

B. Treatment foster carer(s)/family outcomes

B1). Measures of skills, knowledge and attitudes

In Chamberlain 1992, foster carers in the ‘Enhanced support and training’ (ES & T) group gave positive feedback about the weekly group meetings. They reported that the group had helped them deal effectively with their foster child's problems and they would recommend the group to other foster carers. The triallists report that most foster carers rated the groups at 6.5 on a seven-point scale (i.e., between ‘very informative’ and ‘extremely informative’). Additionally, caseworkers’ (subjective) impressions of the ability of this group of foster carers to manage children's behaviour problems were favourable.

B2). Interpersonal functioning

No study set out to examine this as an outcome.

C. TFC agency outcomes

Additional tables Table 10 ; Table 11 provide overviews of data relating to placement stability and other placement outcomes respectively.

D. COSTS

None of the included studies reported cost-benefit data, though one study reported costs per young person of each condition (Chamberlain 1991). These were $3,000 per month for young people in the TFC condition and $6,000 per month for those in the State Mental Hospital. The investigators estimated that an average saving of $10,280 per case in hospitalisation costs for those participants in the experimental condition, over the length of time of the study.