Abstract
Child abuse and neglect are common problems across the world that result in negative consequences for children, families and communities. Children who have been abused or neglected are often removed from the home and placed in residential care or with other families, including foster families. Foster care was traditionally provided by people that social workers recruited from the community specifically to provide care for children whose parents could not look after them. Typically they were not related to the children placed with them, and did not know them before the placement was arranged. In recent years many societies have introduced policies that favour placing children who cannot live at home with other members of their family or with friends of the family. This is known as ‘kinship care’ or ‘families and friends care’. We do not know what type of out-of home care (placement) is best for children.
This systematic review was designed to help find out if research studies could tell us which kind of placement is best. Sixty two studies met the methodological standards we considered acceptable. Wherever possible we combined the data from studies looking at the same outcome for children, in order to be more confident about what the research was telling us. Current best evidence suggests that children in kinship care may do better than children in traditional foster care in terms of their behavioral development, mental health functioning, and placement stability. Children in traditional foster care placements may do better with regard to achieving some permanency outcomes and accessing services they may need. Implications for practice and future research are discussed.
Executive summary/Abstract
BACKGROUND
Every year a large number of children around the world are removed from their homes because they are maltreated. Child welfare agencies are responsible for placing these children in out-of-home settings that will facilitate their safety, permanency, and well-being. However, children in out-of-home placements typically display more educational, behavioral, and psychological problems than do their peers, although it is unclear whether this results from the placement itself, the maltreatment that precipitated it, or inadequacies in the child welfare system.
OBJECTIVES
To evaluate the effect of kinship care placement on the safety, permanency, and well-being of children removed from the home for maltreatment.
SEARCH STRATEGY
The following databases were searched to February 2007: CENTRAL, MEDLINE, C2-Spectr, Sociological Abstracts, Social Work Abstracts, SSCI, Family and Society Studies Worldwide, ERIC, PsycINFO, ISI Proceedings, CINAHL, ASSIA, and Dissertation Abstracts International. Relevant social work journals and reference lists of published literature reviews were handsearched, and authors contacted.
SELECTION CRITERIA
Randomized experimental and quasi-experimental studies, in which children removed from the home for maltreatment and subsequently placed in kinship foster care, were compared with children placed in non-kinship foster care on child welfare outcomes in the domains of well-being, permanency, or safety.
DATA COLLECTION AND ANALYSIS
Reviewers independently read the titles and abstracts identified in the search and selected appropriate studies. Reviewers assessed the eligibility of each study for the evidence base and then evaluated the methodological quality of the included studies. Lastly, outcome data were extracted and entered into REVMAN for meta-analysis with the results presented in written and graphical forms.
RESULTS
Sixty two quasi-experimental studies were included in this review. Data suggest that children in kinship foster care experience better behavioral development, mental health functioning, and placement stability than do children in non-kinship foster care. Although there was no difference on reunification rates, children in non-kinship foster care were more likely to be adopted while children in kinship foster care were more likely to be in guardianship. Lastly, children in non-kinship foster care were more likely to utilize mental health services.
AUTHORS’ CONCLUSIONS
This review supports the practice of treating kinship care as a viable out-of-home placement option for children removed from the home for maltreatment. However, this conclusion is tempered by the pronounced methodological and design weaknesses of the included studies.
Linked article:
Linked article:
Key messages
Child abuse and neglect are common problems across the world that result in negative consequences for children, families and communities. Children who have been abused or neglected are often removed from the home and placed in residential care or with other families, including foster families. Foster care was traditionally provided by people that social workers recruited from the community specifically to provide care for children whose parents could not look after them. Typically they were not related to the children placed with them, and did not know them before the placement was arranged. In recent years many societies have introduced policies that favour placing children who cannot live at home with other members of their family or with friends of the family. This is known as ‘kinship care’ or ‘families and friends care’. We do not know what type of out-of-home care (placement) is best for children.
This review was designed to help find out if research studies could tell us which kind of placement is best. Sixty two studies met the methodological standards we considered acceptable. Wherever possible we combined the data from studies looking at the same outcome for children, in order to be more confident about what the research was telling us. Current best evidence suggests that children in kinship care may do better than children in traditional foster care in terms of their behavioral development, mental health functioning, and placement stability. Children in traditional foster care placements may do better with regard to achieving some permanency outcomes and accessing services they may need. Implications for practice and future research are discussed.
1 Background
1.1 DESCRIPTION OF THE CONDITION
Every year a large number of children around the world are removed from their homes because they are abused, neglected, or otherwise maltreated. For example, there were 513,000 children in out-of-home placements in the United States as of September 2005 (USDHHS 2006b), 60,900 children in public care in England as of March 2005 (DFES 2005), 23,965 children in out-of-home care in Australia as of June 2005 (AIHW 2006), 12,185 children in public care in Scotland as of March 2005 (SENS 2005), 4,668 children in public care in Wales as of March 2005 (NAW 2005), 6,120 children in public care in Norway as of December 2006 (SN 2007), and 7,678 children in out-of-home care in Israel as of 2007 (CBS 2007).
The main reasons for the removal of children in the United States are neglect, physical abuse, sexual abuse, psychological maltreatment, abandonment, threats of harm, and drug addiction (USDHHS 2006a). Abuse and neglect are the most prevalent causes of children being removed from the home in other countries as well (e.g. Wales) (NAW 2005). Internationally, child welfare systems are accountable for the safety, permanency, and wellbeing of children in their care. For children removed from the home, child welfare professionals are responsible for placing them in out-of-home settings that will facilitate these outcomes. Specifically, the primary placement options are traditional foster care, kinship care, residential treatment centers, and group homes (USDHHS 2006b; AIHW 2006). Children in out-of-home placements typically display more educational, behavioral, physical, and psychological problems than do their peers (Gleeson 1999), although it is unclear whether this results from the placement itself, the maltreatment that precipitated it, or inadequacies in the child welfare system. In addition to experiencing poor adult outcomes, these children are at risk for drifting in out-of-home care until, in some cases, they “graduate” from the system because of age (Zuravin 1999).
1.2 DESCRIPTION OF THE INTERVENTION
Kinship Care
Kinship care is broadly defined as, “the full-time nurturing and protection of children who must be separated from their parents, by relatives, members of their tribes or clans, godparents, stepparents, or other adults who have a kinship bond with a child” (CWLA 1994, p. 2). This is contrasted with traditional foster care or non-kinship foster care, which is the placement of children removed from the home with unrelated foster parents. Kinship care is known by many other names around the world, including family and friends care in the United Kingdom, kith and kin care in Australia, and kinship foster care in the United States. For this review, kinship care will refer to kinship foster care placements, while foster care will refer to non-kinship foster care placements.
There are several variations of kinship care including formal, informal, and private placements. Formal kinship care is a legal arrangement in which a child welfare agency has custody of a child (Ayala-Quillen 1998). Informal kinship care is when a child welfare agency assists in the placement of a child but does not seek custody (Geen 2000). Private kinship care is a voluntary arrangement between the birth parents and family members without the involvement of a child welfare agency (Dubowitz 1994a).
The most commonly perceived benefits are that kinship care “enables children to live with persons whom they know and trust, reduces the trauma children may experience when they are placed with persons who are initially unknown to them, and reinforces children's sense of identity and self esteem which flows from their family history and culture” (Wilson 1996, p. 387). The primary aims of kinship placements are family preservation, in which the permanency goal is reunification with birth parents, and substitute care, in which kinship care is considered to be a long-term arrangement when restoration is not possible or the permanency goal is adoption or guardianship by kin caregivers (Scannapieco 1999). Kinship care also is considered to be the least restrictive (Scannapieco 1999) and safest setting (Gleeson 1999) on the continuum of out-of-home placements.
Intervention Context
Although an ancient practice in many cultures, formal kinship care is a newer placement paradigm in countries like the United States and Australia due to its recent adoption by the child welfare field as the placement of choice, when appropriate, in the continuum of out-of home care services for children (Ainsworth 1998; Geen 2000; Scannapieco 1999). For example, the Personal Responsibility and Work Opportunity Reconciliation Act of 1996 explicitly required American states to give preference to family members when placing a child outside of the home (Leos-Urbel 2002). The most recent United States legislation, the Adoption and Safe Families Act of 1997, continues this federal commitment towards promoting and supporting kinship care (Ayala-Quillen 1998). In Australia, the Aboriginal Child Placement Principle has resulted in the increased use of kinship placements, although this differs by state or territory (Paxman 2006). In addition, the New South Wales Children and Young Persons (Care and Protection) Act 1998 encourages the “least intrusive” principle, which is interpreted by caseworkers as placements with kin (Spence 2004). In some European countries, there also has been a shift in policy regarding kinship placements. Specifically, the Children Act 1989 (United Kingdom), the Children Act 1995 (Scotland), and the Children Order 1995 (Northern Ireland) are generally supportive of kinship care (Broad 2005a). However, there is no legislation in Israel concerning kinship care and a lack of consensus about how to define and serve the population of children at risk for maltreatment (Schmid 2007).
For the countries included in this review (i.e. re-entry Australia, Israel, Netherlands, Norway, Sweden, and U.S.), there are essential differences in child welfare policy and practice for placing children in out-of-home care. Outside of the U.S., long-term foster or kinship care is the preferred placement, which implies that parents have right of access to their child provided it is not considered damaging, and also a right to express their opinion on important issues like education and religion. In Australia, Israel, Netherlands, Norway, and Sweden, foster care placement is not time-limited and can be extended until the child emancipates from care (e.g. Strijker 2003). Because, the preferred option is long-term stable placements, there are foster children in Norway and Sweden who remain in foster homes throughout their entire childhood (e.g. Sallnas 2004). Thus, the concept of breakdown (premature termination of placement) is therefore a more relevant measure in the evaluation of foster care than is reunification or adoption (Sallnas 2004).
During the past 15 years in many countries, there has been a rapid increase in the number of children removed from home and placed with relatives (Cuddeback 2004). The main reasons for the growth of this placement option include an influx of abused and neglected children into out-of-home care (Berrick 1998), concern about poor outcomes for children leaving care (Broad 2005b), a persistent shortage in foster care homes (Berrick 1998), and a shift in policy toward treating kin as appropriate caregivers with all of the legal rights and responsibilities of foster parents (Leos-Urbel 2002). In New South Wales, Australia, the most important factor accounting for historically high numbers of children in foster care is the low use of residential care (Tarren-Sweeney 2006). However, residential care is the preferred out-of-home placement setting for older children in Israel and Sweden (Mosek 2001; Sallnas 2004).
Similar to other child welfare interventions, kinship care is faced with its fair share of controversial issues. The major controversy centers on the unequal financial support (Brooks 2002) and service provision received by kinship caregivers as compared with traditional foster parents (Dubowitz 1994a). The licensing and certification of kinship caregivers also is a source of much disagreement and dissatisfaction (Gibbs 2000). Relatedly, the appropriate level of oversight of kinship caregivers by child welfare agencies is another area of discord (Cohen 1999). One of the key debates is over the appropriate level of involvement for biological parents prior to and after the removal of their children (Ayala-Quillen 1998). In a comprehensive review of the U.S. literature, Cuddeback 2004 confirmed much of the conventional wisdom about kinship care while identifying many of the weaknesses of quantitative research on the topic. Cuddeback found that kinship caregivers are more likely to be older, single, less educated, unemployed, and poor than are foster parents and noncustodial grandparents. Furthermore, Cuddeback reported that kin caregivers report less daily physical activity, more health problems, higher levels of depression, and less marital satisfaction. Cuddeback also concluded that kinship care families receive less training, services, and financial support than do foster care families. In addition, Cuddeback reported that birth parents rarely receive family preservation services, which means that children in kinship care are less likely than children in foster care to be reunified. Lastly, Cuddeback found inconclusive evidence that children in kinship care have greater problems related to overall functioning than do children in foster care.
1.3 WHY IT IS IMPORTANT TO DO THIS REVIEW
Geen 2004 argues that, “despite the centrality of kinship foster care in child welfare, our understanding of how best to utilize and support kin caregivers, and the impact of kinship foster care on child development, is limited” (p. 144). Specifically, social work researchers have not kept up with the exponential growth of kinship care as a placement option (Berrick 1994; Dubowitz 1994a). Furthermore, much of the research supporting kinship care is anecdotal and conjectural, which does not allow child welfare professionals to make evidence-based decisions from comparisons of children in out-of-home care (Goerge 1994). For example, there is great interest in the safety and well-being of children placed in kinship care, but very little experimental research on these outcomes (Gibbs 2000). Ethical standards preclude the random assignment of children to kinship or foster care, as these placements typically are based on the appropriateness and availability of kinship caregivers or foster parents (Barth 2008). However, Barth 2008 have identified several recent studies which employed propensity score matching as a means of statistically simulating random assignment to placement conditions. We addressed these methodological challenges by identifying and synthesizing the most strongly designed and executed studies available on this topic. Unfortunately, the best available evidence on kinship care was seriously lacking in many ways, especially in regard to controlling for baseline differences in non-randomized studies. Although some would argue that this should disqualify kinship care, along with similar social work interventions, from being systematically reviewed, we believe that practitioners and policymakers benefit more from examining poor evidence than no evidence at all.
2 Objectives of the Review
To evaluate the effect of kinship care placement compared to foster care placement on the safety, permanency, and well-being of children removed from the home for maltreatment.
3 Methods
3.1 CRITERIA FOR CONSIDERING STUDIES FOR THIS REVIEW
Types of studies
Randomized experimental and quasi-experimental studies, in which children placed in kinship care are compared cross-sectionally or longitudinally with children placed in foster care. The types of eligible quasi-experimental designs include studies that employ matching, covariates, or ex post facto comparisons of children in kinship care and foster care. Studies that compare kinship care to more restrictive out-of-home settings (e.g. residential treatment centers) were not considered for this review. Relative to children who are placed in kinship or foster care, children placed in more restrictive settings tend to differ in important ways. These differences complicate inferences about the effects of placement and as such, the review focused on kinship and foster care placements only.
Types of participants
Children and youth under the age of 18 who were removed from the home for abuse, neglect, or other maltreatment and subsequently placed in kinship care.
Types of interventions
Formal kinship care placements, irrespective of whether the kin caregivers were licensed (paid) or unlicensed (unpaid). Thus, studies that exclusively examine informal or private kinship care arrangements were not considered. Studies were considered if participants experienced other placement types in conjunction with the kinship care intervention. For example, the treatment group may include children for whom kinship care was their first, last, or only placement in out-of-home care. However, these children must have spent the majority (i.e. re-entry more than 50%) of their total time in out-of-home care in kinship care.
Types of outcome measures
Eligible studies must analyze child welfare outcomes in the well-being, permanency, or safety domains. Although caregiver and birth parent outcomes are very relevant, they were not considered in this review because child outcomes are what drive the policy and practice of kinship care. However, these outcomes may mediate or moderate the effect of kinship care on child welfare outcomes and should be explored in future research on the topic. Primary outcomes for the review are behavioral development, mental health, placement stability, and permanency. Secondary outcomes include educational attainment, family relations, service utilization, and re-abuse. The following list of outcome domains is meant to be exhaustive, although the examples in each domain are illustrative of the outcomes to be considered in this review.
Behavioral Development
Behavior problems, adaptive behaviors
Measured by case records, caregiver, teacher, and self-reports, and standardized instruments (e.g. Child Behavior Checklist)
Mental Health
Psychiatric illnesses, psychopathological conditions, well-being
Measured by case records, caregiver and self-reports, and standardized instruments
Placement Stability
Number of placements, re-entry, length of placement
Measured by child welfare administrative databases
Permanency
Reunification, adoption, guardianship
Measured by child welfare administrative databases
Educational Attainment
Graduation, grades, test scores
Measured by school records and child welfare administrative databases
Family Relations
Problem-solving, tolerance, commitment, conflicts
Measured by caregiver and self-reports and standardized instruments
Service Utilization
Mental health services, foster support groups, family therapy
Measured by medical records, caregiver and self-reports, and child welfare administrative databases
Re-abuse
Substantiated abuse, institutional abuse
Measured by child welfare administrative databases
3.2 SEARCH METHODS FOR IDENTIFICATION OF STUDIES
Electronic searches
To identify relevant studies, the following online databases were searched in February 2007: Cochrane Library (CENTRAL) searched 2007 (Issue 1) MEDLINE searched 1966 to February 2007 Campbell Collaboration's Social, Psychological, Educational, and Criminological Trials Register (C2-SPECTR) searched March 9th 2007 Sociological Abstracts searched 1962 to February 2007 Social Work Abstracts searched 1977 to February 2007 Social Sciences Citation Index (SSCI) searched 1900 to February 17th 2007 Family and Society Studies Worldwide searched 1970 to February 2007 ERIC searched 1966 to February 2007 PsycINFO searched 1872 to January week 5 2007 ISI Proceedings searched 1990 to February 16th 2007 CINAHL searched 1982 to February week 3 2007 Applied Social Sciences Index and Abstracts (ASSIA) searched 1987 to February 2007 UMI Dissertation Abstracts International (DAI) searched late 1960s to February 2007
The search strategies used can be found in: Appendix 1, Appendix 2, Appendix 3, Appendix 4, Appendix 5, Appendix 6, Appendix 7, Appendix 8, Appendix 9, Appendix 10, Appendix 11, Appendix 12. Both published and unpublished studies were sought, and there were no language, date, or geographic limitations. Preliminary searches indicated that a narrowing of the search strategy using a methodological filter resulted in the exclusion of potentially relevant studies so the searches were run without a filter.
Searching other resources
The most recent volumes of Child Abuse & Neglect, Children and Youth Services Review, Child Welfare, Research on Social Work Practice, and Families in Society were manually searched. In addition, several authors of studies included in this review were contacted for knowledge of other studies not yet identified. Lastly, the reference lists of published literature reviews were screened for relevant studies.
3.3 DATA COLLECTION AND ANALYSIS
Selection of studies
Two reviewers independently read the titles and abstracts of identified articles and reports to select those that described an empirical study of kinship care. A study was obtained if either reviewer believed it is appropriate. Once retrieved, two reviewers used a “keywording” rubric to categorize each study by the type of design, participants, intervention, and outcome measure(s). Two reviewers then determined if each study was eligible for selection based on the aforementioned criteria for considering studies for this review. When consensus regarding selection decisions was not reached, it was resolved through discussion with a third reviewer.
Data extraction and management
Citations for all selected studies were entered into Reference Manager 11, which is an interactive literature management software package. The citations for included studies then were uploaded into the Cochrane Collaboration's Review Manager 4.2.8 software (RevMan). Outcome data were extracted from studies and entered into RevMan, where it was analyzed in the meta-analyses for this review. The statistical results are presented in both narrative form and in figures and tables. Specifically, forest plots generated from RevMan are used to display effect size estimates and confidence intervals from the meta-analyses. In addition, data from the quality assessment process are presented in a table created in RevMan.
Assessment of risk of bias in included studies
Quality Assessment
Existing scales for measuring the quality of controlled trials have not been properly developed, are not well-validated, and are known to give differing (even opposing) ratings of trial quality in systematic reviews (Moher 1999). At present, evidence indicates that “scales should generally not be used to identify trials of apparent low quality or high quality in a given systematic review. Rather, the relevant methodological aspects should be identified a priori and assessed individually” (Juni 2001, p. 45). According to Higgins 2005, “factors that warrant assessment are those related to applicability of findings, validity of individual studies, and certain design characteristics that affect interpretation of results (p. 79). Thus, studies were assessed in regard to the following research quality dimensions: selection bias, performance bias, detection bias, report bias, and attrition bias (Higgins 2005).
Methodological Criteria
To provide guidelines for assessing the methodological criteria of included studies, a “data extraction” rubric was developed. Two reviewers independently extracted data from each study before coming to consensus on the assessment of quality dimensions for each study. The methodological criteria were operationalized as follows: Selection Bias: Was group assignment determined randomly or might it have been related to outcomes or the interventions received? Performance Bias: Could the services provided have been influenced by something other than the interventions being compared? Detection Bias: Were outcomes influenced by anything other than the constructs of interest, including biased assessment or the influence of exposure on detection? Report Bias: Were the outcomes, measures, and analyses selected a priori and reported completely? Were participants biased in their recall or response? Attrition Bias: Could deviations from protocol, including missing data and dropout, have influenced the results?
Measures of treatment effect
A standardized mean difference (SMD) effect size was computed for the continuous outcome variables. For this review, a corrected Hedges’ g was computed by dividing the difference between group means by the pooled and weighted standard deviation of the groups. Specifically, Hedges’ g corrects for a bias (overestimation) that occurs when the uncorrected standardized mean difference effect size is used on small samples. The combined effect size for each outcome was computed as a weighted mean of the effect size for each study, with the weight being the inverse of the square of the standard error. Thus, a study was given greater weight for a larger sample size and more precise measurement, both of which reduce standard error. We computed a 95% confidence interval for each combined effect size to test for statistical significance; if the confidence interval did not include zero, we rejected the null hypothesis that there is no difference between the group means. Odds ratios (OR) were computed for the dichotomous outcome variables. Based on the assumption of proportional odds, odds ratios can be compared between variables with different distributions, including very rare and more frequent occurrences. Specifically, the odds of an event (e.g. reunification) were calculated for each group by dividing the number of events (i.e. re-entry reunified) by the number of non-events (i.e. re-entry not reunified). An odds ratio then was calculated by dividing the odds of the kinship care group with the odds of the foster care group. In addition, 95% confidence intervals were computed and reported for the dichotomous effect size estimates.
Unit of analysis issues
The unit of analysis was children for this review. No attempts were made to adjust for clustering and we were unaware of any such problems including multiple children per family.
Dealing with missing data
Although studies with incomplete outcome data (e.g. missing means, standard deviations, sample sizes) were included in the review, they were excluded from the meta-analyses unless the reviewers could calculate an effect size from the available information. When outcome data were missing from an article or report, reasonable attempts were made to retrieve these data from the original researchers. Overall and differential attrition were accounted for in the quality assessment and sensitivity analyses.
Assessment of heterogeneity
The consistency of results was assessed using the I2 statistic (Higgins 2002; Higgins 2003). If there was evidence of heterogeneity (p value from test of heterogeneity < 0.1 coupled with an I2 value of 25% or greater), we also considered sources according to pre-specified subgroup and sensitivity analyses (see below).
Assessment of reporting biases
Reporting biases were assessed and reported in Table 3.
Data synthesis
As heterogeneity is to be expected with similar interventions provided under different circumstances and by different providers, the data syntheses used a random effects model. If a study reported multiple effect sizes (e.g. grades, behavior problems), the results were included in the meta-analysis for each outcome. If a study reported effect sizes for multiple samples (e.g. male, female), the results were aggregated for the main effects meta-analyses before being used for the subgroup meta-analyses.
Subgroup analysis and investigation of heterogeneity
Subgroup analyses were considered to examine different effects of the intervention (if any) by gender, ethnicity, and age at placement.
Sensitivity analysis
Sensitivity analyses were considered to explore the impact of the quality dimensions on the outcomes of the review. Specifically, the following planned comparisons were considered. Studies that use matching or covariates and studies that do not control for confounders. Studies with outcomes measured by caregiver or teacher reports and studies with outcomes measured by self-reports. Studies with low overall or differential attrition and studies with high overall or differential attrition.
4 Results
4.1 DESCRIPTION OF STUDIES
A comprehensive electronic search of the kinship care literature base using 15 online databases yielded 4791 citations that matched the search terms (see also Figure 1). The 4791 abstracts for these citations were reviewed and 263 references were identified as meeting the initial criteria of being an empirical study on kinship care. Of the 263 references, 251 were acquired with 12 being unavailable. These 251 articles/reports were then keyworded to determine whether they met the inclusion criteria for the review. As a result, 90 studies were deemed to be potentially appropriate to be included in the review. However, two studies were subsequently categorized as being duplicates and 26 studies did not meet the inclusion criteria upon further review. Thus, 62 eligible studies were included in the evidence base with an overall interrater agreement of 81% before consensus. As displayed in the Table of Included Studies, outcome data were extracted from these 62 studies with 27 studies reporting bivariate data only, 16 studies reporting multivariate data only, and 19 studies reporting both bivariate and multivariate data.
Excluded Studies
As displayed in the Table of Excluded Studies, 189 of the 251 available studies were excluded from the review for the following reasons: 81 studies were excluded because there was no formal kinship care group or the kinship care group was not disaggregated from the foster care group; 30 studies were excluded because there was no foster care comparison group or the foster care group was not disaggregated from other out-of-home placement types; 27 studies were excluded because they were non-empirical (e.g. literature reviews); 19 studies were excluded because they were survey, descriptive, or qualitative research designs; 13 studies were excluded because child welfare outcomes were not reported; 10 studies were excluded because they reported on an intervention other than out-of-home placement; five studies were excluded because they were based on an adult sample; and three studies were excluded because the kinship and foster care placements were considered to be permanent.
More than a year after searches were run, a further nine references not identified through electronic searches were supplied by a UK-based peer reviewer of this paper for scrutiny (Adams 1969a, Rowe 1984, Millham 1986a, Berridge 1987a, Rowe 1989, Farmer 1991, Kosenen 1993, Hunt 1999a, Sinclair 2000a). Of these, three were excluded (Adams 1969a, Farmer 1991, and Kosenen 1993). Six studies remain awaiting assessment prior to a minor update of this review (Berridge 1987a, Millham 1986a, Hunt 1999a, Rowe 1984, Rowe 1989, Sinclair 2000a).
Studies awaiting assessment
Three relatively old studies remain awaiting assessment (including Adams 1969a (currently unavailable), Rowe 1984 and Rowe 1989.
Location of Studies
All but five of the 62 studies were conducted in the U.S. The five international studies were Holtan 2005 conducted in Norway, Mosek 2001 conducted in Israel, Sallnas 2004 conducted in Sweden, Strijker 2003 conducted in the Netherlands, and Tarren-Sweeney 2006 conducted in Australia.
Participants
As displayed in the Participant Baseline Characteristics Table (Table 5), 52 of the 62 studies reported data for at least one of the following participant characteristics: age at placement, gender, ethnicity, removal reason, or urbanicity.
For age at entry into the specific placement, there was an overall unweighted mean age at placement of 7 years 11 months based on eight studies. Furthermore, seven studies reported a mean age at placement by placement type. For the kinship care group, the unweighted mean age at placement was 4 years 7 months. For the foster care group, the unweighted mean age at placement was 4 years 4 months.
For gender, there were overall unweighted frequencies of 53% female and 47% male children based on 38 studies. Furthermore, 20 studies reported gender frequencies by placement type. For the kinship care group, the unweighted frequencies were 50% female and 50% male. For the foster care group, the unweighted frequencies were 54% female and 46% male.
For ethnicity, there was an overall unweighted frequency of 49% African-American children based on 37 U.S. studies. Furthermore, 17 studies reported the frequency of African-American children by placement type. For the kinship care group, the unweighted frequency was 63% African-American. For the foster care group, the unweighted frequency was 56% African-American. In addition, there was an overall unweighted frequency of 21% Hispanic children based on 31 studies. Furthermore, 12 studies reported the frequency of Hispanic children by placement type. For both the kinship care and foster care groups, the unweighted frequency was 21% Hispanic.
For removal reason, there was an overall unweighted frequency of 63% of children removed for neglect based on 24 studies. Furthermore, seven studies reported the frequency of children removed for neglect by placement type. For the kinship care group, the unweighted frequency was 69% of children removed for neglect. For the foster care group, the unweighted frequency was 66% of children removed for neglect.
For urbanicity, there was an overall unweighted frequency of 80% of children from urban settings based on nine studies. In addition, there was an overall unweighted frequency of 17% of children from rural settings based on four studies. However, no studies reported the urbanicity of children by placement type.
Interventions
As displayed in the Intervention Characteristics Table (Table 6), all 62 studies reported data for at least one of the following intervention characteristics: caregiver licensure, timing of placement, length of stay, or timing of data collection.
For caregiver licensure, eight studies reported information on whether kinship caregivers were licensed or unlicensed. Specifically, five studies included licensed kinship placements, two studies included unlicensed kinship placements, and one study included both licensed and unlicensed kinship placements.
For the timing of placement, 25 studies reported information on whether children were in their first, last, or only kinship or foster placement. Specifically, the kinship or foster placement was the first in 16 of the studies, the last in six of the studies, the only placement in one study, and either the first or last placement depending on the outcome being measured in two studies.
For length of stay, there was an unweighted mean length of placement of 31.1 months for the kinship care group and 30.1 months for the foster care group based on 11 studies. In addition, there was an unweighted mean length of stay in out-of-home care of 52.5 months for the kinship care group and 49.5 months for the foster care group based on 10 studies.
For the timing of data collection, 41 studies used a cross-sectional data collection approach while 21 studies used a longitudinal data collection approach.
Outcome measures
There were eight outcome categories and 28 specific outcomes considered in this review (including the same outcome measured both dichotomously and continuously). The following narrative contains the definitions and instrumentation used to measure the outcome variables in which bivariate data were extracted for the meta-analyses. The Outcomes Measures Table (Table 7) displays the outcomes and measures for all 62 studies in the review.
Behavioral Development
The two behavioral development outcomes were behavior problems and adaptive behaviors. Behavior problems were defined dichotomously as the presence or absence of internalizing (e.g. withdrawn, passive) and externalizing (e.g. aggressive, delinquent) problem behaviors and continuously as the level of these behaviors. The continuous outcome was measured by the total problems scale of the Child Behavior Checklist (CBCL) completed by caregivers in seven studies (Davis 2005; Holtan 2005; Jones-Karena 1998; Rudenberg 1991; Strijker 2003; Tarren-Sweeney 2006; Timmer 2004), the Behavior Problems Index completed by caregivers in one study (Brooks 1998), and caregiver reports in one study (Metzger 1997; Surbeck 2000). The dichotomous outcome was measured by case records in two studies (Iglehart 1994; Landsverk 1996). Adaptive behaviors were defined continuously as the level of competence or positive behaviors and were measured by the total competence scale of the CBCL completed by caregivers in two studies (Holtan 2005; Tarren-Sweeney 2006), the adaptive composite score on the Vineland Adaptive Behavior Scales (VABS) completed by caregivers in two studies (Belanger 2002; Jones-Karena 1998), and caregiver reports in one study (Surbeck 2000).
Mental Health
The two mental health outcomes were psychiatric disorders and well-being. Psychiatric disorders were defined dichotomously by the presence or absence of mental illness and continuously by scores on a measure of psychopathology. The dichotomous outcome was measured by paid claims data in one study (Bilaver 1999), case records in one study (Harris 2003; Iglehart 1994), and the DSM-IV in one study (McMillen 2005). The continuous outcome was measured by the Devereaux Scales of Mental Disorders completed by caregivers in one study (Belanger 2002). Well-being was defined dichotomously by the presence or absence of positive emotional health and continuously by the level of well-being or selfworth. The dichotomous outcome was measured by child self-reports in one study (Wilson 1999), R.C. Monitoring Protocol completed by caseworkers, caregivers, and child in one study (Harris 2003), and caseworker reports in one study (Tompkins 2003). The continuous outcome was measured by the Personal Attribute Inventory for Children completed as a child self-report in one study (Metzger 1997).
Placement Stability
The four placement stability outcomes were number of placements, length of placement, placement disruption, and re-entry as measured by secondary data from administrative databases. Number of placements was measured both continuously by the number of out-of-home placements and dichotomously by experiencing either two or fewer or three or more placement settings. The dichotomous outcome was used in four studies (Courtney 1997b; Harris 2003; Metzger 1997; Zimmerman 1998). The continuous outcome was used in two studies (Belanger 2002; Davis 2005). Length of placement was measured continuously by length of stay in the kin or foster placement in five studies (Berrick 1994; Brooks 1998; Cole 2006; Davis 2005; Surbeck 2000) and continuously by length of total stay in out-of-home care in five other studies (Belanger 2002; Clyman 1998; Jenkins 2002; Sivright 2004; Tompkins 2003). It should be noted that longer lengths of stay in placement or in care are considered negative outcomes in the U.S., as reunification within 12 months is the primary permanency goal for children placed in short-term kinship or foster care. Placement disruption was measured dichotomously by whether the kin or foster placement ended without permanency in two studies (Sallnas 2004; Testa 2001). Re-entry was measured dichotomously by whether there was a re-entry to out-of-home care after achieving permanency in one study (Frame 2000).
Permanency
The four permanency outcomes were reunification, adoption, guardianship, and still in placement. All four outcomes were measured dichotomously by secondary data from administrative databases in ten studies (Barth 1994; Berrick 1999; Mcintosh 2002; Sivright 2004; Smith 2002; Smith 2003; Testa 1999; Testa 2001; Wells 1999; Zimmerman 1998). Reunification was defined as a return home to biological or birth parents after placement in out-of-home care. Adoption was defined as a termination of parental rights with legal custody transferred to adoptive parents (in most cases non-relatives). Guardianship was defined as an allocation of parents rights with legal custody to relative caregivers (in most cases relatives). Still in placement was defined as remaining in either kinship or foster care at the time data were collected for the study.
Educational Attainment
The three educational attainment outcomes were repeated a grade, graduation, and grade level and all were measured dichotomously. It should be noted that these outcomes are all U.S. measures of educational attainment. Repeated a grade was defined by whether a child had been retained in one or more grades as measured by caregiver reports in four studies (Berrick 1994; Brooks 1998; Metzger 1997; Sripathy 2004). Graduation was defined by whether a child completed high school and was measured by case records in one study (Christopher 1998). Grade level was defined by whether a child's academic performance was below their actual grade level and was measured by child self-reports in one study (Iglehart 1995) and case records in one study (Iglehart 1994).
Family Relations
The two family relations outcomes were attachment and conflict. Attachment was defined as perceived level of relatedness or attachment between child and caregiver and was measured continuously by child self-reports in one study (Chapman 2004), the Attachment Q-Sort Version 3 Assessment completed by the child in one study (Chew 1998), caregiver reports in one study (Strijker 2003), Assessment of Interpersonal Relations completed by the child in one study (Davis 2005), and the Child Well-Being Scales completed by caregivers in one study (Surbeck 2000). Attachment was measured dichotomously by the Ainsworth Strange Situation Procedure based on observations of caregiver and child in one study (Cole 2006), case records in one study (Jenkins 2002), and the Offer Self-Image Questionnaire completed by the child in one study (Mosek 2001). Conflict was defined continuously as the level of family functioning as measured by the Index of Family relations completed by caregivers in one study (Berrick 1997).
Service Utilization
The three service utilization outcomes were mental health services, physician services, and developmental services defined dichotomously as whether a child actually received services (not just referred). Mental health and physician service utilization was measured by paid claims data in one study (Bilaver 1999), caseworker reports in two studies (Metzger 1997; Tompkins 2003), case records in three studies (Jenkins 2002; Scannapieco 1997; Sivright 2004), caregiver reports in two studies (Berrick 1994; Sripathy 2004), and The Young Kids Early Services Assessment (TYKES) in one study (Clyman 1998). Physician services were measured by paid claims data in one study (Bilaver 1999) and the TYKES in one study (Clyman 1998).
Re-abuse
The two re-abuse outcomes were recurrence of abuse and institutional abuse as measured dichotomously by secondary data from administrative databases. Recurrence of abuse was defined as whether a new substantiated incident of intrafamilial abuse or neglect (i.e. re-entry by birth or biological parent(s) not kin caregiver(s) or foster parent(s)) occurred after a previous substantiated incident and was reported in one study (Fuller 2005). Institutional abuse was defined as whether a substantiated incident of abuse or neglect occurred in an out-of-home placement setting (i.e. re-entry by kin caregiver(s) or foster parent(s) not birth or biological parent(s)) and was reported in two studies (Benedict 1996a; Zuravin 1993).
4.2 RISK OF BIAS IN INCLUDED STUDIES
The included studies were assessed on methodological quality in regard to selection bias, performance bias, detection bias, report bias, and attrition bias. Specifically, each study was rated either low risk, moderate risk, or high risk based on two sub-questions for each of these areas. As displayed in the Quality Assessment Ratings Table (Table 3), the quality assessment indicates that the evidence base contains studies with at least moderate risk in all five categories with the highest risk associated with selection bias and the lowest risk associated with attrition bias.
For selection bias, five studies (Belanger 2002; Clyman 1998; Holtan 2005; Metzger 1997; Testa 2001) were rated low risk, 39 studies were rated moderate risk, and 18 studies were rated high risk. The studies rated high risk did not attempt to equate the kinship care and foster care groups through matching or controlling for covariates AND did not provide evidence on the comparability of the groups on setting (e.g. urbanicity), placement characteristics (e.g. age at placement, removal reason), or child demographics (e.g. gender, ethnicity). The studies rated moderate risk either attempted to equate the groups OR provided evidence on the comparability of the groups. The studies rated low risk attempted to equate the groups AND provided evidence on the comparability of the groups. For example, these studies provided evidence that the groups were comparable at baseline in regard to placement history, visits to biological parents, and caregiver characteristics (e.g., family composition, age, education). The primary reasons that studies were assessed to have moderate or high risk for selection bias were the lack of equating procedures and the non-reporting of placement and demographic data.
For performance bias, four studies (Berrick 1997; Holtan 2005; Metzger 1997; Sivright 2004) were rated low risk, 54 were rated moderate risk, and four were rated high risk. In the studies that were rated high risk, the kinship care and foster care groups experienced differential exposure to the intervention (e.g. length of stay) AND received differential services during placement (e.g. caseworker contact). In the studies that were rated moderate risk, the groups either experienced differential exposure OR received differential services. In the studies that were rated low risk, the groups did not experience differential exposure AND did not receive differential services. The primary reasons that studies were assessed to have moderate or high risk for performance bias were uncertainty regarding both the length of stay and receipt of services during placement.
For detection bias, four studies (Benedict 1996a; Jenkins 2002; Scannapieco 1997; Zuravin 1993) were rated low risk, 54 were rated moderate risk, and four were rated high risk. In the studies rated high risk, the kinship care and foster care groups were not defined in the same way (e.g. caregiver licensure, caregiver characteristics) AND there was evidence of biased assessment resulting from the type of placement (e.g. caregiver reports only). In the studies rated moderate risk, the groups were not defined in the same way OR there was evidence of biased assessment. In the studies rated low risk, the groups were defined in the same way AND there was no evidence of biased assessment. The primary reasons that studies were assessed to have moderate or high risk for detection bias were uncertainty in how the groups were defined and the use of only caregiver or self-reports to measure the outcome. Although biased assessment is not necessarily due to the type of placement, it may differentially impact the detection of a placement's effect on child welfare outcomes.
For report bias, 19 studies were rated low risk and 43 studies were rated moderate risk. In the studies rated moderate risk, the instrumentation used to measure the outcomes was specified completely (e.g. data collection procedures) OR reliability and/or validity information was reported for the instrumentation. In the studies rated low risk, the instrumentation was completely specified AND reliability and/or validity information was reported. The primary reason that studies were assessed to have moderate risk for report bias was the lack of reliability and/or validity information.
For attrition bias, 25 studies were rated low risk, 35 studies were rated moderate risk, and two studies were rated high risk. In the studies rated high risk, all subjects were not accounted for in the reporting of results (e.g. low response rate, missing outcome data) AND attrition could have influenced the results (e.g. significant difference between participants and nonparticipants). In the studies rated moderate risk, all subjects were not accounted for OR attrition could have influenced the results. In the studies rated low risk, all subjects were accounted for AND attrition could not have influenced the results. The primary reason that studies were assessed to have moderate or high risk for attrition bias was the loss of subjects due to missing outcome data.
4.3 EFFECTS OF INTERVENTIONS
Meta-analyses
There were sufficient data for meta-analysis for 16 of the 28 outcomes in the review, as only outcomes with three or more studies were considered for effect size calculation. As a result, meta-analyses were generated for all outcome categories except for re-abuse. The results for these 16 outcomes are reported in regard to the statistical significance of the effect, the direction and magnitude of the effect size, the 95% confidence interval around the effect size estimate, and the evidence of heterogeneity for the individual effect sizes. The effect sizes were drawn exclusively from the studies reporting bivariate data, and thus do not reflect adjustment by covariates. It should be noted that all SMD effect sizes that are negative indicate better outcomes for the kinship care group, while all OR effect sizes that are less than 1.0 also indicate better outcomes for the kinship care group.
Behavioral Development
There was a statistically significant, small overall effect size for the 10 studies (Brooks 1998; Davis 2005; Holtan 2005; Jones-Karena 1998; Metzger 1997; Rudenberg 1991; Strijker 2003; Surbeck 2000; Tarren-Sweeney 2006; Timmer 2004) that reported sufficient bivariate data to generate effect size estimates for behavior problems. Specifically, the overall effect size estimate was g = -.24 with a confidence interval of -.13 to -.35 (see Analysis 1.1). Thus, children in kinship care had lower reported levels of internalizing and externalizing behavior problems than did children in foster care. The test of heterogeneity was not significant for this outcome (p = .16; I2 = 31.1%). There was a statistically significant overall effect size for the five studies (Belanger 2002; Holtan 2005; Jones-Karena 1998; Surbeck 2000; Tarren-Sweeney 2006) that reported sufficient bivariate data to generate effect size estimates for adaptive behaviors. Specifically, the overall effect size estimate was g = -.45 with a confidence interval of -.19 to -.70 (see Analysis 1.2). Thus, children in kinship care had higher reported levels of competence than did children in foster care. The test of heterogeneity was significant for this outcome (p = .01; I2 = 68.4%).
Mental Health
There was a statistically significant overall effect size for the four studies (Bilaver 1999; Harris 2003; Iglehart 1994; McMillen 2005) that reported sufficient bivariate data to generate effect size estimates for psychiatric disorders. Specifically, the overall effect size estimate was OR = .46 with a confidence interval of .44 to .49 (see Analysis 2.1). Thus, children in foster care were 2.2 times more likely than were children in kinship care to experience mental illness. The test of heterogeneity was not significant for this outcome (p = .93; I2 = 0.0%). There was a statistically significant overall effect size for the three studies (Harris 2003; Tompkins 2003; Wilson 1999) that reported sufficient bivariate data to generate effect size estimates for well-being. Specifically, the overall effect size estimate was reported OR = .52 with a confidence interval of .51 to .53 (see Analysis 2.3). Thus, children in kinship care were 1.9 times more likely than were children in foster care to report positive emotional health. The test of heterogeneity was not significant for this outcome (p =.95; I2 = 0.0%).
Placement Stability
There was a statistically significant overall effect size for the four studies (Courtney 1997b; Harris 2003; Metzger 1997; Zimmerman 1998) that reported sufficient bivariate data to generate effect size estimates for placement settings. Specifically, the overall effect size estimate was OR = .36 with a confidence interval of .27 to .49 (see Analysis 3.1). Thus, children in foster care were 2.6 times more likely than were children in kinship care to experience three or more placement settings. The test of heterogeneity was not significant for this outcome (p = .31; I2 = 16.8%).
There were five studies (Berrick 1994; Brooks 1998; Cole 2006; Davis 2005; Surbeck 2000) that reported sufficient bivariate data to generate effect size estimates for length of placement. Although the overall effect size was in favor of children in foster care, the effect was not statistically significant. Specifically, the overall effect size estimate was g = .86 with a confidence interval of -.98 to 2.70 (see Analysis 3.3). However, the analysis could not rule out zero as a likely population value. The test of heterogeneity was significant for this outcome (p < .001; I2 = 99.4%).
There were five studies (Belanger 2002; Clyman 1998; Jenkins 2002; Sivright 2004; Tompkins 2003) that reported sufficient bivariate data to generate effect size estimates for length of stay in out-of-home care. Although the overall effect size was in favor of children in foster care, the effect was not statistically significant. Specifically, the overall effect size estimate was g = .21 with a confidence interval of -.07 to .48 (see Analysis 3.6). However, the analysis could not rule out zero as a likely population value. The test of heterogeneity was significant for this outcome (p < .001; I2 = 78.8%).
Permanency
There were seven studies (Berrick 1999; Mcintosh 2002; Smith 2002; Testa 1999; Testa 2001; Wells 1999; Zimmerman 1998) that reported sufficient bivariate data to generate effect size estimates for reunification. Although the overall effect size was in favor of children in foster care, the effect was not statistically significant. Specifically, the overall effect size estimate was OR = 1.13 with a confidence interval of .92 to 1.41 (see Analysis 4.1). However, the analysis could not rule out zero as a likely population value. The test of heterogeneity was significant for this outcome (p < .001; I2 = 77.8%).
There was a statistically significant overall effect size for the six studies (Barth 1994; Berrick 1999; Smith 2002; Testa 1999; Testa 2001; Zimmerman 1998) that reported sufficient bivariate data to generate effect size estimates for adoption. Specifically, the overall effect size estimate was OR = 2.50 with a confidence interval of 1.05 to 5.94 (see Analysis 4.2). Thus, children in foster care were more likely to be adopted than were children in kinship care. The test of heterogeneity was significant for this outcome (p < .001; I2 = 98.5%).
There was a statistically significant overall effect size for the four studies (Berrick 1999; Testa 1999; Testa 2001; Zimmerman 1998) that reported sufficient bivariate data to generate effect size estimates for guardianship. Specifically, the overall effect size estimate was OR = .26 with a confidence interval of .10 to .72 (see Analysis 4.3). Thus, children in kinship care were more likely to have relatives assume legal custody than were children in foster care. The test of heterogeneity was significant for this outcome (p < .001; I2 = 93.1%).
There was a statistically significant overall effect size for the seven studies (Barth 1994; Berrick 1999; Sivright 2004; Smith 2002; Smith 2003; Testa 2001; Zimmerman 1998) that reported sufficient bivariate data to generate effect size estimates for the still in placement outcome. Specifically, the overall effect size estimate was OR = 2.24 with a confidence interval of 1.66 to 3.03 (see Analysis 4.4). Thus, children in kinship care were more likely to still be in care than were children in foster care. The test of heterogeneity was significant for this outcome (p < .001; I2 = 88.0%).
Educational Attainment
There were four studies (Berrick 1994; Brooks 1998; Metzger 1997; Sripathy 2004) that reported sufficient bivariate data to generate effect size estimates for the repeated a grade outcome. Although the overall effect size was in favor of children in kinship care, the effect was not statistically significant. Specifically, the overall effect size estimate was OR = .67 with a confidence interval of .43 to 1.05 (see Analysis 5.3). However, the analysis could not rule out zero as a likely population value. The test of heterogeneity was not significant for this outcome (p = .10; I2 = 51.9%).
Family Relations
There were five studies (Chapman 2004; Chew 1998; Davis 2005; Strijker 2003; Surbeck 2000) that reported sufficient bivariate data to generate effect size estimates for the continuous attachment outcome. Although the overall effect size was in favor of children in kinship care, the effect was not statistically significant. Specifically, the overall effect size estimate was g = -.01 with a confidence interval of -.30 to .28 (see Analysis 6.1). However, the analysis could not rule out zero as a likely population value. The test of heterogeneity was significant for this outcome (p = .09; I2 = 50.2%).
There were three studies (Cole 2006; Jenkins 2002; Mosek 2001) that reported sufficient bivariate data to generate effect size estimates for the dichotomous attachment outcome. Although the overall effect size was in favor of children in kinship care, the effect was not statistically significant. Specifically, the overall effect size estimate was OR = .88 with a confidence interval of .33 to 2.30 (see Analysis 6.3). However, the analysis could not rule out zero as a likely population value. The test of heterogeneity was not significant for this outcome (p = .12; I2 = 52.7%).
Service Utilization
There was a statistically significant overall effect size for the nine studies (Berrick 1994; Bilaver 1999; Clyman 1998; Jenkins 2002; Metzger 1997; Scannapieco 1997; Sivright 2004; Sripathy 2004; Tompkins 2003) that reported sufficient bivariate data to generate effect size estimates for mental health service utilization. Specifically, the overall effect size estimate was OR = 1.69 with a confidence interval of 1.18 to 2.42 (see Analysis 7.1). Thus, children in foster care were more likely to receive mental health services than were children in kinship care. The test of heterogeneity was significant for this outcome (p < .001; I2 = 97.2%).
There were four studies (Bilaver 1999; Clyman 1998; Scannapieco 1997; Tompkins 2003) that reported sufficient bivariate data to generate effect size estimates for physician service utilization. Although the overall effect size was in favor of children in foster care, the effect was not statistically significant. Specifically, the overall effect size estimate was OR = 2.93 with a confidence interval of .46 to 18.59 (see Analysis 7.3). However, the analysis could not rule out zero as a likely population value. The test of heterogeneity was significant for this outcome (p < .001; I2 = 99.3%).
Multivariate analyses
As studies that reported multivariate data controlled for covariates such as age at placement, gender, ethnicity, socioeconomic status, geographic region, behavior and health problems, placement reason and history, and caregiver variables, they potentially provide a stronger level of evidence regarding the effect of kinship care on child welfare outcomes. Thus, results from the weaker quasi-experimental designs comprising the meta-analytical data could also be considered stronger evidence if corroborated by the multivariate results which are summarized in the Outcomes for Studies with Multivariate Analysis Table (Table 1). It should be noted that some studies reported both bivariate and multivariate data and were included in both analyses.
Overall, the multivariate data generally support the results generated from the meta-analyses. For example, two studies reporting multivariate behavior problems data (Bennett 2000; Holtan 2005) found that children in kinship care had significantly lower CBCL scores than did children in foster care. Surbeck 2000 and Zima 2000 did not find a significant difference between the groups and did not report the direction of the effect. Furthermore, three studies reporting multivariate adaptive behaviors data (Belanger 2002; Bennett 2000; Brooks 1998) found that children in kinship care had significantly greater adaptive behaviors than did children in foster care. Again, Zima 2000 did not find a significant difference between the groups on this outcome and did not report the direction of the effect. Three studies reporting multivariate data on mental health service utilization (Clyman 1998; Leslie 2000a; McMillen 2004) found that children in foster care were significantly more likely to utilize services than were children in kinship care. For mental health, one study reporting multivariate data (Metzger 1997) found that children in kinship care had significantly greater well-being than did children in foster care.
The greatest amount of multivariate data was reported for the permanency outcomes. Similar to the nonsignificant meta-analysis results for reunification, the findings from the nine studies reporting multivariate data also were inconclusive. Specifically, three studies (Berrick 1999; Courtney 1996a; Grogan-Kaylor 2000) found that children in kinship care were more likely to reunify and two studies (Connell 2006a; Courtney 1996b) found that children in foster care were more likely to reunify. Courtney 1997a and Frame 2002 found no significant difference between the groups on reunification and did not report the direction of the effect. Wells 1999 and Zimmerman 1998 also found no significant difference between the groups on reunification, but reported that children in foster care reunified at a lower rate. As for adoption, two studies reporting multivariate data (Barth 1994; Courtney 1996b) found that children in foster care were significantly more likely to be adopted than were children in kinship care, although Courtney 1996a found that children in kinship care were more likely to be adopted. Furthermore, Connell 2006a found no significant difference on adoption and reported an identical risk ratio for both groups. Lastly, two studies reporting multivariate data (Berrick 1999; Smith 2003) found that children in foster care were significantly less likely to still be in care than were children in kinship care.
The studies reporting multivariate data also provided evidence for some of the outcomes that had insufficient data for effect size calculation. For example, the most compelling evidence from the multivariate analyses was for re-entry, in that all seven studies (Berrick 1999; Courtney 1995; Courtney 1997a; Frame 2000; Frame 2002; Jonson-Reid 2003; Wells 1999) reported that children in kinship care were significantly less likely to re-enter care than were children placed in foster care. Furthermore, all three studies that reported multivariate placement disruption data (Chamberlain 2006; Connell 2006b; Testa 2001) found that children in kinship care were less likely to disrupt from placement than were children in foster care. For the safety outcomes, two studies reporting multivariate data (Benedict 1996a; Zuravin 1993) found that children in kinship care were less likely to experience institutional abuse than were children in foster care. The multivariate results were inconclusive for recurrence of abuse, as Jonson-Reid 2003 found that children in kinship care were less likely to experience recurrence of abuse than were children in foster care, while Fuller 2005 found that children in kinship care were more likely to experience recurrence of abuse than were children in foster care.
Bivariate analyses
As summarized in the Outcomes for Studies with Bivariate Analysis Table (Table 2), there were several studies that reported findings from bivariate analyses but did not report sufficient information for effect size calculation. Typically, these studies reported nonsignificant findings in the narrative but did not include the relevant data in a table. For example, two studies (Landsverk 1996; Sripathy 2004) found no difference between children in kinship care and foster care on the level of behavior problems as measured by the CBCL. However, Berrick 1994 confirmed the results from the meta-analysis and multivariate analysis in that children in kinship care had significantly lower scores on the total problems scale of the CBCL than did children in foster care. As for adaptive behaviors, Sripathy 2004 found no difference between children in kinship care and foster care on the level of adaptive behaviors as measured by the CBCL. For length of placement, two studies (Scannapieco 1997; Zimmerman 1998) found that children in foster care had significantly shorter lengths of stay than did children in kinship care, while Metzger 1997 found that children in kinship care had significantly lower lengths of stay than did children in foster care. Lastly, Zimmerman 1998 found no difference between the groups on re-entry rates.
Sensitivity analyses
Sensitivity analyses comparing studies with high attrition and low attrition were planned but were not conducted because attrition rates could not be accurately determined for the quasiexperimental studies included in the review. Specifically, all of these studies were posttest only, so there often was incomplete data on how many children were originally placed in kinship or foster care and no pre-measures to indicate how many children “dropped out” of the study by the time of the post-measures data collection. There were missing data in some of the studies, in that multiple measures had different sample sizes, presumably because data were either not available from case files or not collected. However, the missing data is presumed to be missing at random, thus no sensitivity analysis is warranted. Furthermore, sensitivity analyses comparing studies with low and high risk for attrition bias were not conducted because only two studies were rated high risk.
Sensitivity analyses comparing studies with child self-reports and parent/teacher/caregiver reports were planned for the review, but were not conducted because of the lack of such comparisons for the included outcomes. For example, there were four studies that measured well-being by child self-reports, but only one study that also measured it using caregiver reports. Furthermore, there was only one study each measuring psychiatric illness, family relations, and behavior problems by child self-report. Lastly, three studies used child selfreports for educational attainment outcomes, but there was only one each for test scores, graduation, and grade level; thus comparisons were not possible with the studies that used other measures.
Sensitivity analyses comparing studies that controlled for confounders with those that did not was not possible using statistical techniques because of differences in the type of data reported. Specifically, the studies that controlled for confounders used multivariate analyses rather than matching (except for Rudenberg 1991; Testa 2001). As such, much of the multivariate data was reported as correlation and beta coefficients or odds and risk ratios. Thus, these data could not be used in the meta-analyses to generate multivariate effect sizes to compare with the bivariate data effect sizes. However, we employed vote counting for the multivariate studies to provide some comparison with the results from the bivariate studies. In addition, sensitivity analyses comparing studies with low and high risk for selection bias were not conducted because only five studies were rated low risk and no more than two studies shared similar outcomes.
Subgroup analyses
There were insufficient data to examine different effects of the intervention by gender, ethnicity, and age at placement. Specifically, only Holtan 2005 reported outcome data by gender for each placement type, only Smith 2002 reported outcome data by ethnicity for each placement type, and no studies reported outcome data by age at placement for each placement type.
5 Discussion
5.1 SUMMARY OF MAIN RESULTS
Based on a preponderance of the available evidence, it appears that children in kinship care experience better outcomes in regard to behavior problems, adaptive behaviors, psychiatric disorders, well-being, placement stability, and guardianship than do children in foster care. Furthermore, there was no detectable difference between the groups on reunification, length of stay, family relations, or educational attainment. However, children placed with kin are less likely to achieve adoption and utilize mental health services while being more likely to still be in placement than are children in foster care. The multivariate results generally support these findings while indicating that children in kinship care are less likely to re-enter out-of-home care or have a disrupted placement than are children in foster care. However, these conclusions are tempered by the pronounced methodological and design weaknesses of the included studies and particularly the absence of conclusive evidence of the comparability of groups. It is clear that researchers and practitioners must do better to mitigate the biases that cloud the study of kinship care.
Although this review supports the practice of treating kinship care as a viable out-of-home placement option for children removed from the home for maltreatment, policies mandating kinship placements may not always be in the best interest of children and families. Professional judgment from child welfare practitioners also should be used to assess the individual needs of children and the ability of kin caregivers to attend to these needs.
5.2 OVERALL COMPLETENESS AND APPLICABILITY OF EVIDENCE
Because all of the effect sizes are small to medium in magnitude, the findings generated from this review may have limited practical consequences. However, the applicability of the evidence is still worth considering, especially for the key outcomes. For example, the lack of a baseline measurement of initial behavioral functioning makes ambiguous the conclusion that children in foster care have lower levels of current behavioral functioning. Furthermore, caregiver reports may be biased because foster parents have more incentive to report behavioral and mental health issues, whereas relatives are more apt to view the behavior as acceptable and thus less likely to report it as problematic.
The mixed findings for the permanency outcomes could be interpreted in the context that long-term kinship care arrangements satisfy the definition of permanency in many countries, as kinship caregivers are allocated the parental rights for a child. Thus, an undesirable outcome (i.e. re-entry remaining in care) might actually be desirable if the kinship care placement is considered to be safe and stable. Adoption and guardianship are secondary permanency goals, which are considered only after reunification has been ruled out. Furthermore, these permanency outcomes are fundamentally dependent on the public and legal policy of individual countries. For example, adoption is not a viable permanency option in many countries outside of the U.S. including Australia, Israel, Netherlands, and the Nordic nations.
The commonly held idea that foster parents are more “system involved” may explain the greater propensity for children in foster care to receive mental health services. Furthermore, the training and supervision of foster parents may contribute to the higher identification of mental health problems, and as such contribute to higher levels of service utilization. The lower licensure rate for kin caregivers may be another factor in the unequal receipt of services for children in kinship care. However, the greater likelihood for children in foster care to utilize mental health services may have less to do with the type of placement and more to do with these children having a greater need for services.
5.3 QUALITY OF THE EVIDENCE
The major limitation encountered in this systematic review is the weak standing of quantitative research on kinship care (Cuddeback 2004). Specifically, the “differences between the children who enter kinship care and those who enter nonkinship care” lead to a lack of confidence regarding the comparability of groups and the subsequent lack of control over contaminating events such as family preservation services (Barth 2008, p. 218). In general, the included studies also have moderate to high risks of performance, detection, report, and attrition bias, which compromise the tenability of the findings from the systematic review.
Another concern regarding the quality of evidence is the potential misalignment between the intervention and child welfare outcomes, in that the fullest representation of the effects of kinship care has yet to be truly measured (Cuddeback 2004). When compared to traditional foster care, in which the relationship between foster parents and the “system” is more standardized, the effect of kinship care may be more difficult to detect. For example, there seemingly is a lack of implementation fidelity within and across countries in regard to kinship care implementation. Furthermore, kinship placements, especially with unlicensed caregivers, are often more private and out of the control of child welfare agencies than are foster placements. The concepts, terminology, and outcomes typically ascribed to out-of-home care may not always be appropriate for kinship placements. As a result of these limitations, it is more appropriate to research kinship care after it has been fully and consistently integrated into the fabric of child welfare policy and practice.
5.4 POTENTIAL BIASES IN THE REVIEW PROCESS
One potential bias in the review process is that the usefulness of the meta-analysis results is weakened by challenges confronted during the effect size calculations. Specifically, the heterogeneity statistic was significant for 10 of the 16 outcomes, which indicates that the effect sizes were not always consistent within the same construct. In addition, bivariate data were not reported in every study, which restricted the meta-analysis of some outcomes to the bare minimum of three studies and eliminated other outcomes from consideration. Another potential bias is that many studies analyzed a small sample of children, while others utilized a much larger data set. As a result, studies with large sample sizes essentially eliminated the effects from studies with small sample sizes for certain outcomes (e.g. psychiatric disorders, well-being, placement settings). In addition, publication bias is always a concern with systematic reviews. It was not plausible to investigate the presence of publication bias in this review because the results were widely distributed across so many studies. However, the review included unpublished reports, dissertations, and theses, along with many studies that reported nonsignificant results.
5.5 AGREEMENTS AND DISAGREEMENTS WITH OTHER STUDIES OR REVIEWS
The results of this review are in strong agreement with the “substantive synthesis of research” conducted by Cuddeback 2004.
6 Authors’ Conclusions
6.1 IMPLICATIONS FOR PRACTICE
Although the implications of this review depend on how individual countries interpret the results, several recommendations for social work professionals and policymakers did emerge. If the goal of kinship care is to enhance the behavioral development, mental health functioning, and placement stability of children, then the evidence base is supportive. However, the findings from the review do not support implementing kinship care solely to increase the permanency rates and service utilization of children in out-of-home care. The primary implication for practitioners to consider is whether kinship placements would be even more effective with increased levels of caseworker involvement and service delivery (Geen 2000). However, the potential benefits of greater financial and therapeutic support must be weighed against the independence that some kin caregivers demand. Relatedly, the main implications for policymakers is whether licensing standards should be required for kin caregivers (Geen 2000), and whether additional financial resources should be made available to these providers (Hornby 1996).
On the other hand, there may be a cost-effectiveness component to placing children with relatives in light of the comparable well-being and permanency outcomes and lower payments and fewer services offered to kin caregivers. As such, this could play an important role in how child welfare agencies view their current approach to kinship care. That being said, foster care should continue to be an essential out-of-home care option, as children in these placements also experience positive outcomes and appropriate kinship placements are not always available.
6.2 IMPLICATIONS FOR RESEARCH
To address the major limitations of research on kinship care, there is a demand for studies that employ generalizable samples, equivalent groups, and repeated measurements (Berrick 1994). Cuddeback 2004 advocates for longitudinal designs to investigate the outcomes of children over time, the development of psychometrically sound instruments of family and child functioning that allow for more reliable comparisons across groups and studies, and greater emphasis on controlling and understanding selection bias through the use of emerging statistical models. Furthermore, the duration effect or the relationship between length of stay in out-of-home care and child welfare outcomes should be explored in greater depth. There also is a need to disaggregate the effects of kinship care across important subgroups of target participants, settings, and intervention variations. For example, there are few studies that reliably measure the effect of kinship care on caregiver outcomes (Gibbs 2000).
As for other topics, Testa 1992 calls for research on the financial implications of kin caregivers becoming licensed, while Cuddeback 2004 recommends studies that examine the relationship between certification and the provision of services to kin caregivers. Studies that focus on the educational outcomes of children in kinship care is certainly warranted, as education is essential to effectively integrating into adult life. In addition, research on informal and voluntary kinship care arrangements should be a top priority for social work researchers.
Qualitative research that explores the underlying dynamics of kinship care along with the factors associated with positive outcomes is a natural outgrowth of this systematic review. Specifically, investigating the lived experiences of different types of kin caregivers (e.g. grandparents, other relatives, family friends) would greatly enhance our understanding of this placement option.
As research on this topic is predominantly U.S. based, studies from other countries are sorely needed, especially as kinship care is increasing in popularity elsewhere in the western world. For example, the different permanency goals should be examined in greater depth to determine which outcome offers greater practical permanency to children removed from the home.
For kinship care to remain a viable option in the social work repertoire, researchers must work more closely with practitioners to design, implement, and disseminate innovative studies of the intervention. For example, new predictor variables and outcome measures should be included in data collection instruments to facilitate richer analyses on the effect of kinship care.
Lastly, the Methods for Future Updates Table (Table 4) displays methods such as sensitivity and subgroup analyses that were not conducted in this review but should be included in future updates.
Footnotes
7 Acknowledgements
Thank you to the Nordic Campbell Center, Danish National Institute of Social Research for funding this review. Thank you to Trine Bak Nyby, Krystyna Kowalski, Jane Dennis, Geraldine Macdonald, and Julia Littell for their timely feedback and generous support during the writing of the protocol and review. Thank you to Jo Abbott, Trial Search Coordinator for the Cochrane DPLPG, and Merinda McLure, Applied Human Sciences Librarian at Colorado State University (CSU), for assistance in developing and executing the search strategy for the review. Thank you to Toby Lasserson (Cochrane Airways Group, London, UK), Celia Almeida (Cochrane DPLPG, Bristol, UK), Soyna Curtis, and Professor Jelena Marinkovic and Dr. Jelena Marinkovic both from University of Belgrade, Serbia for translating the foreign language articles for the review. Thank you to Jeff Valentine and the content reviewers along with Brian Cobb and Jeffrey Gliner of CSU for their helpful suggestions. Special thanks to Keri Batchelder of CSU for her assistance in acquiring the studies, extracting data, and conducting quality assessments for the review. Thank you to Frank Ainsworth, Marianne Berry, Morten Blekesaune, and Amy Holtan, as this protocol incorporates elements of their jointly registered Cochrane and Campbell Collaboration protocol prepared in 2004 on the same topic.
Potential Conflicts of Interest
Marc Winokur, Amy Holtan, and Deborah Valentine have no vested interest in the outcomes of this review, nor any incentive to represent findings in a biased manner.