Abstract

BACKGROUND
Court Appointed Special Advocates (CASA) is a national network of nonprofit organizations that train and support community volunteers to serve as child advocates representing the interests of children in foster care in the United States (Piraino, 1999). For the past several decades, federal child welfare policy has prioritized three central goals for children who enter foster care: safety, permanency, and well-being (Carnochan, Samples, Lawson, & Austin, 2013; Courtney, Needell, & Wulczyn, 2004; Testa, 2008). The safety goal mandates that the child welfare system protect children from abuse and neglect while maintaining them in their families of origin whenever possible. The well-being goal mandates that the child welfare system increase the capacity of families to care for their children, and that children receive appropriate services to meet their educational, physical, and mental health needs. Finally, the permanency goal directs the child welfare system to attend to multiple objectives for children in foster care. Specifically, child welfare agencies must seek permanent, stable homes for children who enter foster care by reunifying children with their families of origin as quickly as possible, or by finding alternative permanent, legal homes for children who cannot be reunified with their families of origin. The permanency goal must be sought while maximizing placement stability by limiting the number of out-of-home placements that children experience while in the custody of the state.
With federal policy setting the contextual background, the CASA intervention seeks to support the dependency court system in attaining the safety, permanency, and well-being goals of children in foster care through the use of trained lay volunteers to serve as independent advocates for children. The CASA goal of improving child welfare outcomes under the domains of safety, permanency, and well-being is reflected in the National CASA Association's mission statement, which asserts that they promote volunteer advocacy “so that every abused or neglected child can be safe, establish permanence and have the opportunity to thrive” (“Mission Statement,” n.d.).
1.1 The Problem, Condition or Issue
CASA serves as voluntary adjunct to the functioning of the public child welfare system toward the goals of improving safety, permanency, and well-being outcomes for children who are placed in out-of-home care. The root problem that the child welfare system seeks to target is the maltreatment of children by their caregivers. Children are placed out-of-home care through the child welfare system when maltreatment results in harm or risk of harm that makes it unsafe for them to remain in their homes. Child maltreatment encompasses both abuse (acts by caregivers which result in harm or risk of harm) and neglect (failures to act by caregivers which result in harm or risk of harm).
The prevalence of child maltreatment is typically measured using official federal statistics on children who are reported to public child protection agencies. In 2013 (the most recent year for which there are statistics available), there were 3.5 million reports of child maltreatment (involving 6.4 million children) made to child welfare authorities in the U.S. (USDHHS, 2015). Of the children reported for alleged maltreatment, about 679,000 were found to be “victims,” meaning that their cases were determined to be valid (i.e., substantiated) reports of abuse or neglect, and some 255,000 children entered foster care (USDHHS, 2015). While these official figures represent a sizable number of children victimized by maltreatment in the U.S., these figures only encompass children whose cases were brought to the attention of public authorities; many more children are believed to be unreported victims of maltreatment each year. The most recent National Incidence Survey, which attempts to measure the actual prevalence of maltreatment in the population (not just those reported to child protection agencies), suggests that between 1.25 and 3 million children are actual victims of child maltreatment per year in the United States (Sedlak et al., 2010).
Maltreatment is most prevalent among young children, with 47 percent of documented child victims under age 6 (USDHHS, 2015). While it is common for multiple types of maltreatment to co-occur in reported child welfare cases, neglect is overwhelmingly the most common type of substantiated maltreatment (it is present in some 80 percent of all founded reports of maltreatment), followed by physical abuse (18%), sexual abuse (9%) and psychological abuse (9%)(USDHHS, 2015).
There are consistently observed differences in child welfare system contact by race, with African-American and Native-American children disproportionately likely to be reported, substantiated as victims, and placed in out-of-home care compared to children of other races (USDHHS, 2015). Minority overrepresentation in the child welfare system is presently the subject of intense debate and research, focused on whether this disproportionality is driven primarily by disparate risk in the underlying populations or by racial bias within the system (see, for example, Drake et al., 2011). The racial demography of the foster care population is a factor that may impact CASA service delivery, which has been critiqued for the noted lack of racial concordance between the population of CASA volunteers and the population of children they serve. Research on CASA volunteer demographics consistently shows that they are overwhelmingly (80-90%) White, in contrast to the foster care population, in which children of color are distinctly overrepresented (Caliber, 2004; Mensing, 2008; NCASAA, 2012a).
While the current goals of child welfare system (permanency, safety, and well-being) have been stable for the last two decades, the historical development of the U.S. child welfare system has been characterized by perennially evolving models of service, targets of intervention, and preferred outcome goals (Myers, 2008). Some prominent examples of major shifts in the functions of the child welfare system over time include debates over kinship vs. foster placement, prioritization of family preservation vs. child safety, extended attempts at reunification vs. faster termination of parental rights, and whether to employ a broad vs. narrow scope of intervention (Berrick, 2012; Myers, 2008; Testa, 2008). One of the more recent, and somewhat controversial, developments in child welfare service delivery involves interventions that do not involve “investigation” or a finding of maltreatment, collectively called differential response or alternative response strategies. In these approaches, full child welfare investigations are reserved for high-risk cases that pose direct threats to child safety, while lower-risk cases are targeted for “assessment” of family service needs in a manner that is voluntary and purportedly less punitive and adversarial than traditional child welfare intervention. In addition to these large-scale child welfare system changes and reforms that occur over time, there are also substantial variations in child welfare procedures and approaches across state and local jurisdictions (Berrick, 2012).
Despite the continual evolution and regional variation of priorities, policies, and practices in child welfare, the CASA intervention is able to maintain a stable intervention strategy and relationship with the child welfare system because they are typically only involved with cases that end with the placement of children into out-of-home care and that are associated with the opening of a dependency court case. In other words, though child welfare system approaches and methods of intervention can and do shift over time and place, the role of CASA is relatively unaffected by these changes. While the number of child maltreatment cases (including cases involving out-of-home placement) has been declining in the U.S. since the early 90's (Finkelhor & Jones, 2006), the demand for CASA volunteers over the same time period has continued on a growth trajectory as CASA agencies strive to provide services to an increasing proportion of children entering out-of-home care (NCASAA, 2011, 2013).
1.2 Description of the intervention
CASA volunteers are assigned to represent children in the dependency court system at the discretion of the individual judges who oversee their cases. While a judge can appoint a CASA to a child's case for any reason that he or she chooses, prior research suggests that judges may be more likely to assign CASA to cases that are especially complicated in some respect, such as cases with prior child welfare system involvement (Abramson, 1991; Caliber, 2004; Siegel et al., 2001), cases involving more severe maltreatment or higher levels of risk (Caliber, 2004; ORS, 2005; Waxman et al, 2009), or cases with parental substance abuse or multiple parental problems (Litzelfelner, 2000; Siegel et al., 2001). Though there are over 950 local CASA agencies in 49 states (NCASAA, 2013), there are jurisdictions not covered by a CASA program, so these services are not available to all foster children. Even where programs do exist, the demand for CASA services is routinely higher than the supply of available volunteers, so many children in the foster care system do not receive CASA services even where there are local CASA agencies. Nationally, less than half (36%) of children in foster care receive CASA services (NCASAA, 2013), while the balance of foster children receive only child welfare services without CASA representation.
CASAs are lay volunteers from the community who receive special training in mentorship, advocacy, foster care, and the child welfare system prior to being eligible to represent children in court cases. Once assigned to a child's case, CASA volunteers provide recommendations to the court independently of judges, attorneys, social workers, therapists, family members, and others who might be involved with the case. CASA volunteers typically serve one child at a time, and they often represent that child for the life of the case, which is until legal permanency is established (Piraino, 1999). Since 1977 when CASA was established in a single jurisdiction in Seattle, Washington, it has expanded to a network that includes 70,000 volunteers in 49 states, serving an estimated 237,000 children as of 2013, according to the National CASA Association (2013). Each local CASA agency is an independent non-profit organization; the National CASA Association is a national organization that provides training and curriculum support, technical assistance, recruitment, marketing, and funding support to state and local CASA organizations.
There are different variations (or models) of CASA representation in the dependency court process. The differences between these models are related to the relationship of the CASA volunteer to the role of the guardian ad litem (GAL). A GAL is an external party (often an attorney) appointed by the court to represent a child's legal interests in dependency court proceedings. Depending on the structure, preferences, and statutory requirements of local court jurisdictions, CASA advocates may serve as the GAL, or they may serve in a supplemental role (“friend of the court”) in addition to a GAL. The National CASA Association reports that 59% of existing CASA programs operate with a “friend of the court” model, 34% operate with a CASA-as-GAL model, and 7% use a team model combining components of both (NCASAA, 2010).
CASA is a broadly defined intervention. While some key components of representation are present across programs, there is wide variability across local agencies in terms of the activities that CASAs engage in to fulfill their roles as advocates (Piraino, 1999). The National CASA Association maintains a list of core program standards that CASA agencies must adhere to, but as independent organizations, local CASA programs may develop their own priorities, policies, and protocols as long as they fall within the broad program standards set by the National CASA Association. In the 2012 version of National CASA's local program standards, the roles and responsibilities to which volunteers are required to adhere are (NCASAA, 2012b, p. 30-31): Obtain facts of the case and the child by reviewing records and interviewing relevant individuals (the child, parents, social worker, etc.) Identify and advocate for the best interest of the child Act as a facilitator to seek cooperative solutions among parties Provide a report of findings and recommendations at every court hearing Appear at all court hearings and provide testimony as needed Maintain regular face-to-face contact with the child (once a month at minimum) Make recommendations to the court for needed services for the child and/or family Make permanency recommendations Monitor service plans and court orders for timely compliance Inform the court of important case information Work with community systems (such as mental health and educational systems) to advocate for the child's interests and needs Participate in case conferences with CASA staff Participate in training Maintain records about the case Return records to the agency after case closure
While these standards mandate certain actions, they leave much room for state and local agency-level variation in practices and procedures performed as part of the CASA intervention. CASA as an intervention involves many uncertainties in what is given and to whom, as described above. In addition to the lack of a uniform, prescriptive “model” of CASA activities and the lack of standardized guidelines specifying who receives CASA services, there may also be substantial variations in when the intervention is given and for how long. The appointment of a CASA volunteer can occur at any point during the case. Thus, there is no standard for when the intervention is administered (because CASAs can be assigned at any point in the life of a case), nor how long it lasts (because cases are widely variable in duration, ranging from days to years).
There may also be substantial differences in other key CASA processes that may impact the nature of the intervention being delivered at the local level, including how volunteers are recruited, how volunteers are matched to the child(ren) that they will represent, and what subpopulations of foster children may be targeted for CASA services at the organizational level.
1.3 How the intervention might work
There are many theorized benefits that CASA representation may impart. CASA volunteers are typically more stable in their representation than other professionals in high-turnover roles with whom a foster child may interact, such as social workers, service providers, and attorneys (Brennan et al., 2010; Clark, 1988). CASA also brings independent community oversight into otherwise opaque court processes (Collins-Camargo et al., 2007; Schorr & Kahn, 2000). Finally, CASA workers maintain very low “caseloads,” typically representing no more than one or two children at a time (NCASAA, 2013), presumably allowing for more individualized attention on which to base case recommendations.
Aside from these process-related features of CASA, there remains the issue of whether CASA is an intervention that improves child welfare case outcomes under the key child welfare domains of safety, permanency, and well-being. It is these outcomes that are of concern in the present review.
On their website (“Evidence of Effectiveness,” n.d.) and in publications (e.g., NCASAA, 2013), the National CASA Association asserts that foster children who are assigned a CASA volunteer have a host of improved “key outcomes” compared to foster children who are not assigned a CASA volunteer. They assert that children with a CASA: (1) are more likely to be adopted; (2) are less likely to re-enter foster care after leaving; (3) are less likely to remain in long-term foster care; (4) are more likely to have a permanency plan; (5) receive more services; (6) spend less total time in foster care; (7) have fewer placements while in foster care, and; (8) exhibit better well-being outcomes (i.e., better school performance and more protective factors).
Michael Piraino, the current CEO of the National CASA Association, has described the unique contributions of CASA volunteers that are believed to connect their activities to the improved outcomes of the children they represent (Piraino, 1999). Piraino argues that the value that a CASA volunteer adds to a case, which translates to improved child and case outcomes, is that the CASA performs necessary functions that are not covered under the roles of the other professionals involved in a dependency court case. He describes CASA as a necessary augment to attorney representation because neither of these two roles alone is sufficient to cover the essential activities that result in optimal attainment of child welfare goals.
In this framework of CASA functioning, children's needs are optimally served, and their goals optimally met, by having both a volunteer advocate (CASA) and an attorney involved in each foster child's case, with each having equal status and input in the court's processes and decision-making (Piraino, 1999). The necessity of these dual functions to serve children, according to Piraino, is that children involved in dependency court cases have both legal and non-legal representation needs. He argues that while attorneys meet the legal representation needs of children, due to lack of time and resources (stemming from large caseloads), attorneys and child welfare caseworkers are often not inadequately resourced to attend to the non-legal representation needs of the children they serve. This is the gap that CASA fills and that is purported to result in improved outcomes for the children who receive CASA services. In other words, attorneys bring legal knowledge and advice on behalf of a child, but this alone is asserted to be insufficient because someone is still needed to provide representation of non-legal needs of the child. CASA volunteers bring representation of children's non-legal needs and interests to the court process through unique activities that do not fall under the purview of attorney representation.
Some of the specific activities of CASA volunteers that are theorized to promote positive outcomes by uniquely representing the non-legal needs of the child are fact-finding (information gathering) about the case, attendance and reporting of facts at court hearings, and advocacy in the court's processing of the case (Piraino, 1999). Piraino notes that some of these roles are actually potentially incompatible with the attorney's role in a case; attorneys may be limited in their fact-finding activities or they may not be able to testify in court due to ethical constraints of their profession, and they may not have adequate training to assess the social (i.e., non-legal) needs of children. For these reasons, the functions that CASA volunteers bring to a case are asserted to be unique, necessary, and crucial to the process of representing the full breadth of a child's needs in dependency court processes (Piraino, 1999).
Summarizing this description of how CASA services are theorized to result in improved outcomes for foster children, Piraino states, “By more effectively combining these forms of representation [legal and non-legal], with appropriate regard for the independence and unique contributions of each, all decision-makers in the child protection system can be better equipped to arrive at decisions that help each child find a safe, permanent home as quickly as possible” (pp. 69-70).
In discussing the theorized ways in which the intervention might be effective, it also bears consideration that there may be negative effects of the intervention. In their analysis of different methods of child welfare advocacy, Litzelfelner and Petr (1997) note the potential for professional conflict that could result from child welfare caseworkers perceiving CASA volunteers as adversaries when their recommendations do not converge, especially given that CASA volunteers are unlikely to have the level of training on maltreatment or the educational background (e.g., a college degree in social work) that child welfare caseworkers may be required to have. Speaking to the limited training and lack of educational requirements for CASAs, Litzelfelner and Petr (1997) state, “Some may also question the legitimacy of a nonprofessional volunteer advising the court on such important decisions as when a child can safety return home from foster care” (p. 400). Although there is no known empirical research that demonstrates that the potential conflicts between caseworkers and CASAs impact foster child outcomes, it is not inconceivable that there could be consequences for children when CASA/caseworker relationships are not agreeable.
Aside from these observations, there are also voices among child welfare stakeholders that do not regard CASA as a beneficial intervention. The National Coalition for Child Protection Reform [NCCPR] is a non-profit advocacy organization that is deeply critical of CASA's role in the child welfare system. The NCCPR, which includes prominent child welfare legal scholars among its supporters and board members, is opposed to the entire notion of the CASA model of intervention, asserting that it actively harms the children it is meant to help (NCCPR, 2010). Some of their criticisms include: the unfavorable findings of one CASA outcome study (Caliber, 2004) in regard to length of time in care and child safety, the perceived role of CASA in maintaining racial disproportionality in the child welfare system due to the often incongruous demographics (race, income, etc.) of volunteers and foster children, the perception that CASA as an organization is opposed to the reunification of children with their families of origin, and the assertion that CASA volunteers use only their own personal (and biased) judgments to decide what is in the best interests of children. While the positions of the NCCPR are not aligned with the prevailing positive regard for CASA gleaned from survey research with child welfare and judicial professionals (Berliner & Fitzgerald, 1998; Collins-Camargo et al., 2005; Leung, 1996; Litzelfelner, 2008; ORS, 2005; Weisz & Thai, 2003), it bears consideration that an intervention without demonstrated effectiveness may also be associated with undesired or harmful effects.
1.4 Why it is important to do this review
The purpose of this review is to synthesize the existing research on CASA's effectiveness as an intervention for improving child welfare outcomes pertaining to safety, permanency, and well-being. Two of the authors of this proposed study previously published an initial scoping review and critical analysis of the literature on CASA outcomes (Lawson & Berrick, 2013). Their review found that assertions about the effectiveness of CASA as an intervention for improving child welfare case outcomes are tentative at best due to equivocal, contradictory findings on many outcomes of interest, and widespread methodological weaknesses prominent among existing studies. The methodological quality of the literature base is generally poor, with selection bias, small sample sizes, and questionable design, analysis, and interpretation prevalent among studies.
Lawson and Berrick (2013) concluded that the most consistently reported findings from existing CASA studies are that, compared to children without a CASA, children with CASA representation may receive more services, may have fewer foster placements while in care, and may be more likely to be adopted (as opposed to attaining other permanency outcomes). However, despite the frequency and consistency with which these findings are reported, the methodological flaws in the studies that produced them prohibit the inference of strong conclusions on these outcomes.
Most saliently, selection bias appears to be a vexing problem inhibiting confidence in any findings on these outcomes. Research on baseline characteristics of children who receive CASA representation compared children who do not consistently show systematic differences between these groups. In the aggregate, cases that receive CASA services are more severe and complex than other cases, making it impossible to deduce whether differences in outcomes are due to the impact of the CASA intervention or whether they are merely artifacts of pre-existing differences between comparison groups. Generally, random assignment to condition is the methodological strategy for eliminating selection bias. However, to date, only one randomized trial has ever been completed to examine the impact of CASA on case outcomes, and that study (Abramson, 1991) is hampered by a multitude of grave methodological flaws, including small sample sizes, baseline differences in comparison groups, treatment diffusion, and violation of data independence based on assignment to condition by family group rather than by individual.
Many outcome variables of interest have inconsistent findings across studies. For example, in regard to case duration, some studies find that CASA cases stay in the system longer compared to cases without CASA (Condelli, 1998; Smith, 1993), while others report that CASA cases are of shorter duration (Calkins & Millar, 1999; McRoy & Smith, 1998; Powell & Speshock, 1996) and others find no differences in duration between CASA and non-CASA cases (Caliber, 2004; Litzelfelner, 2000; Poertner & Press, 1990; Siegel et al., 2001). Similarly, findings are contradictory in relation to whether children with CASA representation experience fewer placements (Calkins & Millar 1999; Litzelfelner, 2000), more placements (McRoy, 1998; Smith, 1993), or equivalent numbers of placements (Brennan et al., 2010; Caliber, 2004; Leung, 1996; Poertner & Press, 1990; Siegel et al., 2001) while in foster care compared to children without CASA. Finally, permanency outcomes are likewise murky across studies; the two studies that have produced findings indicating that children receiving CASA are more likely to be adopted have major methodological weaknesses, most notably the exceptionally small subsamples of cases attaining an outcome of adoption (Abramson, 1991, n=5; Poertner & Press, 1990, n=13) with which to make between-group comparisons.
While the Lawson and Berrick review contributed a unique critical assessment of the empirical support for claims of CASA as an evidence-based intervention, it is limited in that it was not a systematic review (because it did not locate all unpublished studies) and it did not offer effect size computations. Other than the review by Lawson and Berrick (2013), one other prior review of CASA outcome studies has been conducted. Youngclarke, Ramos & Granger-Merkle (2004) published a narrative review examining the impact of CASA on child welfare case processes and outcomes. In this review, Youngclarke and colleagues attempted to systematically locate all published and unpublished studies that had comparatively examined the effectiveness of CASA programs in the U.S. since 1977. Their search process resulted in 20 included studies, and their review concluded that there were tentative indications that children with CASA representation did as well or better than children without CASA on a number of process and outcome variables. While there are certainly some strengths of this prior review, notably the systematic search process that yielded numerous unpublished studies, the review is almost 10 years old and is not without shortcomings.
First, while the authors of the Youngclarke et al. review used explicit inclusion and exclusion criteria to examine all published and unpublished comparison studies on CASA up to the date of its publication in 2004, several comparative studies of CASA outcomes have been conducted since then, including one study with a large sample size that examines relevant outcomes of interest (Caliber, 2004). In addition, the Youngclarke et al. review included two studies that do not appear to meet the stated inclusion criteria documented in their methods: one study (Duquette and Ramsay, 1986) did not separate the CASA group from the non-CASA (attorney-only) group in the analysis, and one study (Weisz & Thai, 2003) had no objective measures of child outcomes (only subjective survey responses). Further, the Youngclarke et al. review is narrative, with no effect size computations or meta-analytic synthesis of effects across studies. The included studies were synthesized using a “vote-counting” approach, and the review contains numerous instances of findings being “counted” as evidence even where the original study showed no statistical differences between groups.
Finally, Youngclarke and colleagues' conclusions appear to overstate the evidence in some instances. For example, the authors conclude that reduced recidivism/re-entry to foster care is “the most profound finding” of their review. However, only three studies were evaluated on this outcome variable (Abramson, 1991; Poertner & Press, 1990; Powell & Speshock, 1996). Two of these (Abramson, 1991; Poertner & Press, 1990) found no statistical difference between the CASA and non-CASA groups, yet all were “vote-counted” as evidence of effectiveness and the authors concluded that the impact of CASA services on recidivism was “consistent” and “large.” Thus, although a prior “systematic review” has been conducted, and though the authors of that review caution that the findings are tentative, their assertions of CASA effectiveness in improving outcomes does not seem to be adequately supported by the evidence presented in their review.
The proposed review aims to update and improve upon the prior published work on CASA effectiveness by using systematic review methods and meta-analytic techniques to provide a systematic, transparent, and less biased estimate of the effects of CASA on key child welfare outcomes to inform practice and policy. It is hoped that by synthesizing the existing research using systematic methods, and reducing bias through exclusion criteria and statistical controls, some of the inconsistencies and contradictions of prior research may resolve and produce a clearer picture of the average treatment effects of CASA intervention on the outcomes of interest.
Implications for policy and practice
The effectiveness of CASA as an intervention for improving child welfare case outcomes is an important question that has implications for policy and practice. In any environment, CASA agencies struggle to maintain adequate funding, as evidenced by the limited capacity that often results in wait-lists to receive services. In the current economic climate, CASA agencies are fighting all the harder for limited funding. At the same time, funders are increasingly calling for demonstration that programs are effective in improving targeted outcomes. Currently, there is no clear evidentiary basis to indicate whether CASA is an effective intervention program for improving child welfare outcomes in the domains of safety, permanency, and well-being.
The National CASA Association website lists five “Strategic Objectives” that they are dedicated to achieving. Among these objectives are that “every court in the United States recognizes that a CASA/GAL volunteer is essential for a successful outcome,” that “every potential donor” places CASA “at the top of their priority list,” and that “every government official” places CASA representation “at the top of their agenda” (“Strategic Objectives,” n.d.). These objectives make very strong assertions about the effectiveness of CASA at improving case outcomes, and about the appropriateness (in fact, the necessity) of increased funding for expansion of their services.
Currently, available research does not support the strong assertions that CASA makes about the known effectiveness of their services in improving outcomes. If further research, such as this proposed systematic review and meta-analysis, could clarify that CASA is in fact an empirically-supported intervention for helping courts achieve the core mandated goals of the child welfare system, these strategic objectives may be viewed as reasonable goals. However, if further research indicates that CASA intervention has not been shown to be effective at contributing to improved outcomes under the domains of interest, these strategic objectives may be premature at best, or possibly even misguided. There may be better uses of scarce resources toward improving targeted outcomes if research does not suggest CASA is an empirically-supported intervention.
This review is positioned to shed clearer light on the effects of CASA intervention as measured by the available research literature, and may potentially inform policy and funding decisions to allocate limited child welfare resources most effectively.
OBJECTIVES
The objective of this review is to clarify whether Court-Appointed Special Advocates (CASA) is an effective intervention for improving key outcomes for foster children in the U.S. by gathering, summarizing, and integrating the results of existing empirical research. The review will examine studies that compare the outcomes of foster children who receive CASA services with the outcomes of foster children who are served only by traditional child welfare services without CASA representation. The review is intended to produce more reliable estimates than have previously been available on the treatment effects of CASA on foster child case outcomes through the synthesis of individual study findings.
The specific questions that will be answered in pursuit of this objective are listed below under their domains of interest: Permanency Outcomes Are children who have a CASA volunteer less likely to be in long-term foster care than children without a CASA volunteer? Are children who have a CASA volunteer more likely to attain a permanency outcome of adoption than children without a CASA volunteer? Are children who have a CASA volunteer more likely to attain a permanency outcome of reunification than children without a CASA volunteer? Case Duration Do children with a CASA volunteer spend less time in the dependency court system than children without a CASA volunteer? Do children with a CASA volunteer spend less time placed out-of-home than children without a CASA volunteer? Placement Stability Do children with a CASA volunteer who are placed out-of-home have fewer placements while in care than children without CASA volunteer? Recidivism Are children with a CASA less likely to be re-referred to child welfare, or to re-enter foster care after leaving, than children without a CASA?
The objectives of this study relate directly to the policy and practice implications discussed in the previous section. Since National CASA asserts that their effectiveness in improving safety, permanency, and well-being outcomes should drive the prioritization of increased funding and expansion of services, it is reasonable and important to ask whether the existing evidence supports the claims that CASA services are linked to improvements in the target outcomes of interest.
METHODOLOGY
3.1 Criteria for including and excluding studies
3.1.1 Types of study designs eligible for inclusion
To be included in the review, studies must use an experimental or quasi-experimental design to examine the impact of CASA on at least one of the outcome variables of interest. Studies must include a comparison of treatment and control conditions to which child subjects are randomly or non-randomly assigned. Both prospective and retrospective studies will be eligible for inclusion, and this feature will be coded as a variable for analysis. In eligible studies, the treatment condition will reflect cases to which a CASA/lay volunteer was appointed in addition to the child welfare services that all foster children receive. The control condition will reflect cases that receive only the “standard” intervention of child welfare services without the assignment of a CASA/lay volunteer.
The demonstrated differences in case complexity between treatment and control conditions in existing studies indicate a strong likelihood of selection bias resulting in non-equivalent groups in many studies. Thus, to be eligible for inclusion, baseline equivalence of the analytic groups must be demonstrated, or statistical controls must be used in the analysis to control for between-group differences at baseline. Extant CASA studies use different combinations of variables to examine baseline equivalence of treatment and control groups (see Lawson & Berrick, 2012, for a table outlining variables used to assess for group differences by study).
To be eligible for inclusion in the review, studies needs to exhibit baseline equivalence on the outcome variables that are of interest to this review and on characteristics that could otherwise impact the outcomes, such as maltreatment type or severity, prior child welfare involvement, race/ethnicity, and child age.
The review will exclude any single group, pretest-posttest designs or other non-experimental studies lacking a comparison (non-CASA) group. Qualitative studies will be excluded.
3.1.2 Types of participants
For inclusion in this review, studies must involve children age 0-17 who have entered foster care through the child welfare system due to maltreatment.
“Foster care” in this context is a broad term that includes children in the custody of the state due to child welfare involvement under the dependency court system. It can include children placed in many different out-of-home settings, including foster family homes, congregate care homes, residential treatment facilities, or kinship placements. The foster care designation can sometimes occasionally even involve children who are residing in the care of their families of origin (for example, when there is a trial reunification period while a child remains in the legal custody of the state). All of these placement types will be considered “foster care” as long as subject children are in the formal legal custody of the state through the child welfare system with dependency court oversight.
Studies involving children who are in out-of-home care for reasons other than maltreatment will be excluded from the review, regardless of whether they receive CASA services. This will exclude children who are placed out-of-home informally without court custody and oversight, as well as those placed out-of-home through other legal mechanisms, such as juvenile delinquency court rather than the child welfare system.
If there are studies in which the sample is mixed between children who meet inclusion criteria and those who do not (for example, the sample also includes children receiving CASA due to juvenile delinquency court involvement), the study will be considered eligible for inclusion if data are provided for the subsample of participants who meet participant criteria.
3.1.3 Types of interventions
This review will focus on CASA programs and similar lay court volunteer programs. The review will include any studies that examine the effectiveness of CASA (or the CASA model of lay volunteer court representation in child welfare foster cases) in relation to any of the outcome variables of interest. For purposes of operationalizing the definition of the intervention, the review will include studies of interventions that are part of the CASA network of agencies, or other similar interventions that use lay volunteers from the community who receive special training to represent and advocate for foster children in dependency court cases as assigned by local judicial officials.
Two prior studies (Abramson, 1991; Wert, 1986) examine CASA-style programs that were not yet called CASA when the studies were conducted, yet they are considered completely equivalent to CASA, both by the National CASA Association and by prior reviewers. In addition, some agencies that are a part of the CASA network elect not to use CASA in their local agency names (for example, Child Advocates Inc. is the name of the CASA-affiliate in Harris County, Texas). Studies using the CASA model of intervention as defined above, whether or not they are called CASA or part of the CASA organization, will be included provided that they meet the other inclusion criteria of the study.
The review will exclude studies that involve other youth mentoring programs that do not use the CASA model of using community lay volunteers to represent the interests of foster children in the context of dependency court processes through the child welfare system.
3.1.4 Types of outcome measures
The outcome measures of interest for this review directly relate to the federally-mandated goals of the child welfare system, and to the outcome domains purported to be targets of CASA intervention: safety, permanency, and well-being.
The specific outcome variables as described in the objectives of this protocol are: permanency outcome (adoption, reunification, or long-term foster care), case duration (i.e., number of days or months the case is open), placement stability (i.e., the total number of placements or total number of placement changes experienced by a child while in care), and recidivism (re-referral to child welfare or re-entry to foster care within a designated follow-up period).
The outcome measures related to case outcome, case duration, and placement stability align with the target outcome domain of permanency. The outcome measure of placement stability also aligns with the outcome domain of well-being, as placement disruptions have been demonstrated to be negatively associated with the emotional/behavioral well-being of foster children (Rubin et al., 2007). Other aspects of children's well-being might or might not be supported or improved by CASA, but this review is limited to the evaluation of placement stability because this is the only measure in this domain for which there are available data. Finally, the outcome measure of maltreatment recurrence/recidivism is related to the outcome domain of child safety.
3.1.5 Duration of follow-up
The review will include studies involving cases that have reached their final outcome/closure as of study publication. Because the duration of follow-up will vary across studies, the number of months post-intervention that data were collected will be coded as a variable and included in the analysis.
Reports of intermediate outcomes on pending cases (i.e. cases that were still open and pending final disposition at study publication) will be excluded from the analysis, as they cannot provide information on the effectiveness of the intervention without having reached a final outcome.
In studies where there are both final outcomes reported for closed cases, and intermediate outcomes reported for pending/open cases, the closed cases will be included in the analysis and the pending cases will be excluded.
3.1.6 Types of settings
This review will include studies of U.S. populations only. Although CASA-type interventions exist elsewhere (e.g., CAFCASS in the U.K.), there is substantial variation in child welfare systems and legal statutes across countries, making the interventions and outcomes not directly comparable. For example, Gilbert, Parton, and Skivenes (2012), have identified several dimensions on which child welfare/child protection systems may systematically vary by country, including attitudes on the nature of maltreatment (individual deviance vs. family dysfunction), and the intervention mechanism used by the state (adversarial/legalistic, vs. therapeutic). According to Gilbert and his colleagues, these dimensions translate to two broad system approaches, which they call child protection vs. family service orientations, with the U.S. falling clearly in the child protection orientation, entailing legalistic investigations of maltreatment followed by typically coercive/involuntary out-of-home placements when necessary. CASA is targeted specifically to align with the goals and processes of the U.S. child welfare system, and therefore it is not useful to include similar interventions from other countries that are oriented to potentially disparate goals and processes in their respective child welfare systems. Non-U.S. studies will therefore be excluded.
3.1.7 Other Criteria
This review will not be restricted by publication status or year published/conducted. Studies will be limited to those that were conducted in the United States and published in English.
3.2 Search methods for identification of studies
3.2.1 Search strategy
Several search strategies will be utilized in an attempt to identify and retrieve published and unpublished studies. Hand searches will be conducted in the most recent issues (i.e., past year) of the journals in which we find relevant studies as recent issues may not yet be indexed in the databases. Broadly speaking, we will search internet sites, electronic databases, and research registries, conduct steps to capture gray literature, and review the reference lists of existing reviews of CASA literature to search for studies meeting the inclusion criteria. More specifically, we will systematically search: Internet websites and search engines Google (to search for publicly-available reports of internal research by local CASA agencies) Google Scholar National CASA Association website Electronic databases EBSCO Academic Search Premier ERIC MEDLINE PsycINFO Social Service Abstracts Social Work Abstracts Sociological Abstracts LexisNexis Academic Cochrane Database of Systematic Reviews, part of The Cochrane Library Cochrane Central Register of Controlled Trials (CENTRAL), part of the Cochrane Library
3.2.2 Search terms and keywords
Database specific strategies will be explored for each database (i.e., age limiters). The strategy for search terms will focus on identifying studies using the intervention and population of interest in order to broadly capture as many CASA studies on as many outcome variables as possible for screening. The search terms relevant to locating research on CASA-modelled interventions are:
CASA OR “court appointed special advocate” OR “lay volunteer” OR “child advoca*” OR “lay represent*” OR “volunteer represent*”
AND
“child welfare” OR “foster child*” OR “foster care” OR “foster placement” OR “dependency
court“ OR ”family court“ OR ”kinship placement“ OR ”out of home care“ OR ”substitute care“ OR ”out of home placement“ OR ”state custody“ Research registries Office of Juvenile Justice and Delinquency Prevention (CASA's primary federal funder) publication database Child Welfare Information Gateway ( Grey literature sources Proquest Dissertations and Theses Database Contact with relevant organizations and individuals Authors of prior studies will be contacted in an attempt to obtain unpublished research findings or published research findings with insufficient raw data for effect size computation The National Data Archive on Child Abuse and Neglect at Cornell University maintains a listserv for child maltreatment researchers to exchange information. An email will be sent to the listserv soliciting published or unpublished studies on CASA intervention. Feedback from administrative personnel at the National CASA Association will be solicited, once the database and grey literature searches are complete, with the intent to identify any additional known sources of data. Bibliographies of previous literature reviews and all included studies. The reference lists from all prior reviews noted above and all studies located once the search methods and inclusion criteria are applied will be reviewed at full text stage.
Two reviewers will independently read and evaluate titles and abstracts of all records resulting from the searches. We will obtain the full text for any report that may meet the eligibility criteria. Two reviewers will independently read the full text reports and will then compare notes to obtain a consensus list of eligible studies. Any disagreement about a study's eligibility will be resolved through discussion and by consultation with a third reviewer.
An example of a full search strategy in PsychINFO is provided in Appendix A.
3.3 Data collection and analysis
3.3.1 Description of methods used in primary research
Studies to be included in this review will employ experimental or quasi-experimental research designs that compare outcomes for an intervention group to those for a control or comparison condition.
Many potentially eligible studies use a quasi-experimental retrospective case comparison design using administrative case records for data extraction. In these studies, cases are sampled from closed child welfare case records and separated into groups based on whether CASA services were received, and then the case characteristics and outcomes of the groups are recorded and statistically compared. One study that exemplifies these methods likely to meet inclusion criteria for the proposed review is the retrospective case comparison study by Calkins & Millar (1999). In this study, all of the closed cases involving children entering custody in a single year (1994) were sampled and separated based on those that received CASA services (n=68) and those that had no CASA representation (n=121). Initial demographic comparisons showed no baseline differences between groups. The outcome variables measured included number of placements children had while in foster care, total length of time spent in foster care, whether permanency was attained, and whether reunification was the permanency outcome.
Other potentially eligible studies use a quasi-experimental prospective case comparison design, in which children entering care in a selected jurisdiction during a designated time interval are assigned to condition based upon whether they are appointed to receive CASA services by the court. Often children in the comparison (no-CASA) condition are selected using matching variables (such as age, race, and gender) to match them to children in the treatment group. In these designs, children in both the treatment and comparison groups are tracked to measure attainment of outcomes of interest and then groups are statistically compared. A study that exemplifies these methods that is likely to meet inclusion criteria is the prospective case comparison study by Litzelfelner (2000). In this study, all children who entered care and were assigned a CASA during an 18-month period (n=119) were assigned to the treatment condition. Control group children (not appointed a CASA) were selected from among those entering care during the same time period based on the matching variables of age, race, and maltreatment type (n=81). After a 24-month follow-up period, data was collected from case records on outcome variables that included whether the case had reached closure, case duration, number of placements while in care, whether adoption was attained as the permanency outcome.
3.3.2 Criteria for determination of independent findings
We are interested in several primary outcomes for this review, including case duration, number of placements, type of permanency outcome, and recidivism. We anticipate that included studies will use multiple measures for each outcome, multiple follow-up time points, and possibly more than one counterfactual condition. These circumstances create statistical dependencies that violate assumptions of standard meta-analytic methods. In order to ensure independence of study-level effect sizes, we will include only one effect size estimate from each independent sample on each outcome construct.
For cases in which a study uses multiple measures of the same construct, we will code data for each measure and create a study level average across the measures. In cases where multiple points of follow-up are provided, we will code follow-up points to conduct a separate analysis for effect sizes comparing studies with similar points of follow-up. In the case of multiple counterfactual conditions, we will select the comparison condition that is most similar to those in the other included studies.
As some studies may be reported in multiple reports or multiple reports reported in single studies, care will be taken to ensure that the studies are reporting independent findings. If it is unclear whether reports and studies provide independent findings, the authors of the reports will be contacted.
3.3.3 Details of study coding categories
We will code all studies meeting the inclusion criteria by using a coding instrument that was developed to specify and systematize the information to be extracted from each eligible study (Appendix B). The coding instrument will include items related to bibliographic information and source descriptors; methods and procedures; nature and implementation of the intervention; sample characteristics; and outcome data needed to calculate effect sizes.
Risk of Bias. Two review authors will independently assess risk of bias using the Cochrane Collaboration's Risk of bias tool (Higgins et al., 2011) for RCTs and the new extended Cochrane Risk of Bias tool for non-randomized studies of interventions (ACROBAT-NRS) for non-randomized trials. Risk of bias in each domain will be reported within and across studies in the results section using narrative and graphs. We anticipate that most studies included in this review will be at high risk of bias, thus we do not plan to restrict analyses based on risk of bias nor do we anticipate using risk of bias as a moderator variable. We plan to present all included studies and provide a narrative discussion of risk of bias to include discussion of the potential limitations of the review as well as implications of bias in the interpretation of the results in the Discussion section of the review. If sufficient studies are identified (Higgins & Green, 2011, section 10.4.1, suggest as least ten studies) we will examine the symmetry of funnel plots in attempt to obtain information about possible publication bias. If asymmetry is present, we will consider possible reasons for this while taking into account that an asymmetric funnel plot is not necessarily caused by publication bias and publication bias does not necessarily cause asymmetry in a funnel plot. We will also examine publication bias using the Egger's regression test.
3.3.4 Study Coding Procedures
To ensure reliability of coding procedures and decisions, two trained coders will independently code 100% of the included studies. If greater than 20 studies are eligible for inclusion in the review, a third coder may be recruited to assist with coding. The coders will be trained and will pilot test the code form together using diverse types of studies and will discuss any items that are unclear and ensure mutual understanding of all items. Following pilot testing of the form, two coders will independently code 100% of the included studies.
Coders will compare coding and will identify and discuss discrepancies, which will be resolved through consensus. If consensus cannot be reached between the two coders, a third member of the review team will be consulted, and the first author will make the final coding decision with input from all collaborators. We will use Excel to manage data.
3.4 Data synthesis
3.4.1 Statistical procedures and conventions
We will conduct descriptive analyses on variables of interest from all included studies to provide information regarding: Study characteristics (e.g., study design, sample size, etc.) Study participants Relevant intervention characteristics Risk of bias
Following descriptive analysis, we will estimate effect sizes for each included study. We will calculate the magnitude of effect using the standardized mean difference effect size with Hedges' g correction for continuous outcomes and odds ratios for outcomes presented as dichotomous variables. We will use CMA version 3.0 to calculate effect sizes and to convert effect sizes to Hedges' g from odds ratio or other effect size metrics as necessary. All effect sizes will be calculated using a 95% confidence interval. We will code all effect sizes so that a positive effect reflects that the outcomes favor the treatment group.
Although not common practice in U.S. child welfare research, we may encounter studies in which levels higher than that of the individual (e.g., programs, courts) are assigned to condition and the analysis was conducted at the individual level. In these cases, while the point estimates are unbiased, the standard errors of the effect estimates are likely to be underestimated. Our approach will therefore be as follows. If a study is cluster-randomised and an effect size is provided, we will check to confirm that the investigators had controlled for the effect of clustering in their results and will contact the authors for further information if this is unclear. If the clustering effect had not been controlled for, we will request individual participant data to calculate an estimate of the intra-cluster correlation coefficient (ICC). If that is not available, we will obtain external estimates of the ICC from similar studies, and use these data to analyse effect sizes and confidence intervals using the generic inverse variance method (Higgins & Green, 2011: Section 16.3.3).
Following the estimation of individual study level effects, we will conduct separate meta-analyses to pool RCT studies and for each outcome construct. A weighted mean effect will be calculated by weighting each study level effect size by the inverse of its variance. We will assume a random effects model and will report the variance component for each analysis. RCT studies will be pooled separately from QED studies; however, if RCT and QED studies are found to be homogenous, as assessed using the Q-test, studies will be pooled to allow for greater statistical power.
Following the estimation of summary effects, we will conduct a test of homogeneity (Q-test) to compare the observed variance to what would be expected from sampling error. The I2 statistic will also be used to describe the percentage of total variation across studies due to the heterogeneity rather than chance and we will also report τ2. We will construct a forest plot displaying study-level mean effect sizes and 95% confidence intervals for the included studies to provide opportunity for visual analysis of the precision of the estimated effect sizes, detection of studies with extreme effects, and information regarding heterogeneity of studies.
We will conduct moderator analysis to examine characteristics of the study methods, interventions, and participant characteristics that may be associated with effect size. Specifically, we will test the relationship of effect size to the following variables: study design; type of CASA model; duration of intervention; publication status. Because the variables that we plan to examine as moderators are categorical variables, we will use the Analog to the ANOVA Sensitivity analysis will be conducted to examine the potentially biasing affects of outliers. If necessary, additional sensitivity analysis will be conducted if other issues arise that may impede our confidence in the estimated pooled effect size estimates. Analysis will be conducted using Comprehensive Meta-Analysis 3.0 (CMA).
3.4.2 Dealing with missing data and incomplete data
Where possible, we will conduct meta-analyses using data from all allocated participants, and will report when that is not the case. Where studies do not report intention-to-treat analyses, we will attempt to obtain missing data by contacting the study authors. We will assess whether incomplete data was dealt with adequately by the study authors, and how data on attrition was reported compared with the total randomised. In the event that missing data cannot be provided, we will report and calculate using the available data only without imputation.
3.4.3 Treatment of qualitative research
We do not plan to include qualitative research.
REVIEW AUTHORS
ROLES AND RESPONSIBLIITIES
Who is responsible for the below areas? Please list their names: Content: Lawson, Berrick Systematic review methods: Maynard, Lawson Statistical analysis: Maynard Information retrieval: Lawson, Maynard
SOURCES OF SUPPORT
No support has been requested or is planned at this time.
DECLARATIONS OF INTEREST
Dr. Lawson and Dr. Berrick were previously engaged in planning a primary research study on CASA in California. The project (active from January 2011 to December 2012) was funded by the Walter S. Johnson foundation and was intended to be a randomized controlled trial to study the effect of CASA on case outcomes. The study was not executed because the researchers were unable to obtain approval from judges and agencies in enough jurisdictions to gather a sufficient sample size for the study.
With approval from their funding source, Dr. Lawson and Dr. Berrick subsequently used a portion of the study funding to consult with California CASA on the CASA administrative database to implement changes that would allow for better tracking of individual child outcomes in multiple domains over time for children referred for CASA services.
Dr. Lawson and Dr. Berrick have previously published a (non-systematic) review of the CASA literature.
PRELIMINARY TIMEFRAME
It is estimated that this review will be completed within 12 months following approval of the protocol.
PLANS FOR UPDATING THE REVIEW
The lead reviewer will be responsible for updating the review approximately every 3-5 years.
AUTHOR DECLARATION
By completing this form, you accept responsibility for preparing, maintaining and updating the review in accordance with Campbell Collaboration policy. The Campbell Collaboration will provide as much support as possible to assist with the preparation of the review.
A draft review must be submitted to the relevant Coordinating Group within two years of protocol publication. If drafts are not submitted before the agreed deadlines, or if we are unable to contact you for an extended period, the relevant Coordinating Group has the right to de-register the title or transfer the title to alternative authors. The Coordinating Group also has the right to de-register or transfer the title if it does not meet the standards of the Coordinating Group and/or the Campbell Collaboration.
You accept responsibility for maintaining the review in light of new evidence, comments and criticisms, and other developments, and updating the review at least once every five years, or, if requested, transferring responsibility for maintaining the review to others as agreed with the Coordinating Group.
The support of the Coordinating Group in preparing your review is conditional upon your agreement to publish the protocol, finished review, and subsequent updates in the Campbell Library. The Campbell Collaboration places no restrictions on publication of the findings of a Campbell systematic review in a more abbreviated form as a journal article either before or after the publication of the monograph version in Campbell Systematic Reviews. Some journals, however, have restrictions that preclude publication of findings that have been, or will be, reported elsewhere and authors considering publication in such a journal should be aware of possible conflict with publication of the monograph version in Campbell Systematic Reviews. Publication in a journal after publication or in press status in Campbell Systematic Reviews should acknowledge the Campbell version and include a citation to it. Note that systematic reviews published in Campbell Systematic Reviews and co-registered with the Cochrane Collaboration may have additional requirements or restrictions for co-publication. Review authors accept responsibility for meeting any co-publication requirements.
