Structured well-being assessments in public child welfare: Observations across two states

Abstract

Children entering custody within the child welfare system have been found to have high levels of trauma and significant behavioral health needs. In this paper, authors demonstrate how a structured functional well-being assessment can be used with the custody population to promote an understanding of behavioral health needs, inform case planning, and measure functional improvement over time. Specifically, this paper will: (a) briefly describe how two states implemented a common standardized assessment of functioning to inform case planning and measure well-being progress of children in the custody of a public child welfare system (b) examine what this common assessment tool reveals about the strengths and needs of children entering custody across two sites and (c) describe the magnitude of change in functional improvement measured across 6 months. This paper will contribute to the existing knowledge by sharing possible themes in functioning related to children entering custody while examining changes in functioning over time. Implications for practice, policy, and future research will be discussed.

Keywords

Child welfare functional assessment out-of-home care wellbeing

Introduction

In 2017, the child welfare system conducted an investigation or provided an alternative response for more than 3.5 million children in the United States, an increase of 15% since 2013. Approximately 674,000 of these were identified as victims of maltreatment (U.S. Department of Health & Human Services, 2018a). On the last day of 2015 there were over 420,000 children in custody. Although the number of children in care had been decreasing, recent data demonstrates these numbers are now rising (U.S. Department of Health & Human Services, 2018b). Considering this volume, responding to the needs of children involved in the child welfare system, and promoting their wellbeing is of paramount importance.

Children entering state custody, are at a greater risk for significant trauma and associated behavioral health needs (Casenueva et al., 2011). However, these youth often do not receive timely and adequate behavioral health services (Cooper & Vick, 2009). In the general population, over 68% report exposure to at least one traumatic experience (Copeland et al., 2007). Numerous previous studies have explored trauma experiences in the child welfare-involved population (e.g., Briggs et al., 2012; Dorsey et al., 2012; Salazar et al., 2013). Griffin and colleagues’ (2011) found that the proportion of those youth reporting at least one type of trauma was more than 95%. Despite prior research demonstrating that children in custody have mental and behavioral health needs (e.g. Burns et al., 2004; Lehmann et al., 2013; Oswald et al., 2010; Randsalu & Laurell, 2018), one early study showed that over 95% of state child welfare systems did not provide adequate physical and mental health services to children in custody (McCarthy et al., 2004).

Child welfare and behavioral health service delivery systems can be fragmented and do not effectively collaborate to provide appropriate evidence-based treatment (Cooper & Vick, 2009). As a result, these traumatic experiences and their associated effects on the emotional and behavioral wellbeing of the children in out-of-home may not be identified (Ai et al., 2013). For example, children may not be referred for behavioral health assessment until they begin to exhibit externalizing behaviors, leaving those with internalized symptoms untreated (Conradi et al., 2011). This issue has long been documented. For example, it was estimated that over 75% of youth in custody with diagnosable disorders receive no treatment (Kataoka et al., 2002). Long term effects can be far reaching, including underperformance in school, unhealthy relationships, unemployment, and involvement in the criminal justice system (U.S. Department of Health & Human Services, 2014).

The use of more comprehensive trauma-informed screening and assessment practices that expand beyond clinical functioning and expand across life domains may increase the likelihood that children will not slip through the cracks by not being referred for services. Identifying the needs, and building on the strengths, of children in custody has been described as critically important for the wellbeing of children in the child welfare system (Conradi et al., 2011). However, a study of service providers conducted by the National Child Traumatic Stress Network found that they seldom received in-depth information about a child’s trauma history when the child was referred to them (McMahon & Forehand, 2005). Further, documented barriers lie in the relationship and interaction between workers in the child welfare and behavioral health systems. Kerns and colleagues (2016) found a shared desire among these professionals to collaborate in the promotion of effective service delivery, and there was a need for structured processes to promote communication and care coordination.

Structured assessment of well-being and functioning

The public child welfare system in the United States aims to achieve three outcomes: safety, permanency, and child well-being. Outcome measures for safety and permanency have been built into state quality improvement systems while the systematic and structured measurement of well-being is less common (Rosanbalm et al., 2016). Rosanbalm and colleagues (2015) note that a comprehensive assessment of child functioning and well-being across life domains is critical for building a case plan directly linked to a child’s existing strengths and needs.

Within the context of a trauma-responsive system, the literature has begun to assert the importance of universal screening for trauma and behavioral health needs, and the use of standardized functional assessment to identify treatment needs (e.g. Conradi et al., 2011; Lang et al., 2017). Research suggests clinicians are influenced by a wide range of factors when deciding on clinical interventions including ease of implementation, flexibility, peer recommendation, accessibility to training and supervision, approval for insurance reimbursement, length of treatment, and personal experience (Nelson et al., 2006; Nelson & Steele, 2008). He and colleagues (2015) found improved child welfare and mental health collaboration, and provision of appropriate services, through the use of standardized tools. The literature also suggests that structured assessments are associated with greater diagnostic accuracy than unstructured approaches (Ponniah et al., 2011). Likewise, research indicates that standardized assessments are perceived as leading to identification of more risk and protective factors, being of a higher quality and more useful for treatment planning compared to unstructured assessment techniques (Andershed & Andershed, 2016). As opposed to a symptom-based assessment, a functional assessment examines multiple domains of social-emotional functioning. It is asserted that this approach has the potential to drive the development of outcomes focused treatment plans (Wotring et al., 2005) and the selection of interventions that may lead to improved well-being (Bracken et al., 1998).

To assist in this process, structured assessments of well-being and functioning, such as the Child and Adolescent Needs and Strengths Tool (CANS) or the Child and Adolescent Functional Assessment Scale (CAFAS), are increasingly recognized as an essential component in identifying the treatment needs of children in state custody (Grady & Drisko, 2014; Igelman et al., 2007; Kisiel et al., 2017). Further, data from these tools can be used to monitor progress in functioning and well-being over time. Although the literature has begun to explore ways in which functional assessments are being implemented in various sites (Akin et al., 2017; Kisiel et al., 2017; Rosanbalm et al., 2016) few studies have disseminated findings related to the utilization of functional assessment data to increase our understanding of the strengths and needs of children entering custody, or have described the progress made over time on well-being indicators measured by functional assessment tools (Jaudes et al., 2016). The Children’s Bureau has invested in three cohorts of demonstrations grants (HHS-2011-ACF-ACYF-CO-0169; HHS-2012-ACF-ACYF-CO-0279; and HHS-2013-ACF-ACYF-CO-0637) to investigate whether trauma-responsive child welfare systems are better able to address the safety, permanency and wellbeing needs of child in custody. Across these three grant cohorts, 18 sites were funded to test whether child welfare system changes related to (a) behavioral health and trauma screenings, (b) functional assessments, and (c) evidence-based treatment and case planning matching identified needs, would improve child outcomes. Two state-initiatives funded through this mechanism are the subject of this paper, with a focus on the functional assessment portion of this work.

Purpose

The purpose of this article is to illustrate a demonstration of how standardized functional assessment can be used with the custody population to promote understanding of behavioral health needs, inform case planning, and measure functional improvement over time. Specifically, this paper will: (a) briefly describe how two states implemented a common standardized assessment of functioning to inform case planning and measure well-being progress of children in the custody of a public child welfare system (b) examine what this assessment tool reveals about the strengths and needs of children entering custody across two sites and (c) describe the magnitude of change in functional improvement measured across 6 months. This paper will contribute to the existing knowledge by sharing possible themes in functioning related to children entering custody while examining changes in functioning over time. Implications for practice, policy, and future research will be discussed.

Methods

This section illuminates the descriptive and exploratory methodology used across two sites with an emphasis on comparison, and discussion of how such processes may be used to benefit child- and organization-level decision-making in child welfare and behavioral health agencies. Institutional Review Board approval was obtained by each participating university.

Measurement

The CANS contains core items in each domain and then additional items from which sites can select. Studies One and Two included different numbers of items in each domain. Five domains were used: (1) life domain functioning (Site One 16 items, Site Two 11 items), (2) child strengths functioning (Site One 12 items, Site Two 10 items), (3) child emotional /behavioral needs functioning (Site One 13 items, Site Two 10 items), (4) child risk behaviors functioning (Site One 13 items, Site Two 7 items), and (5) caregiver needs/strengths functioning (Site One 20 items, Site Two 12 items). Domains consist of a varying number of individual items to be rated from zero to three. Individual items rated with a two or a three are considered “actionable” and should be addressed in the child’s service or treatment plan. In addition to individual item ratings, dimension scores can be calculated by summing items within each domain (Praed Foundation, 2016).

The CANS is described as a communimetric tool, which combines clinimetrics (Feinstein, 1987) with communication theories. The tool can be customized to meet the needs of the user through selection of domains and submodules to be completed (Lyons, 2009). The CANS enjoys widespread use in child welfare and behavioral health due to its ability to add clarity and structure to the case planning process which allows for examination of strengths and needs of the child and their caregiving system (Rosanbalm et al., 2016). In each site CANS data was collected and entered into their respective data system according to their implementation protocols (Site One n = 1499; Site Two n = 348).

Research has examined the reliability and validity of the CANS. Anderson and colleagues (2003) obtained an inter-rater reliability of 0.85 between researchers conducting a retrospective case reviews and of 0.81 between researchers’ case review assessments and those completed prospectively by caseworkers at the time of client admission, which is consistent with reliability assertion of the Praed Foundation (2016). The CANS has demonstrated an adequate degree of concurrent validity with another commonly used assessment tool, the CAFAS (Dilley et al., 2007). Studies conducting factor analyses using the CANS have produced a similar three-factor solution (caregiver problems, internalizing behaviors, externalizing behaviors) suggesting a consistent underlying structure and some degree of construct validity (Rosanbalm et al., 2016). Cordell and colleagues (2016) identified particular CANS items associated with the most elevated levels of clinical need. The CANS has demonstrated predictive validity in terms of a variety of factors, including treatment decisions (Anderson & Estle, 2001), service utilization and placement disruption (Kisiel et al., 2009), the restrictiveness of placement (Lardner, 2015), and placement recommendations (Chor et al., 2013). Both sites selected the Child and Adolescent Needs and Strengths tool for the standardized functional assessment and progress monitoring measure primarily due to its low cost, broad assessment domains, inclusion of strengths, and its collaborative nature.

Sample

Inclusion criteria for Site One included: (1) the child’s universal screening indicated the need for functional assessment (i.e., a positive screen-in), (2) the child had three or more completed CANS assessment records (i.e., an initial assessment and two or more re-assessments), and (3) the child was administered the version of the CANS used for children aged 5 years old or older at both the first and third assessment points. This yielded a total 1502 cases. An additional three cases were excluded due to missing data across all CANS domains resulting in a final sample size of 1499 cases.

Inclusion criteria for Site Two included: (1) the child entered custody in the past 30 days (2) the child had two or more completed CANS assessments, and (3) the child was administered the version of the CANS used for children aged 5 years old or older at both the first and second assessment points. This yielded a total 1475 CANS administered across 348 children. During the given period of time, the average number of children entering custody in this project site, over the age of 5, was 397 with an average of 1300 children in custody statewide.

Results

Implementation of standardized assessment of functioning across two sites

Site One

This project was implemented in an average-sized southern state with approximately 9300 children in the state’s custody, and 7500 entering custody annually over past 5 years. The target population included all children entering custody, who were to be universally screened for trauma exposure and symptomology, and behavioral health needs within 10 days of placement. The results of the screening triggered referral, to include the results of the screener to a behavioral health provider for a comprehensive functional assessment using the CANS tool. The clinical and functional assessment is required to include administration of the Child and Adolescent Needs and Strengths (CANS) within 30 days of referral. Results of the CANS, diagnosis, recommendations for evidence-based treatment and other services are entered into a web-based data system and electronically transmitted into the state child welfare agency’s management information (SACWIS) system where it becomes available for the worker to use in case planning and decision-making. The CANS is to be re-administered every 90 days to measure progress, and report services provided since the last assessment, changes noted and recommendations for further treatment as appropriate. This process serves not only to inform the assessment of functioning and treatment selection, but to facilitate information exchange, progress monitoring and optimal collaboration between the child welfare worker and the behavioral health clinician.

Site Two

This project took place in a small rural state with approximately 1300 children in custody and 800 entering custody on average annually over the past 5 years. In this project all children entering state custody were to be assessed using the CANS. No other screening tool was completed prior to the CANS. The CANS was administered by either a child welfare worker or a mental health worker in a community based mental/behavioral health care agency within 30 days of custody entrance. The CANS was then to be re-administered every 6 months to monitor progress and inform ongoing case planning. In districts where the CANS was completed by a provider outside of the public child welfare office, the provider was to share the results with the child welfare worker. The child welfare worker was then to address the identified needs and strengths in the case planning process. In some districts, the CANS was completed in a joint collaborative meeting that included the family, the child welfare worker, mental health clinician and other individuals involved in the case thus capitalizing on the “communimetrics” nature of the tool while bolstering system-level collaborative infrastructure and service continuity for families.

Summary of similarity and differences in implementation

As has been noted, each of the sites has taken a different approach to who administers the functional assessment and how often it is re-administered. Since Site One reassesses children every 90 days and site two administers every 6 months, or 180 days, the first and third CANS will be used in the comparative analysis. For the purpose of this paper, only children ages 5 and above are included, because of limitations in data availability.

Understanding the strengths and needs of children entering custody

In the section below it is demonstrated how the children from these two states compared on four domains of life functioning upon entrance into custody and out of home care. Due to variation in items used, in addition to some items not being measured in both states, some items have been combined into one. The findings in the table describe the percentage of cases that are identified as “actionable” on each item within a domain, meaning they scored a 2 or 3.

Site One

Table 1 illustrates the percentage of cases scoring at an actionable level at entrance for each item. The item with the highest percentage of cases scoring at an actionable level for each domain are: Life Domain- --family functioning,; Child Strengths—nuclear family strengths,; Child Emotional/Behavioral Needs—impulsivity/hyperactivity,; Child Risk Behaviors—Danger to others; Caregiver Needs and Strengths—supervision.

Table 1.

Site One percentage of cases scoring at an actionable level by item (1st & 3rd CANS) (n = 1499).

	1st CANS		3rd CANS
	n	%	n	%
Life Domain Functioning
Family Functioning	984	65.5%	701	46.8%
Living Situation	553	36.8%	327	21.8%
Social Functioning	466	31.0%	368	24.5%
Sleep	161	10.7%	66	4.4%
Legal	165	11.0%	104	6.9%
School Behavior	348	23.2%	211	14.1%
Communication	96	6.4%	71	4.7%
Sexual Development/Identity	90	6.0%	44	2.9%
Developmental	58	3.9%	45	3.0%
Medical	34	2.3%	24	1.6%
Physical	19	1.3%	14	0.9%
Independent Living Skills	494	32.9%	464	31.0%
Judgment	507	33.8%	355	23.7%
Recreational	242	16.1%	132	8.8%
Elimination	54	3.6%	33	2.2%
Child Strengths
Extended Family Relationships	886	59.8%	835	55.7%
Nuclear Family Strengths	960	64.0%	801	53.4%
Interpersonal	603	40.2%	496	33.1%
Optimism	471	31.4%	409	27.3%
Educational	161	10.7%	78	5.2%
Vocational	893	59.6%	868	57.9%
Talents/Interests	464	31.0%	356	23.7%
Spiritual/Religious	682	45.5%	575	38.4%
Community Life	636	42.4%	442	29.4%
Relationship Permanence	908	60.5%	693	46.2%
Resiliency	545	36.4%	423	28.2%
Resourcefulness	642	42.8%	511	34.1%
Child Emotional/Behavioral Needs
Psychosis	21	1.4%	10	0.7%
Impulsivity/Hyperactivity	483	32.2%	403	26.9%
Oppositional	447	29.8%	335	22.3%
Conduct	225	15.0%	161	10.7%
Adjustment to Trauma	392	26.1%	315	21.0%
Anxiety/Worry	307	20.5%	267	17.8%
Depression	181	12.1%	159	10.6%
Anger Control	441	29.4%	307	20.5%
Attention	314	20.9%	246	16.4%
Substance Use	136	9.1%	55	3.7%
Attachment	151	10.1%	136	9.1%
Somatization	16	1.1%	17	1.1%
Eating Disturbances	10	0.7%	8	0.5%
Child Risk Behaviors
Suicide	57	3.8%	50	3.3%
Self-Injurious Behavior	71	4.7%	52	3.5%
Other Self-Harm	43	2.9%	35	2.3%
Danger to Others	194	12.9%	116	7.7%
Runaway	89	5.9%	43	2.9%
Delinquent Behaviors	89	5.9%	55	3.7%
Intentional Misbehavior	143	9.5%	122	8.1%
Sexual Aggression	21	1.4%	12	0.8%
Fire Setting	10	0.7%	6	0.4%
Bullying	85	5.7%	113	7.5%
Sexually Reactive Behavior	47	3.1%	35	2.3%
Exploited	34	2.3%	15	1.0%
Commercial Sexual Exploitation	8	0.5%	4	0.3%
Caregiver Needs/Strengths
Supervision	431	28.8%	250	16.7%
Involvement with Care	409	27.3%	273	18.2%
Knowledge	319	21.3%	218	14.5%
Residential Stability	230	15.3%	139	9.3%
Social Resources/Natural Supports	306	20.4%	182	12.1%
Physical Health	82	5.5%	55	3.7%
Mental Health	137	9.1%	95	6.3%
Substance Use	286	19.1%	130	8.7%
Developmental	34	2.3%	26	1.7%
Safety	207	13.8%	100	6.7%
Marital/Partner Violence	186	12.4%	96	6.4%
Posttraumatic Reactions	52	3.5%	33	2.2%
Resources	262	17.5%	164	10.9%
Accessibility to Child Care Services	155	10.3%	93	6.2%
Organization	185	12.3%	92	6.1%
Financial Resources	283	18.9%	182	12.1%
Self-Care/Daily Living	109	7.3%	57	3.8%
Educational Attainment	96	6.4%	52	3.5%
Legal	179	11.9%	107	7.1%
Transportation	143	9.5%	110	7.3%

Site Two

Table 2 illustrates the percentage of cases scoring at an actionable level at entrance for each item. The item with the highest percentage of cases scoring at an actionable level for each domain in site 2 are: Life Domain-Family functioning; Child Strengths—Relationship permanence; Child Emotional/Behavioral Needs—Attention deficit/impulsivity/hyperactivity; Child Risk Behaviors—Danger to others; Caregiver Needs and Strengths—Supervision and Social resources/natural supports were tied.

Table 2.

Site Two percentage of cases scoring at an actionable level (1st & 6 m. CANS) (N = 1475 CANS administered; N = 348 with 2 CANS).

	1st CANS		2nd CANS
	N	%	n	%
Life Domain Functioning
Family Functioning	200	57.5%	176	50.6%
Living Situation	116	33.3%	107	30.7%
Cultural Stress	7	2.0%	3	0.9%
Sleep	83	23.9%	57	16.4%
Self Care/Daily Living	26	7.5%	20	5.7%
School Behavior	138	39.7%	111	31.9%
School Achievement	119	34.2%	96	27.6%
School Attendance	29	8.3%	28	8.0%
Job Functioning	8	2.3%	6	1.7%
Developmental	11	3.2%	12	3.4%
Medical/Physical	18	5.2%	9	2.6%
Child Strengths
Family Strengths	153	44.0%	133	38.2%
Interpersonal	150	43.1%	140	40.2%
Optimism	185	53.2%	179	51.4%
Educational System	65	18.7%	53	15.2%
Talents/Interests	116	33.3%	93	26.7%
Spiritual/Religious	16	4.6%	17	4.9%
Community Connections	220	63.2%	195	56.0%
Relationship Permanence	193	55.5%	159	45.7%
Resiliency	192	55.2%	161	46.3%
Child Involvement in Care	172	49.4%	169	48.6%
Child Emotional/Behavioral Needs
Psychosis	3	.09%	5	1.4%
Attention Deficit/Hyperactivity	330	42.0%	146	42.0%
Oppositional	135	38.8%	130	37.4%
Conduct	52	14.9%	45	12.9%
Adjustment to Trauma	216	62.1%	199	57.2%
Anxiety/Worry	152	43.7%	139	39.9%
Depression	94	27.0%	92	26.4%
Anger Control	148	41.5%	123	35.3%
Substance Use	10	2.9%	12	3.4%
Eating Disturbances	11	3.2%	9	2.6%
Child Risk Behaviors
Suicidal thoughts	30	8.6%	28	8.0%
Self-Injurious Behavior	32	9.2%	27	7.8%
Danger to Others	40	11.5%	42	12.1%
Runaway	21	6.0%	17	4.9%
Delinquent Behaviors	22	6.3%	13	3.7%
Fire Setting	2	0.6%	1	0.3%
Sexually Reactive Behavior	27	7.8%	28	8.0%
Caregiver Needs/Strengths
Supervision	93	26.7%	71	20.4%
Involvement with Care	71	20.4%	46	13.2%
Knowledge	77	22.1%	60	17.2%
Residential Stability	37	10.6%	28	8.0%
Social Resources/Natural Supports	93	26.7%	80	23.0%
Physical Health	35	10.1%	24	6.9%
Mental Health	60	17.2%	38	10.9%
Substance Use	29	8.3%	19	5.5%
Developmental	12	3.4%	9	2.6%
Safety	45	12.9%	22	6.3%
Organization	66	19.0%	52	14.9%
Financial Resources	69	19.8%	53	15.2%

Comparison across sites

When comparing individual items within the 5 domains, findings illustrate some variance, however when looking at the item with highest percentage of actionable cases in each site for each domain, the sites identified the same item for every domain except for Child Strengths where site 1 identified nuclear family strengths as the most actionable, and site two identified relationship permanence. Further descriptive analysis considered common items that were identified as actionable by more than 20% of the cases across both sites.

Within the Life functioning domain, more than 20% of cases in both sites had actionable items related to (a) family functioning, (b) living situation, and (c) school behavior/achievement. Within Child Strengths there were seven common actionable items which signifies a lack of strength in these areas: family strengths, interpersonal, optimism, talents/interests, community connections, relationship permanence, and resiliency. Within the Child Emotional/behavioral Needs domain, three items were identified as actionable by more than 20% of cases: (a) attention deficit/impulsivity/hyperactivity, (b) oppositional, and (c) anger control. There were no items in the child risk behaviors in either state that were identified as actionable by more than 20% of the cases. Finally, within the Caregiver Needs/Strengths domain there were four items in common: (a) supervision, (b) involvement in care, (c) residential stability, and (d) social resources/natural supports. Further, findings illustrate that the percentage of actionable items decreases in every category across both states from entrance to 6 months.

Further similarities were observed in the percentage of children with actionable scores on individual items across the population in each state. When considering the strengths and needs of children as they enter custody, 17 items were identified as similar across sites. Authors defined similar by including those items with n greater than 10% of cases and less than 10% difference between items scoring as actionable (see Figure 1).

Improvement in well-being and functioning across 6 months

This section illustrates functional improvement by domain over the course of 6 months of behavioral health treatment. First mean scores are provided for each domain. It should be noted that for each domain, the CANS instrument includes a set of core items, then jurisdictions can choose to use additional ones. Across the board, Site One included more items in each domain which should be taken into account when comparing results.

Site One

Table 3 provides the domain scores for the children entering custody and after 6 months of treatment. Since this site administers the instrument every 90 days, data is provided for the first and third administration so that comparison across sites is possible. T-tests demonstrated that a statistically significant improvement was found for each domain at the p < 0.001 level. In comparing the data from the two CANS administrations in Table 1, additional detail on functional improvement is provided.

Table 3.

CANS domain scores for Site One children at first administration and 6 months into treatment (N = 1499).

CANS Domain	Items (n)	1st CANS		3rd CANS
CANS Domain	Items (n)	M	(SD)	M	(SD)
Caregiver Needs & Strengths	20	8.7	11.7	5.7	10.1	p < .001
Child Behavioral/Emotional Needs	13	7.7	5.2	6.6	5.0	p < .001
Child Risk Behaviors	13	2.8	3.3	2.2	2.9	p < .001
Child Strengths	12	16.0	7.2	14.0	7.2	p < .001
Life Domain Functioning	15	9.4	5.2	7.5	5.0	p < .001
Total Severity Score	73	44.5	22.9	36.0	22.5	p < .001

The range of improvement by domain is as follows: Life Domain Functioning .6% (Physical) to 18.7% (family functioning, demonstrating the largest improvement across all items); Child Strengths 1.7% (vocational) to 14.3% (relational permanence); Child Emotional/Behavioral Needs 0% (somatization) to 8.9% anger control; Child Risk Behaviors −1.8% (bullying) to 5.2% (danger to others); Caregiver Needs/Strengths .6% (Developmental) to 12.1% (supervision).

Site Two

Table 4 provides the domain scores for the children entering custody and after 6 months of treatment. T-tests demonstrated that a statistically significant improvement was found for the caregiver strengths and needs, child behavioral/emotional needs, child strengths, and life domain functioning domains at varying degrees of significance. While movement was observed in the right direction, statistical significance was not found in the child risk behavior domain.

Table 4.

CANS domain scores for Site Two children at first administration and 6 months into treatment (N = 348).

		1st CANS		2nd CANS
CANS Domain	Items (n)	M	SD	M	SD
Caregiver Needs & Strengths	12	5.7	6.0	4.5	5.0	p < .001
Child Behavioral/Emotional Needs	10	9.3	4.1	8.8	4.0	p < .01
Child Risk Behaviors	7	3.5	3.6	3.2	3.5	p = .06
Child Strengths	10	12.8	4.6	11.7	4.6	p < .001
Life Domain Functioning	11	6.7	3.7	5.9	3.2	p <. 001
Total Severity Score	50	26.3	17.2	22.0	15.6	P < .001

In comparing the data from the two CANS administrations in Table 2, additional detail on functional improvement is provided. The range of improvement by domain is as follows: Life Domain Functioning −.2% (developmental) to 7.8% (school behavior); Child Strengths −.3% (spiritual/religious) to 9.8 (relational permanence); Child Emotional/Behavioral Needs −1.31% (psychosis) to 87.2% anger control; Child Risk Behaviors −.6% (danger to others) to 2.6% (delinquent behaviors); and Caregiver Needs/Strengths .8% (developmental) to 7.2% (involvement with care).

Comparison across sites

In terms of functional improvement over 6 months of treatment, the magnitude of change varied across states. In site one there was significant improvement over time in all domains, while in site two there was not a discernable pattern observed in the Child Risk Behaviors domain, although significant improvement was found in all the other domains.

Discussion

The examination of the constellation of socio-emotional functioning of children in state custody across two states provides a contribution to the literature that goes beyond prior studies that have described the behavioral health problems and diagnoses of children in care (e.g. Pecora et al., 2009), or the functioning of a subset of foster care alumni compared to the general population of young adults as examined in the landmark Midwest Study (Pecora et al., 2005). Comparison of the needs and strengths of children across states from different geographical regions provides some insight into what may be observed in the larger population of children in custody. By taking this exploratory approach to the magnitude of functional improvement of children in treatment, the field is similarly informed regarding those needs most likely to see improvement over the first 6 months in care. The implementation of standardized functional assessment and periodic reassessment as illustrated in these states is largely consistent with the recommendations of the Best Practices in Mental Health for Children in Child Welfare Steering Committee and the practice guidelines subsequently identified (Romanelli et al., 2009), and takes a first step in illustrating the extent to which such functional assessment meets the potential envisioned by the Committee and elsewhere in the literature.

Jurisdictions might benefit from discerning which items on the functional assessment they can directly address through service referral or direct practice. If there are items that are less likely to be able to be impacted through child welfare system action, then perhaps this item is less helpful on the functional assessment, thus balancing workforce time with access to service delivery. Comparison of these data between the two states illustrates the importance of selection of items to measure in standardized assessment. Jurisdictions need to make a choice in terms of how long and comprehensive the CANS (or similar instrument) should be and there are trade-offs to be considered. There were 14 items measured in one state but not the other in which at least 10% of the children scored in the actionable category: child involvement in care, measured in Site Two, needed attention in 49.4% of cases. The following items measured in Site One demonstrated substantial need: extended family relationships 59.8%, vocational 59.6%, resourcefulness 42.8%, judgment 33.8%, independent living 32.9%, social functioning 31%, resources 17.5%, recreational 16.1%, marital/partner violence 12.4%, legal (caregiver) 11.9%, legal 11%, accessibility to childcare services 10.3% and attachment 10.1%. If jurisdictions shared this sort of information it, could inform the selection/refinement of instruments used to assess behavioral health needs. There were also some items that were actionable in very small numbers and could be considered for elimination.

In order to determine the ongoing need for treatment, as well as to inform a variety of other decisions on the case level, measurement of functional improvement over time is imperative. The majority of measured items showed improvement over time although for many the magnitude of reduction in the percent of cases needing attention was small. More research is needed in terms of the amenability of certain behavioral health needs within a 6-month period for this population, which could inform clinician decisions regarding where to emphasize treatment. Were the same assessment tool to be used at appropriate junctures in the course of treatment in multiple jurisdictions, many types of relevant research questions could be answered.

This study illustrates that some types of need may be more amenable to change within a 6-month period. Child risk behaviors overall had a lower magnitude of improvement than items in other domains. In contrast, family functioning showed the largest magnitude of improvement for Site One, while relationship permanence having the biggest reduction in actionable cases for Site Two. Other areas showing at least 10% improvement were living situation, nuclear family strengths, community life, supervision, and substance use. Alternatively it could suggest that the functional assessment tool may not be sensitive to measuring change on child risk items. The findings in this paper did find changes over time, which is contradictory to findings from a systematic review of CANS scores over time by Sokol and colleagues (2020).

Limitations

Despite widespread use of the CANS and its use in research, there are some concerns with the instrument (citation and specifics). Although the literature speaks to its reliability and validity generally (Dilley et al., 2007; Praed Foundation, 2016), the ratings of each item are based on clinician judgment. The limited results shared here relate to observations in two states during implementation of grant-funded initiatives focused on practice change. They cannot be assumed to be generalizable to the greater custody care population. The variance seen between the two states may be based on a variety of different factors about which is only speculative. Between two states there are differences in items included in the CANS, protocols for implementation, and potentially even different training and qualifications of the individuals conducting the assessments. While the data shared here should be viewed cautiously as portraying the behavioral health needs and potential growth in the custody population, there are implications for policy, practice, and research.

Implications for practice

Although federal policy requires child welfare to measure outcomes associated with child safety, permanency and wellbeing, the actual measurement of the latter in the Child and Family Services Review (CFSR) has been criticized (e.g. Andrade et al., 2008; Courtney et al., 2004). The measures used in the CFSR focusing on whether the child receives services to meet their physical and mental health needs is a very distal measure of actual improvement in their wellbeing. As jurisdictions move to standardized measurement of functional improvement, this may be a more proximal measure of wellbeing. In addition, standardized, periodic measurement of functional improvement also promotes data informed treatment and case-related decision-making on the frontline level. Information from standardized functional assessment can be useful in placement decisions and determination of appropriate support services to promote placement stability and functional improvement. The literature has suggested use of standardized or structured tools may lead to higher quality treatment planning (Andershed & Andershed, 2016). It is important, however, to build in facilitative administration strategies to train, coach and use data to assess the extent to which workers are purpose-driven in the functional assessment practice, rather than task-oriented in which compliance with completing the tool is emphasized rather than using the tool for its intended purpose (Akin et al., 2019).

Additionally, as a communimetric tool, the CANS provides a useful foundation for collaboration between the child’s child welfare and behavioral health workers. By facilitating interorganizational communication, information sharing, and, optimally, collaboration, the system has the potential to become more trauma-responsive. The literature certainly encourages the implementation of standardized functional assessment in this regard (e.g. Conradi et al., 2011; Wotring et al., 2005). Collecting this sort of information in a standardized way also shows potential for assisting resource parents in meeting the needs of the children in their care, thereby promoting placement stability.

Implications for policy

This paper provides examples of two states’ use of a standardized functional assessment instrument, which has been recommend in the literature to initiate appropriate treatment for children in custody (e.g. Grady & Drisko, 2014; Rosanbaum et al., 2016). In addition to establishing a process which facilitates data-informed decision-making on the case level, the aggregated data generated from such structured consistent measurement has the potential to benefit agency- and system-level decision-making in a variety of ways including: service array reconfiguration and capacity-building around treatment modalities demonstrated effective with certain profiles of children; informing resource parent and kinship caregiver training and support initiatives; behavioral health clinician training and implementation of clinical decision-support tools to aide in treatment modality selection and case planning; and, informing medical necessity and treatment modality approval decisions in Medicaid agencies (Levitt, 2009).

Such instruments vary in their approach, their ability to be customized and their cost of implementation. Jurisdictions desiring to institute this sort of measurement certainly have choices. The data provided from these two states demonstrate how such information can be used to help agencies strike the balance between brevity and measuring strengths and needs in a comprehensive enough manner to address critical issues for the custody population. Were federal agencies to incentivize, encourage or require child welfare agencies to move in the direction of standardized measurement, consideration could be given to working toward how commonly used instruments can be aligned so that data collected across jurisdictions can be used to improve our understanding of this population and how to improve their wellbeing.

Implications for future research

Based only on the two states, some common actionable items are suggested that would have to be examined through research in other jurisdictions to determine if these are actually observed with greater regularity in the custody population. This study is unable to shed light on whether the similarities are due to cultural or geographical population similarities, the emphasis and expertise of clinicians, instrumentation or some other factor. Future research should further explore the similarities across sites found in this study including family functioning and living situation in Life Domain Functioning (which is not surprising given the circumstances which typically result in out-of-home placement), relationship permanence in the Child Strengths Domain, hyperactivity/attention, anger control, adjustment to trauma, and oppositional in the Child Emotional/Behavioral Needs Domain, danger to others in the Child Risk Behavior Domain, and supervision, involvement with care, knowledge, and social resources/natural support in the Caregiver Needs/Strengths Domain. Use of structured functional assessment such as demonstrated in this paper may inform capacity building among treatment and custody providers. Future research should examine differences by age, gender, race, type of maltreatment and other factors.

Because of differences in available data, it was not possible to look at the relationship between functional improvement and treatment modalities, service intensity and other related variables of interest, particularly as they relate to outcome improvement. Use of standardized assessment instruments, particularly across jurisdictions, opens the door to exploring a wide variety of research questions of this type. Further research is needed related to the comparative utility of various functional assessment measurement in terms of inter-rater reliability, the relative sensitivity to change, and best fit for use in the child welfare population. Combined with implementation of particular treatment modalities with fidelity, the field could learn a great deal in terms of treatment of the custody population.

Conclusion

Children entering custody come with an array of strengths and behavioral health needs resulting from their trauma history as well as the experience in placement. In many systems, referral for functional assessment and treatment may be delayed until trauma related needs are highlighted, often through externalizing behaviors. Given what research has taught us regarding trauma exposure, adverse mental health outcomes (Chang et al., 2019), and disruptive emotional problems and behaviors (Gerrity & Folcarelli, 2008), timely assessment and initiation of services is critically important for health and well-being. This paper demonstrates how two jurisdictions have used standardized functional assessment to determine the need for behavioral health treatment early on, and to measure progress over time. It further illuminates potential commonalities among children’s strengths and needs upon entrance into custody and which areas of functioning seem to be more responsive to services and assessment over time. Such data-informed processes show promise in helping agencies to identify progress and promote wellbeing.

Footnotes

Data availability statement

The data that support the findings of this study may or may not be available on request from the corresponding author. The data are not publicly available due to privacy or ethical restrictions.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was part of a demonstration project funded by the Children’s Bureau, Administration on Children, Youth and Families, Administration for Children and Families (90CO1118/90CO1119), U.S. Department of Health and Human Services. The contents of this article are solely the responsibility of the authors and do not necessarily represent the official views of the Children’s Bureau.

ORCID iD

Jessica Strolin-Goltzman

References

A. L.

Foster

L. J. J.

Pecora

P. J.

Delaney

Rodriguez

(2013). Reshaping child welfare’s response to trauma: Assessment, evidence-based intervention, and new research perspectives. Research on Social Work Practice, 23(6), 651–668. https://doi.org/echo.louisville.edu/10.1177/1049731513491835

Akin

Dunderley

Brook

Bruns

(2019). Driving organization and systems change toward trauma-responsive services in child welfare: Supervisor and administrator perspectives on initial implementation. Journal of Public Child Welfare. https://doi.org/10.1080/15548732.2019.1652720

Akin

Strolin-Goltzman

Collins-Camargo

(2017). Successes and challenges in developing trauma informed child welfare systems: A real world case study of exploration and initial implementation. Children and Youth Service Review.

Andershed

(2016). Improving evidence-based social work practice with youths exhibiting conduct problems through structured assessment. European Journal of Social Work, 19(6), 887–900.

Anderson

Estle

(2001). Predicting level of mental health care among children served in a delivery system in a rural state. Journal of Rural Health, 17(3), 259–265.

Anderson

R. L.

Lyons

J. S.

Giles

D. M.

Price

J. A.

Estle

(2003). Reliability of the child and adolescent needs and strengths-mental health (CANS-MH) scale. Journal of Child & Family Studies, 12(3), 279–289.

Andrade

Osterling

K. L.

Austin

M. J.

(2008). Understanding and measuring child welfare outcomes. Journal of Evidence-Based Social Work, 1/2, 135–156.

Bracken

B. A.

Keith

L. K

Walker

K. C.

(1998). Assessment of preschool behavior and social-emotional functioning: A review of thirteen third-party instruments. Journal of Psychoeducational Assessment, 16(2), 153–169.

Briggs

E. C.

Greeson

J. P.

Layne

C. M.

Fairbank

J. A.

Knoverek

A. M.

Pynoos

R. S.

(2012). Trauma exposure, psychosocial functioning, and treatment needs of youth in residential care: Preliminary findings from the NCTSN core data set. Journal of Child & Adolescent Trauma, 5(1), 1–15. https://doi.org/10.1080/19361521.2012.646413

10.

Burns

Phillips

Wagner

Barth

Kolko

Campbell

Landsverk

(2004). Mental health need and access to mental health services by youths involved with child welfare: A national survey. Journal of the American Academy of Child and Adolescent Psychiatry, 43, 960–970.

11.

Casanueva

Ringeisen

Wilson

Smith

Dolan

(2011). NSCAW II baseline report: Child well-being. US Department of Health and Human Services.

12.

Chang

Jiang

Mkandarwire

Shen

(2019). Associations between adverse childhood experiences and health outcomes in adults aged 18–59 years. PLoS One, 14(2), 1–11.

13.

Chor

K. B.

McClelland

G. M.

Weiner

D. A.

Jordan

Lyons

J. S.

(2013). Patterns of out-of-home placement decision-making in child welfare. Children and Youth Services Review, 37(10), 871–882.

14.

Conradi

Wherry

Kisiel

(2011). Linking child welfare and mental health using trauma-informed screening and assessment practices. Child Welfare, 90(6), 129–147.

15.

Cooper

J. L.

Vick

(2009). Promoting social-emotional wellbeing in early intervention services: A fifty-state view. National Center for Children in Poverty, Mailman School of Public Health, Columbia University.

16.

Copeland

W. E.

Keeler

Angold

Costello

E. J.

(2007). Traumatic events and posttraumatic stress in childhood. Archives of General Psychiatry, 64(5), 577–584.

17.

Cordell

K. D.

Snowden

L. R.

Hosier

(2016). Patterns and priorities of service need identified through the child and adolescent needs and strengths (CANS) assessment. Children and Youth Services Review, 60, 129–135.

18.

Courtney

M. E.

Needell

Wulczyn

(2004). Unintended consequences of the push for accountability: The case of national child welfare performance standards. Child and Family Services Review, 26(2), 1141–1154.

19.

Dilley

J. B.

Weiner

D. A.

Lyons

J. S.

Martinovich

(2007). The validity of the child and adolescent needs and strengths assessment [Poster presentation]. Convention of the American Psychological Association.

20.

Dorsey

Burns

Southerland

Cox

Wagner

Farmer

(2012). Prior trauma exposure for youth in treatment foster care. Journal of Child & Family Studies, 21(5), 816–824. https://doi.org/10.1007/s10826-011-9542-4

21.

Feinstein

A. R.

(1987). Clinicmetrics. New Haven, CT: Yale University Press.

22.

Gerrity

Folcarelli

(2008). Child traumatic stress: What every policymaker should know. National Center for Child Traumatic Stress.

23.

Grady

M. D.

Drisko

J. W.

(2014). Thorough clinical assessment: The hidden foundation of evidence-based practice. Families in Society: The Journal of Contemporary Social Services, 95(1), 5–14. https://doi.org/10.1606/1044-3894.2014.95.2

24.

Griffin

McClelland

Holzberg

Stolbach

Maj

Kisiel

(2011). Addressing the impact of trauma before diagnosing mental illness in child welfare. Child Welfare, 90(6), 69–89.

25.

A. S.

Lim

C. S.

Lecklitner

Olson

Traube

D. E.

(2015). Interagency collaboration and identifying mental health needs in child welfare: Findings from Los Angeles County. Children and Youth Services Review, 53, 39–43.

26.

Igelman

Taylor

Gilbert

Ryan

Steinberg

Wilson

Mann

(2007). Creating more trauma-informed services for children using assessment-focused tools. Child Welfare, 86(5), 15–33.

27.

Jaudes

P. K.

Weil

L. E. G.

Prior

J. M.

Sharp

D. P.

Holzberg

McClelland

G. M.

(2016). Wellbeing of children and adolescents with special health care needs in thechild welfare system. Children and Youth Services Review, 70, 276–283. https://doi.org/10.1016/j.childyouth.2016.09.024.

28.

Kataoka

S. H.

Zhang

Wells

K. B.

(2002). Unmet need for mental health care among US children: Variation by ethnicity and insurance status. American Journal of Psychiatry, 159, 1548–1555.

29.

Kerns

S. E.

Pullmann

M. D.

Negrete

Uomoto

J. A.

Berliner

Shogren

Silverman

Putnam

(2016). Development and implementation of a child welfare workforce strategy to build a trauma-informed system of support for foster care. Child Maltreatment, 21, 135–146. https://doi.org/10.1177/1077559516633307

30.

Kisiel

Summersett-Ringgold

Weil

L. E.

McClelland

(2017). Understanding strengths in relation to complex trauma and mental health symptoms within child welfare. Journal of child and family studies, 26(2), 437–451.

31.

Kisiel

Fehrenbach

Small

Lyons

J. S.

(2009). Assessment of complex trauma exposure, responses, and service needs among children and adolescents in child welfare. Journal of Child & Adolescent Trauma, 2(1), 143–160.

32.

Lang

J. M.

Randall

K. G.

Delaney

Vanderploeg

J. J.

(2017). A model for sustaining evidence-based practices in a statewide system. Families in Society, 98(1), 18–26.

33.

Lardner

M. D.

(2015). Are restrictiveness of care decisions based on youth level of need? A multilevel model analysis of placement levels using the child and adolescent needs and strengths assessment. Residential Treatment for Children & Youth, 32(3), 195–207. https://doi.org/10.1080/0886571X.2015.1080993

34.

Lehmann

Havik

O. E.

Havik

Heiervang

E. R.

(2013). Mental disorders in foster children: A study of prevalence, comorbidity and risk factors. Child & Adolescent Psychiatry & Mental Health, 7(1), 1–23. https://doi.org/echo.louisville.edu/10.1186/1753-2000-7-39

35.

Levitt

J. M.

(2009). Identification of mental health service need among youth in child welfare. Child Welfare, 88(1), 27–48.

36.

Lyons

J. S.

(2009). Communimetrics: A theory of measurement for human service enterprises. Springer.

37.

McCarthy

Marshall

Irvine

Jay

(2004). An analysis of mental health issues in states’ child and family service reviews and program improvement plans. National Technical Assistance Center for Children’s Mental Health, Georgetown University Center for Child and Human Development & American Institutes of Research.

38.

McMahon

R. J.

Forehand

R. L.

(2005). Helping the noncompliant child: Family-based treatment for oppositional behavior. Guilford Press.

39.

Nelson

T. D.

Steele

R. G.

(2008). Influences on practitioner treatment selection: Best research evidence and other considerations. Journal of Behavioral Health Services & Research, 35(2), 170–178. https://doi.org/10.1007/s11414-007-9089-8

40.

Nelson

T. D.

Steele

R. G.

Mize

J. A.

(2006). Practitioner attitudes toward evidence-based practice: Themes and challenges. Administration and Policy in Mental Health, 33(3), 398–409.

41.

Oswald

S. H.

Heil

Goldbeck

(2010). History of maltreatment and mental health problems in foster children: A review of the literature. Journal of Pediatric Psychology, 35(5), 462–472.

42.

Pecora

P. J.

Kessler

R. C.

Williams

O’Brien

Downs

A. C.

English

White

Hiripi

Roller White

Wiggins

Holmes

(2005). Improving family foster care: Findings from the northwest foster care alumni study. Casey Family Programs.

43.

Pecora

P. J.

Jensen

Romanelli

L. H.

Jackson

L. J.

Ortiz

(2009). Mental health services for children placed in foster care: An overview of current challenges. Child Welfare, 88(1), 5–26.

44.

Ponniah

Weissman

M. M.

Bledsoe

S. E.

Verdeli

Gameroff

M. J.

Mufson

Fitterling

Wickramaratne

(2011). Training in structured diagnostic assessment using the DSM-IV criteria. Research on Social Work Practice, 21(4), 452–457.

45.

Praed Foundation. (2016). Child and adolescent needs and strengths: 2016 reference guide. Chapin Hall.

46.

Randsalu

L. S.

Laurell

(2018). Children in out-of-home care are at high risk of somatic, dental and mental ill health. Acta Paediatrica, 107(2), 301–306.

47.

Romanelli

L. H.

Landsverk

Levitt

J. M.

Leslie

L. K.

Hurley

M. M.

Bellonci

Gries

L. T.

Pecora

P. J.

Jensen

P. S.

, & the Best Practices for Mental Health in Child Welfare Steering Committee (2009). Best practices for mental health in child welfare: Screening, assessment, and treatment guidelines. Child Welfare, 88(1), 163–188.

48.

Rosanbalm

K. K.

Snyder

E. H.

Lawrence

C. N.

Coleman

Frey

J. J.

van den Ende

J. B.

Dodge

K. A.

(2016). Child wellbeing assessment in child welfare: A review of four measures. Children & Youth Services Review, 68, 1–16.

49.

Salazar

A. M.

Keller

T. E.

Gowen

L. K.

Courtney

M. E.

(2013). Trauma exposure and PTSD among older adolescents in foster care. Social Psychiatry & Psychiatric Epidemiology, 48(4), 545–551.

50.

Sokol

R. L.

Victor

B. G.

Miller

A. L.

Ryan

J. P.

Perron

B. E.

(2020). Changes in child and adolescent needs and strengths (CANS) scores over time: A systematic review. Children and Youth Services Review, 112, 104917.

51.

U.S. Department of Health & Human Services, Administration for Children and Families, Administration on Children, Youth and Families, Children’s Bureau. (2018a). Child maltreatment 2017. https://www.acf.hhs.gov/cb/resource/child-maltreatment-2017

52.

U.S. Department of Health & Human Services, Administration for Children and Families, Administration on Children, Youth and Families, Children’s Bureau. (2018b). Child welfare outcomes 2015: Report to congress. https://www.acf.hhs.gov/sites/default/files/cb/cwo2015.pdf#page=12

53.

U.S. Department of Health and Human Services, Administration on Children, Youth and Families, Children’s Bureau. (2014). Screening, assessing, monitoring outcomes and using evidence-based interventions to improve the outcomes of children in child welfare. https://www.acf.hhs.gov/sites/default/files/cb/wp2_screening_assesing_monitoring.pdf

54.

Wotring

Kodges

Xue

Forgatch

(2005). Critical ingredients for improving mental health services: Use of outcome data, stakeholder involvement, and evidence-based practices. The Behavior Therapist, 28(7), 150–158.