Issues in Differential Response

Literature and Research Assessment

The research studies and published literature included in this review were identified by searching several online databases, including EBSCOHOST (accessed through OhioLink, Ohio’s university library search interface); Google Scholar; the Cochrane Collaboration Registry; and resources provided by the American Humane Association (AHA); and the National Quality Improvement Center on Differential Response in Child Protective Services (NQIC-DR). The search terms we used to identify relevant sources of conceptual and practice literature were “Alternative Response” OR “Differential Response” OR “Multiple Response” AND “Child Abuse” OR “Child Neglect” OR “Child Maltreatment.”

The specific database, search terms, and search strategies used to locate research and evaluation reports differed for each database and were based on the requirements of each search utility. Search terms used to access research and evaluation reports are outlined in Table 1.

OPEN IN VIEWER

Table 1.

Databases and Corresponding Search Terms.

Database	SearchTerms
EBSCOHOST (includes the following):	((Differential OR Multiple OR Alternative OR (Family Assessment)) AND Response) AND (child AND ((protective service) OR welfare))
Academic Search Complete
Medline
Psychology and Behavioral Sciences Collection
Social Work Abstracts
SocINDEX
Sociological Collection
Social Sciences Abstracts
Cochrane Register	(child protection) AND [(Alternative Response) OR (Differential Response) OR (Family Assessment Response)] AND (abuse OR neglect OR maltreatment) in Title, Abstract or Keywords
Google Scholar- Social Sciences	child AND (Differential Response) AND ((protective OR welfare)AND services)
	child AND (Alternative Response) AND ((protective OR welfare)AND services)
	child AND (Family Assessment Response) AND ((protective OR welfare)AND services)

We included both published and unpublished research reports in our review, and we obtained original evaluation reports produced by the researchers who conducted the studies rather than relying on other authors’ interpretations of study results.

We used a snowball method to identify additional resources from bibliographic citations in procured documents. We reviewed DR-related newsletters, conference brochures and presentations, training curricula, workshop handouts, resources listed on the NQIC-DR website, and special issues of the AHA’s publication, Protecting Children, that were devoted to DR-related articles. We identified additional research reports by asking key informants during telephone interviews to provide or direct us to reports from research that had been conducted in their state or jurisdiction. A total of 39 program, practice, and conceptual publications and 18 research reports were ultimately included in our review.

Sample

Our key informant sample included all the states we could identify that either operated a DR program at the time of sampling or had previously operated a DR program but had discontinued it. These states were identified through federal and state documents and websites, reports generated by AHA, and from the attendance roster from the National Differential Response Conference held in 2008 in Columbus, OH. Once the states were identified, we attempted to contact the most appropriate respondents, many of whom were identified either in AHA reports or on the 2008 conference roster. Ultimately, 35 states were identified as having some involvement with DR programs. We included respondents from state agencies and/or from local implementing agencies to ensure a variety of perspectives on both program design and implementation. We attempted to contact potential respondents by telephone and e-mail. If a person of interest was no longer available, we requested contact information for the most knowledgeable person/persons to talk with about their DR program, and we were usually directed to an appropriate respondent. We also identified some of the respondents by e-mailing the state office responsible for child welfare services at their general inquiry address. Of the original 35 selected jurisdictions, 7 did not respond to our queries, and 8 were considering DR but had not yet implemented it. Our final key informant sample included respondents from 20 states, all of whom completed interviews.

Interview Protocol

Key informant interviews were conducted using a semistructured interview format guided by a standardized questionnaire with 47 primarily open-ended questions. The broad categories for inquiry included program description, track assignment, staffing, staff training, substantiation and documentation, case process, program evaluation, implementation issues and challenges, and general feedback. A copy of the interview protocol is included in Appendix C.

Interview Process

We collected interview data between August 2010 and January 2011. The interviewers explained and clarified the purpose of the study and the intended use of the data. Key informants were told that the data would be used for a research project to ultimately guide child welfare practice and to enhance training as it relates to DR. Participants were informed that the interview would take between 30 and 60 min, and they were given the option to schedule their interview at a convenient time. Some of the respondents asked whether they could include colleagues in their interview to allow for a variety of perspectives, and we scheduled teleconferences to accommodate these small group interviews. In three instances, key informants asked whether they could review the topics and questions prior to the interview, and we provided this information. We began each interview with the same questions, but as the interviews progressed, the responses of the informants guided the order in which we asked the remaining questions. We encouraged participants to elaborate on their answers to elicit more complete information. Interviews were from 45 min to an hour in length and in all but one case were conducted by telephone. One interview was conducted in person. Some respondents asked to review the questionnaire following the interview to ensure that we had accurately recorded their data. We e-mailed completed questionnaires to these respondents with instructions to provide us with corrections as needed. These corrections were incorporated into the data.

We elected to maintain the confidentiality of all key informants, even though when asked, most of them did not feel that confidentiality was a concern. The assurance of confidentiality was intended to encourage key informants to provide honest, detailed, and accurate information. We did not evaluate any state individually, nor did we discuss any information about client families, individual case circumstances, or the work of individual staff.

Validity of Data

The project staff used several strategies to ensure the accuracy of key informant data and to limit subjectivity. During telephone interviews, the interviewers used low inference (concrete and unambiguous) descriptors and captured direct quotes from respondents whenever possible. Data gleaned from the interviews was qualitative in nature and was compiled using only thematic analysis. We did not attempt any quantification of interview responses.

Data Analysis and Conclusions

The research method was designed to triangulate data from multiple sources, thereby increasing the accuracy of the information and findings. The research reports were reviewed independently by three experienced child welfare researchers, two of whom were external to IHS/NARCCW. The final written report was reviewed by all members of the research team and by selected external reviewers to further ensure its accuracy. Names and affiliations of collaborating research specialists/reviewers are listed in the authors’ notes at the end of this article.

Finding #1: DR Programs do not Adhere to a Uniform, Standardized Practice Model, nor are Programs Implemented Consistently Across Sites

Our research determined that while underlying principles of DR have been widely articulated in the DR literature, there is considerable variation in how DR programs have been defined, designed, and implemented across states and agencies and over time. Many of these differences were also documented in the National Study on Differential Response in Child Welfare, a survey of 27 states and 2 counties completed in 2006 by the Child Welfare League of America and the AHA (Merkel-Holguin, Kaplan, & Kwak, 2006). In 2008, Kaplan and Merkel-Holguin reported that states were still using “different terminology, definitions, models, services, and service providers, all while shaping the evolution of the approach” (p. 6). They further contended that a clear understanding of the DR approach has been challenged by the lack of common terminology and definitions.

We identified the following differences in how DR programs had been designed and implemented across jurisdictions.

Discussion

Based on our assessment, DR is a compilation of practice philosophy and inconsistently defined practice principles rather than a clearly delineated program intervention model. This appears to have been at least partially intentional. Kaplan and Merkel-Holguin (2008) contend that because of the complexity inherent in any assignment schema, and given the varying circumstances of individual families and children, flexibility in implementation is one of the DR program’s strengths. They further contend that clinical judgment and discretion of individual caseworkers are of great importance, as there are “few hard and fast rules that cannot be altered given the practice wisdom of a specific worker and the approval of a supervisor” (p. 11).

Arguments in support of flexibility in program implementation and service delivery should not be misconstrued as a valid rationale to compromise consistency in the design, implementation, and evaluation of a program model. The success of any program model depends on both theoretical and applied consistency in its definitions, structural design, implementation strategies, and performance standards. In our study, we found very little consistency in how DR reform efforts were undertaken across jurisdictions. There was no consistency in definitions or program components; no clearly defined standards; no established procedures; and wide variances in implementation strategies—in other words, no consistent DR model. Each implementing jurisdiction essentially began its program development without benefit of a standardized program manual to guide implementation and to promote fidelity in implementation. Therefore, implementing jurisdictions often differed in significant ways from previously implemented reform efforts as well as among and within programs or agencies in a single jurisdiction. These differences were not recognized as a fatal threat to effective model development. We did not find support in the DR literature that the model was shaped by empirical science and built incrementally based on the experiences of prior pilot sites with the capacity to test and standardize processes that best achieved desired outcomes. Without a consistent program model which is implemented with fidelity across jurisdictions and comparably evaluated, it is impossible to draw valid conclusions about the effectiveness, benefits and limitations of DR.

Finding #2: Methodological Problems in the DR Research Limit Confidence in Research Findings and Conclusions

Eighteen research reports of DR programs were identified for inclusion in our analysis. Two of these studies used exclusively qualitative methods. Of the 16 research studies that included at least some quantitative analysis, 2 may be classified as observational, 3 as experimental (1 was a follow-up study), and 11 as quasi-experimental. In our review, we identified significant problems in research methodology and implementation that presented threats to internal, external, and construct validity, thereby calling into question the reliability and accuracy of many of the claims and conclusions made in these studies.

Internal validity refers to a study’s ability to demonstrate that a variable or phenomenon being evaluated is really responsible for the study findings and outcomes, rather than some other unidentified confounding variable. We identified threats to internal validity in all of the studies in this body of research. The scope and scale of these threats to internal validity varied among the individual studies. In several of the studies, DR implementation had been paired in time with some other large-scale policy or practice change. One example was the entire system reform that occurred in North Carolina. In this case, DR implementation occurred simultaneously with a new practice model, the introduction of new interventions such as family team meetings, shared parenting meetings in placement cases, a redesign of in-home services, and a more strengths-based structured intake decision-making process (Comer & Vassar, 2008). With changes occurring concurrently and none being effectively isolated in the research, it is impossible to know which change strategies or interventions may have influenced the study outcomes. Further, the definition of the “DR intervention” actually included a variety of interventions that differed among jurisdictions, including screening decisions, track assignment, initial caseworker contact, safety assessment, safety planning, risk assessment, comprehensive family assessment, identification of service needs, service referral, and service delivery, none of which were the same among all the studies we reviewed. The research did not isolate these processes and evaluate them independently, so a claim that DR was responsible for observed outcomes fails to clarify exactly which elements, or groups of elements, had which specific influences, if any, on which observed outcomes.

Threats to external validity were present in all of the studies in our review. External validity refers to the degree to which study findings can be generalized to populations other than the study sample. The absence of uniformity and consistency in programming across DR implementation sites would, itself, be sufficient to caution that no study’s outcomes can be generalized to other sites. Further, none of the studies in our analysis used random sampling procedures to select the study sample. The two experimental studies and one follow-up study did not use random sampling to select the study sample, but they did randomly assign members of the initial study sample to the two comparison groups (Institute of Applied Research, 2004b, 2006, 2010). Because of pervasive threats to external validity in all of these studies, we cannot with confidence assert that the study samples were representative of the entire population of families served in the child welfare system, thereby preventing generalization of these studies’ findings to other program implementation sites.

Construct validity refers to clarity and consistency in the definitions and measures of both the interventions and the outcomes being evaluated. The primary factor affecting construct validity in much of the DR research was that DR and its components were not clearly or consistently defined or implemented, making it impossible to isolate exactly what factors were included in the variables being measured. As indicated earlier, the implementation sites studied by this body of research had considerable latitude in deciding how to define and implement the components of their DR programs. As a result, key programmatic and measurement constructs such as “child safety,” the “assessment track,” “engagement,” “traditional track,” and “family satisfaction,” differed both within and between studies. Therefore, we cannot be certain that the measures used in these studies accurately represented the variables being researched, nor can we assume that a study’s findings would be relevant to other jurisdictions or useful in shaping ongoing and future reform efforts.

While some of the studies controlled for differences between the experimental and control groups, most did not. For instance, some of the studied jurisdictions allowed caseworkers in their agencies to choose whether to work in the traditional or the alternative track. As a result of self-selection, the two groups of caseworkers were likely not equivalent, potentially resulting in unequal treatment of the families in the AR and TR tracks for reasons other than DR programming. For example, individuals who chose to work in DR may have had qualities or characteristics, education, or work histories that affected their job performance and its subsequent effects on families. DR workers may have been more or less experienced, more or less committed to the child protection field, perhaps had different skills or had received special training, or may have had differing personal values about families in the child welfare system and their capacity to change. Any of these variables could have influenced outcomes and confounded the research. However, these individual and group differences were neither clearly identified nor isolated and measured, making it unclear whether DR program components or qualities of the caseworkers—or even some other factor—was associated with the observed outcomes, whether positive or negative.

An additional concern is related to how initial track decisions were made. In 13 of the 18 studies we reviewed, screeners had attempted to separate high-risk families from families at low- or moderate-risk in order to make track assignments. The initial assignment of families to tracks was generally based on screener recommendations, but as noted earlier, these decisions were often made without the use of standardized tools or criteria, raising question about the equivalence of the families within each of the tracks and, therefore, of the two research comparison groups. It is entirely possible that the comparison groups exhibited potentially significant differences from the onset of the research, including different risk potentials and different propensities to recidivate, both of which were primary outcome variables measured in many of the research designs. In four of the studies (Institute of Applied Research, 2004b, 2006, 2010; Ruppel, Huang, & Haulenbeek, 2011) families screened in as eligible for AR were then randomly assigned to AR or TR tracks, thereby potentially improving equivalence between the two study groups, but only if the screening criteria were well defined, consistently applied, and valid measures of risk.

Other methodological issues threatened the equivalence of the study groups. Many jurisdictions reportedly had provided intensive training to caseworkers in the alternative track in competencies related to engagement, assessment, and service planning but did not provide this training to workers in the traditional track (Institute of Applied Research, 1997, 2004b, 2010). These competencies are fundamental to social work practice and, therefore, are essential to effective work in either track. Social workers in the TR track may have been at a disadvantage because they had not been as well trained, and there is no way to know whether they might have demonstrated increased positive outcomes if they had also received this essential training in core skills.

Many of the tools and instruments in this body of research presented potential threats to internal validity. In most of the reports we reviewed, there was no discussion of how the instruments, questionnaires, and scales used in the study had been tested to ensure either validity or reliability. In one study, some of the instruments were scaled with only positive or negative choices, forcing respondents to choose one or the other, even if they were uncertain, ambivalent, or indifferent, which potentially skewed the data toward the poles (Institute of Applied Research, 2010). Failure to validate study measurement instruments increases the potential for ambiguity and measurement error.

Discussion

Field-based research cannot be as well controlled or as rigorously implemented as research conducted in laboratory settings. A primary deterrent to achieving rigor in field-based research is the presence of variables in the study environment that cannot be identified, isolated, or measured, yet which may strongly influence both program implementation and study outcomes. To claim or imply that an intervention being evaluated is responsible for observed outcomes without fully considering the potential role of these other variables can provide a distorted and exaggerated picture of the state of our knowledge about an intervention’s effectiveness.

Both researchers and consumers must use caution when interpreting research findings or using research data to inform critical policy, program, and resource decisions. It is concerning that in most of the studies we reviewed, the claims presented in the research reports frequently overreached what might have legitimately been concluded, considering their many methodological limitations.

Finding #3: There is Insufficient Data to Confirm the Safety of Children Served in Alternative Tracks

The potential to compromise children’s safety remains a primary concern when a CPS program chooses to forgo investigations or other forms of targeted case fact-finding in child maltreatment cases. The DR program literature clearly contends that child safety is of prime importance in both AR and TR tracks, and that the DR research has sought to confirm the safety of children served in alternative tracks. Some of the research included in our analysis did, in fact, attempt to evaluate child safety and concluded that children served in alternative tracks were as safe as children served in traditional, investigation tracks (Center for Child and Family Policy, 2009; Institute of Applied Research, 2004b). Based on our analysis, there is insufficient data to warrant such conclusions. This does not necessarily mean that children in AR tracks are unsafe. It does mean that child safety is not being uniformly assessed, accurately measured, or fully addressed in either DR programming or research.

Our review identified several references in the DR literature that raised the issue of safety of children served in alternative DR tracks. In one study, 37% of those surveyed believed that AR “probably” or “definitely” increased children’s safety. About a third (30.5%) of surveyed community service providers expressed “some” concern about the safety of children served in alternative tracks, while 8.4% expressed “major concern” (Institute of Applied Research, 2004b). Schene (2005) suggested that “many of the families diverted from CPS . . . are not adequately assessed or served by the community, and children continue to be at risk” (p. 6), especially when the CPS agency closes its case. She also recommended that community agencies become more sensitive to protection issues so that emerging risks in the families they serve can be rapidly recognized and addressed.

Discussion

In order to ensure that children served in alternative tracks are safe, safety and risk assessments must be an ongoing casework activity that is incorporated at all stages of case assessment and service planning. Further, the tools used to assess risk of both imminent and future harm must include pretested, validated, and standardized criteria, or safety assessments may not be valid. This is particularly important in DR programs, since the level of risk drives the decision of whether to bypass an investigation and serve a family in an alternative track. Inaccurate safety assessment can potentially lead to inaccurate track assignments.

There are many factors operating in DR programming that potentially contribute to inaccurate assessments of children’s safety. The first concern is whether child safety can be accurately assessed at the time of screening. Information about the family being referred is generally gathered by screeners without the benefit of direct contact with family members or alleged child victims. At times, referents may have limited or inaccurate information about the family and may not know what information is most relevant to report. A screener’s assessment of children’s safety must therefore be considered preliminary, even in the best of circumstances.

The screening function in child maltreatment agencies was never intended to make a clear determination about child maltreatment or risk. A screener’s recommendation to accept a case is an affirmation that there is sufficient potential that a child is at risk to warrant agency follow-up, but that additional case fact-finding is necessary to confirm or rule out the presence of threats to child safety. Yet, screeners in DR programs typically make recommendations for case acceptance, establish the priority for agency response, and recommend a track assignment based on information collected in a referral telephone interview—information that is typically limited in scope and depth and potentially in accuracy.

This is exacerbated by a lack of consistency in the criteria used to make track assignments in many DR programs. Our key informants reported that rather than being directly measured, the level of risk in a family was often inferred from the type of maltreatment alleged in the referral. For example, in many jurisdictions, reports of sexual abuse, severe physical abuse, or child fatalities were automatically tracked to investigation. How serious or severe the maltreatment needed to be to warrant an investigation was not clearly defined. Some states referred all very young child victims to the investigation track, presumably because young children are more vulnerable and, therefore, at higher potential risk of harm. However, this does not consider the possibility of protective capacities in a family that might quickly eliminate any safety threats, therefore obviating the need for an investigation. One state reportedly tracked all neglect referrals to AR and all abuse referrals to TR (Key Informants), even though at least as many children die from neglect as from abuse (AHA: Child Welfare Information Gateway, 2012). Another criteria reportedly used by some agencies for track assignment was the position or role of the person who made the referral. Shusterman, Hollinshead, Fluke, and Yuan (2005) and Yuan (2005) reported that referrals from parents, caregivers, friends, schools, or children were more likely to be assigned to the alternative track, whereas reports from community professionals were more likely to be assigned for investigation. Yuan (2005) also noted that some states may have policies against tracking families to AR if they were referred by certain mandated reporters. We know of no research that correlates the type of referent with child safety.

Adding to the challenge of accurately assessing safety at the time of screening is the apparent latitude given to staff making the track assignment decisions. Some of our key informants reported that their jurisdictions used formal safety protocols or standardized screening tools to assess children’s safety at the time of intake. Others referred only to legislation, administrative rule, or general practice guidelines, none of which was designed to assess children’s safety in individual cases. Research reported over a decade ago by English, Wingard, Marshall, Orme, and Orme (2000) stressed the necessity of establishing consensus in definitions of serious or imminent risk of harm, since track assignments in DR programs were generally based on these constructs, and a lack of consistency in definitions could undermine the accuracy of track decisions. The researchers’ concerns originated from some serious initial allegations that had been classified as low risk and subsequently referred to the alternative track—and, in the words of the researchers, reflected “a serious disregard for the health and well-being of these children or the potential for serious harm” (English, Wingard, Marshall, Orme, & Orme, 2000, p. 387). Waldfogel (1998) similarly asserted that the key to DR program effectiveness is an agency’s ability to accurately sort cases by level of risk.

Further contributing to inaccuracy and inconsistency of track decisions is the lack of training and skill of some of the personnel in screening positions. Because of the inherent complexity of making child safety determinations in the screening environment, all screeners should be skilled clinicians with well-developed interviewing and assessment skills. In some agencies, particularly those with limited staff resources, administrative assistant staff sometimes cover the screening function and gather initial referral information before transferring it to casework or supervisory staff to make decisions (Key Informants).

A more critical issue is that child safety can fluctuate, perhaps significantly and often within relatively short time frames, in response to changes in family composition or circumstances, environmental conditions, or the presence, capacities, and stability of primary caregivers. If track assignment is based solely on an assessment of a child’s condition at the time of referral, children identified as currently safe may actually be at high risk of future harm, and these children may be referred to the alternative track without recognition of the longer term safety concerns. Since formal risk assessment to estimate the potential for serious future harm is not a standard feature of screening, screeners cannot always accurately determine how much protection a child may need in the near or distant future. This is because formal risk assessments—particularly the more reliable and valid actuarial risk assessments—are difficult to complete fully or accurately without face-to-face family contact and a deeper exploration of a family’s circumstances, including the presence and dynamics of previous maltreatment (Baird & Wagner, 2000; Baird, Wagner, Healy, & Johnson, 1999; Rycus & Hughes, 2003).

Inaccurate assessments of children’s safety and risk of future harm not only affect the accuracy of track assignments—they also affect all subsequent case decisions. This is of considerable concern in a dichotomous tracking system such as DR, where everyone expects that families determined to be at “high risk” will be tracked to investigation, and “lower risk” families will be tracked to AR. Therefore, if the information gathered by the screener does not reach the threshold necessary to trigger an investigation, the family may automatically be perceived as “lower risk,” and by inference, the children thought to be “safer” than those children referred for investigation. Caseworkers may not be as vigilant in watching for and identifying emergent safety threats or a reduction in a family’s protective capacities.

Other factors inherent in DR philosophy and policy may also affect children’s safety. A defining characteristic of the alternative DR track is that caseworkers neither substantiate allegations of child maltreatment, nor do they confirm the perpetrator. If substantiate were defined solely as legally documenting an alleged maltreatment incident, and if not confirming a perpetrator were narrowly defined as not entering perpetrators’ names into a Central Registry database, AR caseworkers could comply with these requirements and still complete an accurate fact-finding assessment of both immediate child safety and future risk. However, there are practical differences in how these core DR principles are operationalized by the agencies and caseworkers who implement them, with many claiming it unnecessary (and inherently disrespectful) to push families to talk about an alleged maltreatment incident or to determine who was responsible for its occurrence.

DR research reports and program literature provide examples of minimizing, and even discouraging exploration of specific incidents of child maltreatment in the interest of remaining family-friendly and supportive. For example:

With AR we identify strengths, we don’t focus on an incident that brought the family to child protection . . . . (A caseworker, quoted in a study completed by Institute of Applied Research, 2004b, p. 49)

The allegation is not the reason for services. It is the reason we are coming to the home, but it’s more about what we can do to help. . . . Sometimes I will tell them, “We don’t really have to talk about that [the allegation]—What can I do to help you?” (A caseworker, quoted in a study completed by Institute of Applied Research, 2009, p. 34)

We also found many claims related to children’s safety in the DR research, including the assertion that children served in the AR track were as safe as or safer than children served in the TR track (Institute of Applied Research, 2004b; Schene, 2008) The DR research has attempted to measure child safety in a variety of ways, but none of the studies used standardized and validated measures or safety assessment protocols implemented by well-trained raters.

In this body of research, one often-used measure of child safety was recurrence rates—that is, the percentage of cases that receive a new report and/or a substantiation of maltreatment within a designated follow-up period. In these studies, recurrence rates observed in AR families were compared to recurrence rates observed in the TR comparison group. One study that evaluated data from the National Child Abuse and Neglect Data System (NCANDS) found that for all but one state in their six-state evaluation, the rates of maltreatment recurrence were comparable for children receiving an alternative response and children receiving an investigation (Shusterman, Hollinshead, Fluke, & Yuan, 2005). Other studies have documented similar recidivism rates between lower risk AR cases and higher risk TR cases (Center for Child and Family Policy, Duke University, 2009; Institute of Applied Research, 2000, 2004b). Both DR research reports and the associated DR program literature use this data to support their contention that being served in an AR track—foregoing identification of a perpetrator and fact-finding about the dynamics of child maltreatment—does not compromise children’s safety (Institute of Applied Research, 2000, 2004b; Schene, 2008).

Presuming that families are accurately assigned to tracks based on a valid assessment of risk level, and given the restrictions on the families that are even eligible for assignment to AR (families at low to moderate risk), one would expect maltreatment recurrence rates in this group to be low to begin with, regardless of the level of service provided. We would also expect that their recurrence rates would be lower than the recurrence rates in families tracked to TR, simply because high risk is defined as a higher estimated likelihood of maltreatment recurrence (Baird & Wagner, 2000; Rycus & Hughes, 2003). Even in those studies in which the subjects were randomly assigned to AR (experimental) and TR (control) groups, the study sample was not randomly selected: both study subgroups were drawn from an initial sample of lower risk families who had been identified as eligible for AR (Institute of Applied Research, 2004b, 2006, 2010). Thus, one would expect to see comparable recurrence rates in the two comparison groups, regardless of whether they were served in the AR or TR track. Finally, because of typically low rates of maltreatment recurrence in lower risk families, any attempt to obtain accurate statistical comparisons about recurrence rates would be confounded by the low base rate of maltreatment incidents in these groups. For all the above reasons, lower observed recurrence rates in AR families may not be a meaningful measure of the effectiveness of the AR program in ensuring children’s safety.

Some of the studies we reviewed attempted to augment their measures of recurrence rates by using caseworkers’ ratings of children’s safety. The caseworkers’ opinions about changes in safety status were sampled at the beginning and at the end of the service period. Improvements in these “before and after” ratings were then presented as evidence that children’s safety had improved under AR (Institute of Applied Research, 2004b). This type of measure is weak at best and is very prone to bias, particularly since standardized and objective safety assessment protocols were not always used to inform the ratings. Caseworkers, who had invested considerable time and resources into serving these families, cannot be considered objective or disinterested evaluators, and the potential for bias in their ratings can significantly distort research findings.

The inherent challenges in gathering accurate child safety information at screening would be of less concern if caseworkers in the alternative track quickly and routinely continued the assessment of both imminent and future risk for all their cases. In traditional CPS investigations, caseworkers are expected to conduct in-person interviews with all family members, meet and talk with alleged child victims and siblings, gather information from collateral sources, screen for other types of maltreatment, gather in-depth data to assess risk, and initiate safety plans when needed. As noted above, our research suggests that these activities are not always implemented in the alternative track and in fact, may often be discouraged (Institute of Applied Research, 2004b, 2009). Only half the states represented by our key informants used standardized safety and risk assessment protocols with families served in the alternative track, and only four states (Minnesota, Ohio, North Carolina, and Vermont) and some California AR counties reported use of risk and safety assessment protocols that had been empirically validated. The fact that families assigned to the alternative track have already been designated “lower risk” may, itself, be a disincentive to adequate assessment of specific family dynamics related to risk, particularly if this is perceived as “off putting” to families. Further, it is unlikely that such in-depth scrutiny occurs in community-based service agencies, since their staff may not receive then necessary specialized training nor be given tools with which to monitor risk or to deal with more serious safety concerns as they emerge.

More important, the principles that underlie DR programming may prevent a thorough assessment of risk and safety from occurring in alternative tracks. DR’s stated preference that workers focus on family needs rather than incidents of maltreatment could clearly discourage practitioners from having the sometimes difficult conversations with families that are necessary to fully assess risk and to address safety concerns.

The term incident focused emerged in the late 1990s on the heels of the development of the first child welfare risk assessment protocols. At the time, eliciting information about a family’s maltreatment history and exploring the specific details of a particular maltreatment incident were purported to alienate families and shut down progress in making constructive changes. Rather than exploring the past, caseworkers were encouraged to focus on protecting children from future harm, and a conceptual dichotomy resulted in which practitioners were encouraged to become risk focused rather than incident focused.

The old dichotomy of risk versus incident seems to have reappeared in DR rhetoric but with a subtle shift. Alternative track caseworkers are now instructed to focus on family strengths and needs rather than seeking information about maltreatment incidents. Loman and Siegel reference this DR principle when they contend that the alternative track focuses on “looking forward with the family rather than looking back at an incident” (Institute of Applied Research, 2004b, p. 82). This message appears to have shifted assessment in alternative tracks away from actively exploring the dynamics of maltreatment with families.

The fallacy of this approach is that it is not possible to accurately assess risk of maltreatment recurrence without understanding the nature and dynamics of prior incidents of child maltreatment. Developers of actuarial and other research-based risk assessment models widely confirm that a prior history of child maltreatment is the single factor most highly correlated with future maltreatment (Baird & Wagner, 2000; Children’s Research Center, 2003; Children’s Research Center, 2009; Johnson, 2004; Sledjeski, Dierker, Brigham & Breslin, 2008). Risk assessment protocols that do not incorporate this information have reduced predictive validity.

Equally important, exploring maltreatment dynamics with family members is not an end in itself but is part of a broader family assessment designed to inform subsequent service planning and delivery. If the specific factors, conditions, and dynamics that elevate risk in a family are not accurately identified and fully understood, it is significantly more difficult to provide relevant services to remediate these conditions to prevent maltreatment recurrences (Rycus & Hughes, 1998). Caseworkers cannot help parents strengthen their protective capacities and reduce risk if the parents do not recognize or understand the safety threats operating in their family and how these are expressed. If workers do not know who was caring for a child when maltreatment occurred, or the circumstances of its occurrence, it is difficult to help families construct and implement safety plans to avoid similar situations in the future. This essential information is only available through an exploration and assessment of the dynamics of prior maltreatment, and it is the caseworker’s job to help family members explore and better understand these difficult topics in an environment characterized by honesty, understanding, compassion, and support.

Effective social work relies on building trusting worker–family relationships, which enable caseworkers to empower families, build on their strengths, address their needs, and help them ensure their children’s safety well into the future. Such relationships allow for the honest and realistic exploration of family needs, strengths, and concerns within a context of trust and respect, where family members can participate as partners. Social work is based on the premise that clients have both the capacity and the right to benefit from honest and forthright communication. These fundamental underlying principles make social work—and specifically strengths-based social work—the preferred methodology to bring about constructive personal and interpersonal change. Avoiding unpleasant discussions may make both caseworkers and families feel more comfortable, but depending on case circumstances, it may be dishonest, condescending, and ultimately less helpful toward a goal of helping families acquire specific strategies to reduce risk and ultimately become more effective parents.

Assessing children’s safety is an evolving process. Each contact with a family is a targeted opportunity to continue the case fact-finding that is essential for identifying risk of both imminent harm and future harm. For as long as CPS has involvement with any family, caseworkers must remain constantly open and alert for information that could change their assessment of a child’s safety and when called for, must begin immediate safety planning. By establishing a precedent from the first contact with families that risk factors, maltreatment dynamics, and family protective capacities are important and ongoing topics of discussion, caseworkers can remove much of the stigma and can empower parents to recognize and consider emerging risks and to participate in developing safety plans to address them.

Based upon our analysis, we conclude that the many claims found in DR research studies and program literature related to children’s safety are overreaching. Given the lack of consistent definitions, criteria, and tools to assess safety, the methodological problems in the research, the lack of consistent exploration and fact-finding in many AR tracks about prior maltreatment, and the inconsistent construction and implementation of DR programs from state to state, it is not possible to conclude that a DR model exists that can ensure that children’s safety is not compromised in alternative tracks. Based on currently available data, we simply cannot know whether being served in an alternative track is undermining children’s safety.

Finding # 4: DR Programs Appear to Prioritize Allocating Services and Resources for Families in Alternative Tracks

One of the potential disadvantages of DR is the allocation of already-limited fiscal and service resources to serve populations of lower risk children and families. DR was originally designed as an alternative to investigation for accepted reports of child maltreatment (Schene, 2005). The intent was to prevent lower risk families from being subjected unnecessarily to a CPS investigation, to allow more flexibility when offering services to these families (Child Welfare Information Gateway, 2008), and to more quickly link families with needed services (Key Informants). However, as DR reform has evolved, the targeted recipients in many jurisdictions have been broadened to include families whose CPS reports were not accepted by CPS agencies, but who were believed to be in need of supportive or protective services nonetheless. In some jurisdictions, alternative tracks have been created primarily to provide prevention services to at-risk families without any CPS involvement (Child Welfare Information Gateway, 2008; Conley & Duerr Berrick, 2010; Johnson, Sutton, & Thompson, 2005). This raises concern about the consequences of allocating resources to families in situations not sufficiently serious to warrant CPS intervention, when most CPS agencies operate in an economic environment that limits, often severely, the availability of resources to serve higher risk families and children.

There is considerable support in the DR literature for including lower risk and non-CPS families in DR’s target populations. In 2005, Schene reported, “No state has adequate resources to respond to all the accepted reports of child maltreatment with a careful consideration of risks and needs,” so families with less severe allegations may be simply closed after an investigation, and “. . . children are reported multiple times before a careful, face-to-face assessment is conducted” (p. 4). She contends that the rate of service provision in traditional CPS is characteristically low, with fewer than 30% of maltreatment reports being substantiated, and even fewer cases being opened for services. She concludes that the CPS system provides minimal services to families except in more serious situations where out-of-home placement is needed, and she views this as a primary rationale for the development of DR (Schene, 2005). The federal government concurs that one goal of DR has been to serve low- or moderate-risk families who might not qualify for services under traditional CPS programming (Child Welfare Information Gateway, 2003). Similarly, Connell, Bergeron, Katz, Saunders, and Tebes (2007) contend that unsubstantiated CPS referrals may still represent maltreatment cases that would benefit from services and that DR is an appropriate intervention for these families.

Seven of the states in our key informant interviews operated more than two tracks in their DR programs. The additional tracks were often managed by community service providers, sometimes independent of CPS monitoring or involvement. In some of these tracks, if families declined services, no additional CPS action was taken. Thompson, Siegel, and Loman (2008) relate that Minnesota’s DR program was a means to “respond early to a broader set of families and prevent conditions that are harmful to children” (p. 23). This program targeted the 60% of reported families who were screened out of the child welfare system because there was no specific alleged incident of child maltreatment or because there was insufficient information to warrant a formal CPS response. California reportedly developed a DR track to provide early intervention to families who would not typically receive services (Bagdasaryan, Furman, & Franke, 2008; Thompson, Siegel, & Loman, 2008). In these programs, DR was identified as a means of serving families who were historically underserved or closed out of services. This is consistent with the stated DR goal of front-loading services for lower risk and traditionally underserved families, even though many families in traditional CPS may not receive the services they need because the CPS agencies and communities that serve them lack the resources to effectively do so.

The practice of diverting CPS resources targeted to serve higher risk families and using them instead to serve lower risk families is troubling, when it is at the expense of the higher risk, core populations that CPS agencies are mandated to serve, particularly since many high-risk children depend on CPS intervention to ensure their safety and well-being. This in no way suggests that prevention and early intervention services for lower risk families are not needed—they absolutely are. In the best of circumstances, both child abuse prevention and family support services would be available to all families that needed them, in all communities. However, in an environment of chronically limited resources, CPS agencies should carefully consider the consequences of diverting their resources to serve lower risk families in alternative tracks, when families in traditional tracks cannot be adequately served because of insufficient fiscal, staffing, and community service resources.

Finding #5: DR Literature Misrepresents Traditional CPS to Enhance an Alternative Response Model

A primary concern in both the DR research and its associated program literature is the misrepresentation of traditional CPS to enhance the alternative track. In many documents we reviewed, the traditional, or investigation track was often characterized by pejorative descriptors. The NQIC-DR describes TR as “inflexible,” “adversarial,” and “not able to provide sufficient services” (2009, p. 1). Kirk (2008) claims that TR “is, by its very nature, judgmental, legalistic, and adversarial” (p. 71). Kaplan and Merkel-Holguin (2008, p. 7) contend that TR is “an approach where parents are ‘investigated’ in a quasi law-enforcement method” that is “intrusive” and “threatening” to families. Other authors follow suit and assign similar attributes to TR (Institute of Applied Research, 2004b, 2009; Schene, 2001, 2005) In none of these sources did we find any empirical evidence cited to support these claims.

Kaplan and Merkel-Holguin (2008) also characterize AR as categorically different from TR by contrasting the “values” of the two approaches. They claim that AR values “Engagement versus adversarial approach”; “Services versus surveillance”; “Label of ‘in need of services/support’ versus ‘perpetrator’”; “Encouraging versus threatening”; “Identification of needs versus punishment”; and “Continuum of response versus one size fits all” (p. 7). This paragraph alone would lead a reader to conclude that traditional child welfare practice was and is an adversarial, antagonistic, threatening approach to families, with primary goals of surveillance, of labeling, shaming and punishing offenders, and of using a single intervention strategy, devoid of services, with every family served.

Discussion

Most CPS professionals can provide examples of patently bad practice that is consistent with the above descriptions. However, this characterization does not define traditional CPS. The qualities attributed to TR in the above depictions represent a compilation of practice problems and shortcomings that exist to varying degrees in various places but not in the simplistic and harmful characterization presented. The negative and unprofessional qualities so widely ascribed to traditional child protection in this body of literature are stereotypic generalizations.

This negative portrayal of TR does offer a way to exemplify and amplify the presumed benefits and qualities of DR reform. By attributing patently negative characteristics to traditional CPS, DR proponents can build a paradigm that dichotomizes the traditional and alternative tracks and then relegates them to mutually exclusive categories, thereby creating a “good–bad” comparison. This strategy creates a straw man by presenting distorted, exaggerated, or misrepresented information about TR that can then be used to discredit it and highlights the more positive attributes of the AR track.

As with all straw man constructs, the AR versus TR paradigm can have powerful negative consequences. Straw man depictions are easy to tear down, and it is easy to rally support against them. And because this distorted representation is not an accurate representation of TR, accepting the distortion as fact prevents a legitimate, balanced assessment of the real strengths and limitations of both AR and TR. Further, it also obscures the many inherent commonalities in the two approaches. More accurate characterization of the AR and TR tracks, in philosophy and in fact, is essential if we are to construct testable models of practice, precisely because an honest and balanced assessment of current practice is a fundamental prerequisite and provides the foundation for progressive change and improvement.

Fundamentally, AR and TR have much in common. For the past 20 years, the mainstream CPS literature, university social work education, and state and provincial CPS in-service training programs have promoted an approach to working with families that much more closely parallels the values and methods of AR than the distortions and misrepresentations portrayed as traditional practice. As stated earlier, during the past two decades, family-centered, strengths-based practice principles provided the foundation for revised child welfare practice models that drove child welfare reform initiatives in many states and provinces. The fundamental positive tenets of DR are neither new nor revolutionary in child welfare practice. However, as is true with any large-scale system change, success in achieving a service system based on these principles depends largely on agency leadership and commitment, the availability of financial and staffing resources, legislative and policy support, and the skill of administrators in implementing and sustaining complex structural changes to make best practice possible. These elements have not always been present in the environments where such change is attempted. In this context, the DR reform movement has been a positive force in promoting infrastructure change to support family-centered and strengths-based interventions for families served in child welfare systems.

The original and most identifiable difference between the alternative and traditional tracks is the elimination of a formal investigation for families with less serious maltreatment allegations and a presumed lower risk of future harm to children. The stated rationale for creating the alternative track is that investigations undermine a caseworker’s capacity to engage and partner with families because investigations are overly intrusive and designed to establish blame. While investigations have the potential to be inherently more confrontational than friendly home visits by a supportive and engaging family advocate, primary purpose of a social work CPS investigation is fact-finding, not faultfinding. Even if caseworkers—perhaps in collaboration with law enforcement—must sometimes use forensic interviewing strategies, evidence-preserving data gathering, legal intervention, and monitoring strategies to ensure that children are not re-abused or neglected, a skilled social work investigator can often do this without harming or alienating family members.

Fact-finding regarding child maltreatment dynamics is only the first step in serving family members where child maltreatment has been identified. The more compelling question is, what happens to families after such fact-finding has been completed? The characteristics defined as the “core values” of AR by Kaplan and Merkel-Holguin (2008, p. 7) also form the core values of all family-centered child welfare interventions, and they are especially important for families in the traditional track. With few exceptions, families served in CPS agencies have a right to receive strengths-based, empowering, supportive, and developmental child welfare services regardless of the seriousness of the referral allegations or prior instances of maltreatment. The goal of all child welfare interventions is to help families build their protective capacities and enable them to safely care for their own children. Only when this is clearly impossible, should we consider more intrusive and disruptive interventions such as out-of-home placement.

We must question why this principle is not more widely acknowledged in the DR literature. Are engagement, empowerment, and strengths-based services inappropriate for higher risk families? Is there valid data to confirm that higher risk families cannot be engaged, that they do not have strengths and protective capacities, or that they cannot participate in devising solutions to their own needs and problems? If family-centered, strengths-based practices are preferred because the profession believes them to be more effective and ethical than using unilateral authority to enact family change, don’t the majority of members of higher risk families deserve the same opportunity to benefit from these modalities as do members of low- to moderate-risk families? And even when it is not possible or appropriate to engage perpetrators of serious child maltreatment into collaborative working partnerships, shouldn’t we provide this opportunity to nonoffending parents, children, and extended family members? In some very severe cases, the ongoing use of mandated authority and legal intervention will clearly be necessary to ensure children’s safety, but these families form only a subset of the families served in the traditional track, particularly when families are tracked to TR based solely on the original referral allegation before more in-depth family assessment data have been collected.

This concern raises other pertinent questions, including, exactly how different are the services provided to families in the AR and TR tracks? And, more importantly, how different should they be? In some organizations that claim to have fully implemented family-centered and strengths-based practice models, caseworkers report that they do not view AR as being all that different from what they normally do. In one research study, respondents referred to AR as “TR light” (Institute of Applied Research, 2004b, p. 4). It is clear that many respondents in states with a history of adopting family-centered practices do not view AR as significantly different from TR practice, which suggests they understand the potential benefits of family-centered models to helping most families, including those whose initial contact with the child protection agency required an investigative approach (Key Informants.)

Investigating alleged maltreatment is not the only thing, nor even the primary thing that happens in a traditional child protection track in CPS organizations that promote family-centered practice. Once past the investigation, the remainder of work in a TR track should follow pretty much lockstep with the AR track—beginning with a comprehensive and collaborative assessment of family strengths, needs, and risk, followed by safety planning when needed, service planning, service referrals, ongoing follow-up, and reassessment. While some child protection agencies may still complete an investigation and follow-up solely with monthly monitoring home visits, this has not been considered good child welfare practice for a very long time. A caseworker’s failure to engage families in the collaborative completion of a strengths-need assessment and development of a relevant service plan is a failure of execution or implementation—not a defining characteristic of traditional child protection. Further, promoting stereotypes about TR communicates that the negative characterizations are not only real, but are necessary and appropriate for high-risk cases, which is patently untrue. Family-centered social work must be our first choice of intervention for all families, regardless of risk level.

A related concern is the following assertion from the DR literature. Kaplan and Merkel-Holguin claim that caseworkers working in AR reported being more satisfied with AR because they were performing “real” social work (2008, p. 12). It may be easier to engage and partner with families who have less complex histories and less challenging issues, but “real” social work is the professional capacity to work successfully with those families whose needs are not so easily met or whose problems are not so easily remedied. The implication that it is not possible to do “real” social work with families served in TR tracks is a misleading overgeneralization. It is precisely this aspect of professional child welfare social work that sets it apart from the other disciplines serving maltreated children and their families, and which makes it the appropriate profession to serve as the lead provider in the child welfare field of practice.

The negative portrayal of TR to promote AR serves no useful purpose. It undermines the contributions of child welfare professionals who have worked to integrate principles of family-centered and strengths-based practice into their day-to-day work in the only track available to them at the time and often in systems with infrastructures that were more restricting than enabling.

Finally, we found many additional examples of inaccurate, unsupported, and unfounded promotional claims about DR programming in both DR research reports and the associated program literature. A more detailed delineation of some of these can be found in Appendix B.