Evaluating and Validating the Classification Accuracy of a Screening Instrument to Assess Risk for Commercial Sexual Exploitation of Child Welfare

Abstract

Objectives:

Identifying children and adolescents within child welfare at risk for commercial sexual exploitation (CSE) can ensure referrals to appropriate services. However, screening measures to understand the prevalence of CSE are missing in child welfare. We evaluated the classification accuracy of a screener developed for the purpose of this study, guided by the Sexual Exploitation among Youth (SEY) risk assessment framework used in practice with child welfare–involved young people, (1) to identify young people at high versus low risk for experiencing CSE and (2) to estimate the prevalence of CSE risk for child welfare–involved children and adolescents.

Methods:

We used extant data from the National Survey of Child and Adolescent Well-being study with a nationally representative sample of children and adolescents aged 11-17 years (n = 1054) investigated by child welfare from February 2008 to April 2009. The 26-item screener showed acceptable reliability (α = .73) and test-criterion validity evidence using a CSE proxy outcome (ie, narrowly defined as being paid for sexual relations). We used the receiver-operating curve to classify risk and calculate the optimal cutoff score.

Results:

Higher scores on the SEY screener (range, 0-20 points) increased the odds of experiencing CSE by 34%. The screener was good at discriminating CSE risk at the 6-point cutoff, with 26.7% of child welfare–involved young people identified as being at high risk for CSE.

Conclusions:

Given the absence of accurate prevalence rates of CSE risk in the population, a theoretical cutoff index using an established method can provide an objective decision on how to distinguish risk levels. Prevalence estimates for CSE risk highlight the need for systematic screening in child welfare to identify and provide services for young people at risk.

Keywords

commercial sexual exploitation of children child welfare CSEC screening instrument maltreatment

From 2018 to 2019, the US National Human Trafficking Hotline saw a 19% increase in contacts from survivors of human trafficking, most of whom were victims of sex trafficking.¹ Human trafficking (ie, sexual and labor exploitation) is a public health issue that affects individuals, families, and communities.² Children and adolescents who are trafficked, especially for commercial sexual exploitation (CSE), are at risk for developing mental and physical health problems.³ Those who experience CSE also have high rates of adverse childhood experiences,⁴ which are associated with long-term negative outcomes.⁵ Despite these outcomes, systematic identification of children at high risk for CSE is fraught with challenges related to accurate assessment, especially within child-serving systems. Therefore, prevention efforts are severely lacking, and the prevalence and incidence rates of victimization are not understood. Validation of accurate screening instruments, which have implications for identification of victimization rates and appropriate targets for prevention and intervention efforts, are greatly needed.²

In the United States, the Trafficking Victims Protection Act (TVPA)⁶ stipulated that for children and adolescents aged <18 years, use of force, fraud, or coercion does not need to occur (or be proven) for a commercial sex act to be defined as human trafficking. Furthermore, the Justice for Victims of Trafficking Act⁷ amended the Child Abuse Prevention and Treatment Act (CAPTA),⁸ requiring states to have additional provisions and procedures to identify children and adolescents who are suspected to be victims of sex trafficking. To do so, state child welfare systems amended child abuse statutes to include experiences of CSE. As of 2019, 33 states statutorily defined sex trafficking as a specific type of sexual abuse,^9,10 improving identification and tracking of CSE in those states. The 2015 CAPTA amendment also required child welfare workers to be trained in identification, assessment, and provision of comprehensive services for children and adolescents who are victims of sex trafficking. Despite statutory variability in defining human trafficking more broadly, and child sex trafficking specifically,¹⁰ recognition that CSE of children and adolescents is abuse and should be served by the child welfare system is widespread. It is noteworthy, however, that child welfare involvement also increases the risk for CSE.^11,12 This risk often relates to the traumatic experiences that precipitated entry into the child welfare system, along with related factors such as housing instability and continued experiences of maltreatment.^11,13 Therefore, the child welfare system likely serves many children and adolescents who are at high risk for CSE.

In addition to CAPTA amendments, efforts to increase identification and protection are currently mandated by the Preventing Sex Trafficking and Strengthening Families Act,¹⁴ amending part E (Foster Care and Adoption Assistance) of title IV (Temporary Assistance for Needy Families) of the Social Security Act.¹⁵ Specifically, state plans for foster care and adoption assistance need to include policies and procedures for identifying, documenting, and determining appropriate services for children and adolescents under state agency responsibility that may be at risk for, or victims of, sex trafficking.

Taken together, these policy amendments highlight the importance of identifying children at risk for CSE and the need to ensure that child welfare workers are trained in the assessment of risk for CSE. However, identification practices are rarely conducted systematically using a validated tool, increasing the risk of bias and inaccuracies in prevalence and incidence estimates. Accurate identification of risk for CSE is also necessary for referrals to appropriate services for children’s complex mental and physical needs (eg, neurological issues, physical injury, cardiovascular and gastrointestinal issues, dental issues, depression, suicidal ideation, and posttraumatic stress disorder).^16-18 In addition, children and adolescents with histories of CSE have an increased likelihood of having sexually transmitted infections, experiencing physical abuse, and using drugs or alcohol.¹⁹

CSE also increases the risk for involvement with law enforcement and juvenile justice systems. In some states, minors can be arrested for prostitution-related charges despite TVPA guidelines. Several states have enacted Safe Harbor Laws to protect children and adolescents from involvement in the justice system when they experience CSE; however, states vary in how they implement these laws (eg, full immunity, diversionary options, alternative services).²⁰ Despite these practices, children and adolescents are still served in juvenile justice because of factors that bring them to the attention of law enforcement related to their exploitative situations (eg, truancy or other low-level offenses,¹³ foster care placements,²¹ discrimination²²). Thus, identifying children and adolescents at risk for CSE in child welfare can prevent crossover into the justice system if prevention services are put in place. For example, part IV of the Families First Prevention Services Act²³ highlights prevention of CSE with a focus on serving children and adolescents at risk for CSE.

Currently, several disparate efforts in research and practice settings are developing screening tools for CSE.²⁴ Perhaps the most widely used is the Commercial Sexual Exploitation Identification Tool (CSE-IT) developed and validated by the WestCoast Children’s Clinic.²⁴ Greenbaum and colleagues²⁵ developed and validated a brief screening tool for use in health care settings, and the Vera Institute of Justice developed and validated a tool for human trafficking, although not validated with children and adolescents.²⁶ To date, only the CSET-IT has been scaled up to the state level, although implementation processes vary widely.

Accurately identifying children and adolescents at risk for CSE can lead to better estimates of prevalence rates that can subsequently inform trends in victim rates. Importantly, better estimates can target prevention (for those at risk for CSE) and intervention (for confirmed victims of CSE) services and guide allocation of funding for children and adolescents in need (eg, the US Department of Justice, Office of Justice Programs’ grants to combat human trafficking).²⁷ In addition, many jurisdictions and community-based programs have developed collaborative responses to serve children and adolescents who are at risk for, or confirmed victims of, CSE, contributing to a small but growing body of literature on best practices.^28-32 Yet, the scope of the problem is not well understood without prevalence or incidence data on CSE risk among children and adolescents in the child welfare system. To address these gaps, we evaluated and validated the classification accuracy of a screener developed for the purpose of this study, and guided by an assessment framework currently used with child welfare–involved children and adolescents, to identify high versus low risk for experiencing CSE. A secondary objective was to estimate the prevalence of CSE risk for child welfare–involved children and adolescents.

Methods

We selected the items for our screening instrument developed for this study based on the Sexual Exploitation among Youth (SEY) risk assessment framework developed by the Transition, Education, and Resources for Females (TERF) program³³ of New Directions in Manitoba, Canada. Child protection legislation in Canada, as in the United States, requires protecting children from CSE. Canada’s Protection of Communities and Exploited Persons Act³⁴ explicitly outlines that no person can purchase sexual services from children and adolescents aged <18 years. In addition, victims of CSE are often served by their child welfare systems. Since 1986, TERF has been providing services to child, adolescent, and adult victims of sex trafficking and CSE. The SEY tool was created in 2006 by staff members who were survivors of human trafficking. This tool has undergone several revisions based on consultation with experts and focus groups, providing content-related validity evidence. Content-related evidence can come from experts who develop and assign items to categories purported to measure the outcome of interest.³⁵ The SEY tool has 8 sections: abuse, absconding, addiction, behaviors and appearance, cognitive ability and mental health, relationships, exploitation, and self-worth/belonging, for a total of 39 items. Our instrument, generated for the purpose of this study from the SEY tool, included items across these sections except for self-worth/belonging. The SEY tool is typically completed by social workers or guardians as part of referral to the TERF program and to the child welfare StreetReach program. The StreetReach program is part of the Manitoba government’s interdisciplinary prevention strategy to address CSE victimization of children and adolescents.³⁶ The tool is scored by staff members using information obtained from the child’s or adolescent’s social history, during intake, and from other available sources (eg, social media).

Despite the current use of the SEY tool in community and child welfare practice, however, empirical evidence on the tool’s ability to discriminate between high and low risk for CSE is limited. Because access to a database of completed SEY assessments is not feasible, use of extant data from cohort 2 of the National Survey of Child and Adolescent Well-being (NSCAW),³⁷ a nationally representative longitudinal study of children and families involved with the child welfare system, was used. Specific to the current study, self-reported items from NSCAW were mapped onto similar items from the SEY tool across the 8 SEY sections to evaluate and validate the risk classification of our study-developed screener.

Participants

We restricted the analytic sample to 1054 children and adolescents aged 11-17 years (mean age = 14.5 y; SD = 1.85) investigated by child welfare from February 2008 through April 2009 (Table 1). Our study complied with relevant ethical standards for human subjects protections based on The Pennsylvania State University Institutional Review Board.

Table 1.

Demographic characteristics of the analytic sample (n = 1054) of children and adolescents aged 11-17 years investigated for alleged childhood maltreatment by Child Protective Services, February 2008–April 2009^a

Characteristic^b	No.
Race and ethnicity
Non-Hispanic White	400
Non-Hispanic Black	286
Hispanic	244
American Indian/Alaska Native	75
Asian/Native Hawaiian/Pacific Islander	34
Unknown	15
Placement type
Biological/adoptive parent	711
Foster care	130
Informal kin	80
Formal kin	67
Group home/residential program	56
Other	10
Sex
Female	584
Male	470
Maltreatment type
Physical abuse	221
Neglect (FTP and LoS)	211
Sexual abuse	117
Emotional abuse	53
Other	294
Unknown	5
Missing	153

Abbreviations: FTP, failure to provide; LoS, lack of supervision.

The analytic sample was drawn from the National Survey of Child and Adolescent Well-Being (NSCAW) study, a nationally representative, longitudinal study of children and families intended to understand the well-being, functioning, service needs, and service use of children and adolescents who come in contact with the child welfare system. Extant data from the NSCAW study are available from the National Data Archive on Child Abuse and Neglect.³⁶

Characteristics are not mutually exclusive. Information on sex rather than gender identity was limited by availability of existing data.

Measures

Indicators of risk

Mapping of NSCAW items occurred at the construct level (ie, SEY sections), resulting in an overlap of 26 items (Table 2). One point was given for an affirmative response to each item (eg, prior maltreatment, physical injury, hitchhiked). We calculated the total risk score by summing across all indicators (mean = 4.19 points, SD = 3.40). Cronbach’s alpha indicated acceptable reliability (α = .73) for the aggregated NSCAW items.

Table 2.

Self-reported commercial sexual exploitation risk indicators from a sample of children and adolescents aged 11-17 years (n = 1054) investigated for alleged childhood maltreatment by Child Protective Services, February 2008–April 2009^a

Item no.	SEY section	Risk indicator (SEY item)	Risk indicator (NSCAW item)^b
1	Abuse	History of maltreatment	Reports of prior maltreatment
2		Witnessed violence in the family	Active domestic violence
3		Child and family services involvement	Living in out-of-home care
4		Family impacted by poverty	Trouble paying for basic necessities
5	Absconding	Run away from home/placement	Run away from home in the past 6 mo
6	Addiction	Frequency of using drugs/alcohol	In a car driven by someone who was using alcohol/drugs
7			Used alcohol/drugs to feel better about self or fit in
8			Used alcohol/drugs while alone
9			Forgot things while using alcohol/drugs
10			Family/friends said to cut down on alcohol/drug use
11			Got into trouble while using alcohol/drugs
12		Intravenous administration of drugs	Used needles to inject illegal drugs
13	Behaviors and appearance	Skipped classes or school	Skipped classes or school in the past 6 mo
14	Behaviors and appearance	Signs of physical trauma	Physical injury
15	Cognitive ability and mental health	Diagnosed or suspected disability	Major special needs or behavior problems
16		Diagnosed or suspected disability	Intelligence quotient <70
17		Indicators of mental health concerns	Mood disorder
18			Internalizing problems (clinical range)
19			Externalizing problems (clinical range)
20			Attention problems (clinical range)
21		Signs of emotional trauma	Posttraumatic stress disorder symptoms (clinical range)
22	Relationships	Isolated or bullied	Felt alone at school
23	Exploitation	Acquiring unexplained items or cash	Begged for money or things in the past 6 mo
24		Received rides with unknown people	Hitchhiked in the past 6 mo
25		—	Forced first sexual activity
26		—	Age of first sex <14 y

Based on responses to the National Survey of Child and Adolescent Well-Being (NSCAW) study, a nationally representative, longitudinal study of children and families intended to understand the well-being, functioning, service needs, and service use of children and adolescents who come in contact with the child welfare system. Extant data from the NSCAW study are available from the National Data Archive on Child Abuse and Neglect.³⁶ Study items were guided by the Sexual Exploitation among Youth (SEY) risk assessment framework.³⁴

NSCAW risk indicator items were mapped on the SEY sections (ie, construct of interest) with one-to-one item correspondence matched when possible.

Child sexual exploitation

Because CSE determination was difficult to ascertain in the NSCAW dataset, we used a proxy measure for CSE as the reference standard for the outcome condition. Specifically, we measured CSE using a single item that asked, “In the past 6 months, have you been paid for having sexual relations with someone?” yielding a dichotomous yes/no response.

Analysis

We used logistic regression to predict the risk for CSE and provide test-criterion validity evidence based on the instrument’s relationship to other variables.³⁵ Because it is not appropriate for the Standards for Educational and Psychological Testing to “dictate minimal levels of test-criterion correlation (or) classification accuracy . . . for any given purpose,”^35(p141) we conducted a receiver-operating curve (ROC) analysis for guidance on classification accuracy (defined as the degree to which assignment of test takers to specific categories is accurate).³⁸ Specific to our study, ROC analysis will generate local cutoff scores to determine the degree to which a screener will identify children and adolescents who are at high versus low risk for CSE in the population.

To empirically determine the optimal cutoff distinguishing high versus low risk for CSE victimization, we calculated a Youden index (J), maximizing the balance between sensitivity and specificity.³⁹ Because true prevalence rates in the population are unknown, this index gives equal weight to sensitivity and specificity with no constraints on true prevalence rates in the population. That is, J provides a theoretical cutoff score to distinguish high and low risk and is defined as J = max^c[Se(c) + Sp(c) – 1], where Se = sensitivity and Sp = specificity, which ranges from 0 to 1 (higher values indicate better discrimination).

Finally, we used results from the ROC analysis and theoretical cutoff in subsequent analyses to calculate the validity indexes for screening accuracy (ie, sensitivity, positive predictive value [PPV], specificity, and negative predictive value [NPV]).³⁸ Sensitivity refers to the screener’s ability to correctly identify the proportion of children and adolescents at high risk for CSE as truly at high risk (ie, true positive). For children and adolescents classified as being at high risk for CSE, PPV is the probability that they are truly at high risk for CSE. Specificity is the screener’s ability to identify children and adolescents who are at low risk for CSE as truly at low risk for CSE (ie, true negative). For children and adolescents who are classified as not being at risk for CSE, NPV is the probability that they are not at risk for CSE.

Results

Total scores on the risk screener for the sample (n = 1053) ranged from 0 to 20 points (mean = 4.19; SD = 3.40). Results of the logistic regression showed that total risk score significantly predicted the likelihood of CSE (β = 0.290; SE = 0.05; odds ratio = 1.34 [95% CI, 1.22-1.46]; P < .001). Specifically, a 1-unit increase in the total score of the screener increased the odds of experiencing CSE by 34%. Results of the ROC analyses indicated that the screener was good at discriminating high versus low risk for CSE (area under the curve = 0.85 [95% CI, 0.77-0.92]; P < .001; Figure). (Additional tables with different values for Se, Sp, and J across different test totals [ie, 1-20] are available from the authors upon request.)

Figure.

The receiver-operating characteristic curve showing true-positive (sensitivity) versus false-positive (1 − specificity) rates of the screener correctly identifying the proportion of children and adolescents aged 11-17 years at high risk for commercial sexual exploitation. The hypothetical cutoff value of 6 points derived from the Youden index calculation (sensitivity = 0.83 [horizontal line], specificity = 0.74, 1-specificity = 0.26 [vertical line]) distinguishes those at high versus low risk. The dashed diagonal line represents the reference point where the classification occurs by chance alone.

Results of the Youden calculation (J = 0.57) indicated that an optimal threshold of 6 points hypothetically distinguished between children and adolescents at high versus low risk of experiencing CSE (Figure). Specifically, children and adolescents who responded affirmatively on any 6 items were considered at high risk for CSE (Tables 3 -5). This cutoff score yielded the following validity indexes: sensitivity = 83%, PPV = 7%, specificity = 74%, and NPV = 99%.

Table 3.

Descriptive information and inter-item correlations for commercial sexual exploitation outcome and each risk indicator (including missing data percentages) for the analytic sample (n = 1054) of children and adolescents aged 11-17 years investigated for alleged childhood maltreatment by Child Protective Services, February 2008–April 2009^a

Item no.	Risk items	Yes response, no. (%)	Legitimate skip/refused/don’t know, no. (%)^b	Missing or not interviewed, no. (%)	Risk item
Item no.	Risk items	Yes response, no. (%)	Legitimate skip/refused/don’t know, no. (%)^b	Missing or not interviewed, no. (%)	1	2	3
1	Child sexual exploitation	23 (2.2)	46 (4.4)	—	1.00
2	Reports of prior maltreatment	612 (58.1)	21 (2.0)	153 (14.5)	0.06	1.00
3	Active domestic violence	120 (11.4)	28 (2.7)	153 (14.5)	–0.03	–0.01	1.00
4	Living in out-of-home care	250 (23.7)	11 (1.0)	153 (14.5)	0.07^c	0.14^d	0.01
5	Trouble paying for basic necessities	208 (19.7)	39 (3.7)	153 (14.5)	–0.08^c	0.07	0.16^d
6	Run away from home	146 (13.9)	43 (4.1)	—	0.13^d	0.07^c	0.02
7	In a car driven by someone who was using alcohol/drugs	269 (25.5)	45 (4.3)	—	0.12	–0.01	–0.06
8	Used alcohol/drugs to feel better about self or fit in	137 (13.0)	556 (52.8)	—	0.23^d	0.12^c	–0.07
9	Used alcohol/drugs while alone	116 (11.0)	556 (52.8)	—	0.19^d	0.06	–0.08
10	Forgot things while using alcohol/drugs	114 (10.8)	556 (52.8)	—	0.19^d	0.03	–0.06
11	Family/friends said to cut down on alcohol/drug use	121 (11.5)	562 (53.3)	—	0.07	0.02	–0.05
12	Got into trouble while using alcohol/drugs	111 (10.5)	559 (53.0)	—	0.17^d	0.03	–0.06
13	Used needles to inject illegal drugs	24 (2.3)	38 (3.6)	—	0.24^d	0.04	–0.02
14	Skipped classes or school	207 (19.6)	45 (4.3)	—	0.10^e	0.04	–0.01
15	Physical injury	83 (7.9)	698 (66.2)	—	0.15^d	0.06	–0.03
16	Major special needs or behavior problems	313 (29.7)	18 (1.7)	153 (14.5)	0.10^e	0.12^e	–0.06
17	Intelligence quotient <70	136 (12.9)	—	45 (4.3)	0.08^e	0.01	–0.02
18	Mood disorder	128 (12.1)	—	18 (1.7)	0.07^c	0	–0.01
19	Internalizing problems	135 (12.8)	—	7 (0.7)	0.04	0.01	–0.05
20	Externalizing problems	255 (24.2)	—	7 (0.7)	0.05	0.03	–0.07
21	Attention problems	97 (9.2)	—	7 (0.7)	0.02	0.03	0.01
22	Posttraumatic stress disorder symptoms	86 (8.2)	—	14 (1.3)	0.03	0.03	–0.02
23	Felt alone at school	330 (31.3)	51 (4.8)	—	0.06	0.07^c	0.04
24	Begged for money or things	56 (5.3)	44 (4.2)	—	0.25^d	–0.02	–0.01
25	Hitchhiked	26 (2.5)	45 (4.3)	—	0.19^d	–0.06	–0.01
26	Forced first sexual activity	71 (6.7)	711 (67.5)	—	0.11^c	0.04	–0.02
27	Age of first sex <14 y	259 (24.6)	717 (68.0)	—	0.04	0.10	0.02

Abbreviation: —, not applicable.

This category of response is based on skip logic within the relevant section of the instrument.

Significant at P < .05; determined by Spearman correlation for categorical data.

Significant at P < .001; determined by Spearman correlation for categorical data.

Significant at P < .01; determined by Spearman correlation for categorical data.

Table 4.

Inter-item correlations for commercial sexual exploitation outcome and each risk indicator (including missing data percentages) for a sample (n = 1054) of children and adolescents aged 11-17 years investigated for alleged childhood maltreatment by Child Protective Services, February 2008–April 2009^a

Item no.		Risk item
Item no.		4	5	6	7	8	9	10	11	12	13	14
4	Living in out-of-home care	1
5	Trouble paying for basic necessities	0.05	1
6	Run away from home	0.04	–0.03	1
7	In a car driven by someone who was using alcohol/drugs	0.13^b	0.02	0.25^b	1
8	Used alcohol/drugs to feel better about self or fit in	0.16^c	–0.05	0.26^b	0.40^b	1
9	Used alcohol/drugs while alone	0.18^b	–0.05	0.19^b	0.40^b	0.53^b	1
10	Forgot things while using alcohol/drugs	0.10	–0.01	0.17^b	0.44^b	0.39^b	0.42^b	1
11	Family/friends said to cut down on alcohol/drug use	0.09	–0.04	0.19^b	0.30^b	0.48^b	0.44^b	0.50^b	1
12	Got into trouble while using alcohol/drugs	0.06	–0.09	0.21^b	0.38^b	0.42^b	0.41^b	0.52^b	0.51^b	1
13	Used needles to inject illegal drugs	0.05	–0.03	0.21^b	0.17^b	0.22^b	0.19^b	0.24^b	0.20^b	0.20^b	1
14	Skipped classes or school	0.07^d	–0.02	0.35^b	0.27^b	0.20^b	0.20^b	0.22^b	0.13^c	0.15^c	0.05	1

Significant at P < .001; determined by Spearman correlation for categorical data.

Significant at P < .01; determined by Spearman correlation for categorical data.

Significant at P < .05; determined by Spearman correlation for categorical data.

Table 5.

Item no.		Risk item
Item no.		15	16	17	18	19	20	21	22	23	24	25	26
15	Physical injury	1
16	Major special needs or behavior problems	0.06	1
17	Intelligence quotient <70	0.01	0.17^b	1
18	Mood disorder	0.07^b	0.13^b	0.02	1
19	Internalizing problems	0.03	0.10^b	–0.01	0.52^b	1
20	Externalizing problems	0.06	0.11^b	–0.03	0.35^b	0.39^b	1
21	Attention problems	0.07^b	0.11^b	0.03	0.39^b	0.42^b	0.37^b	1
22	Posttraumatic stress disorder symptoms	0.07^b	0.05	–0.05	0.35^b	0.43^b	0.19^b	0.30^b	1
23	Felt alone at school	0.04	0.09^b	0.01	0.28^b	0.22^b	0.12^b	0.20^b	0.16^b	1
24	Begged for money or things	0.04	0.09^b	–0.02	0.08^b	0.03	0.14^b	0.07^b	0.04	0.02	1
25	Hitchhiked	0.14^b	0.03	0.02	0.10^b	0.05	0.14^b	0.08^b	0.12^b	0.03	0.26^b	1
26	Forced first sexual activity	0.21^b	0.09	0.05	0.16^b	0.14^b	0.11^b	0.11^b	0.12^b	0.07	0.12^b	0.10	1
27	Age of first sex <14 y	0.09	0.13^b	–0.05	0.02	0.02	0.01	0	0.05	0.02	0.06	0.13^b	0.17^b

Significant at P < .001; determined by Spearman correlation for categorical data.

Significant at P < .01; determined by Spearman correlation for categorical data.

Significant at P < .05; determined by Spearman correlation for categorical data.

Using a theoretical cutoff value of 6 points to recode the total score for our sample by creating a binary measure to indicate high or low risk for CSE, we found that 26.7% (n = 281) of the sample was at high risk for CSE.

Discussion

We sought to evaluate and validate the classification accuracy of a screener to assess risk for CSE among a representative sample of child welfare–involved young people. Results provided validity evidence for the test-criterion relationship and discriminated between young people at low and high risk for CSE. These results provide evidence that the indicators used from the NSCAW data, guided by the SEY tool, were successful at identifying young people at high risk for CSE. Systematic use of validated screeners enhances our understanding of the prevalence and scope of CSE risk. Using a cutoff score of 6 points, our results indicated that 26.7% of our sample of child welfare–involved young people should be categorized as high risk for CSE. This percentage is likely a conservative estimate because the reference standard used insinuated monetary payments for sexual relations. The TVPA states that monetary exchange for sex is not needed for CSE classification. Therefore, young people could exchange anything of value (eg, place to sleep, meal, any item of commercial value) for what is often referred to as survival sex, severely underestimating true population rates.

These results provide preliminary evidence that the SEY tool, which was the guiding framework for the indicators in our study, is good at identifying young people at high risk for CSE in practice. The tool is currently in use across Manitoba, Canada, and was developed based on the practice and lived experience of CSE. Although the need for systematic and validated screening is great, caution must be taken during implementation. Specifically, the information obtained through a screener must be used proactively to prevent CSE and ameliorate its developmental impact rather than further negatively labeling children and adolescents. Unfortunately, young victims have experienced judgment from child welfare staff members and service providers when labeled as CSE.^13,40 Therefore, screeners must be used as a guide for services within emerging best practices for working with children and adolescents who are at high risk for CSE (eg, harm reduction,⁴¹ multidisciplinary teaming,³² trauma-focused interventions^28,42) and prioritizing referral and access to services once children and adolescents are identified. Assessing the utility of the actual SEY assessment, including protocols for how the information is used to inform prevention, is a fundamental next step for future research. Further research is also needed to understand how the accuracy of risk classification may be influenced by the information source. Specifically, the original SEY tool used in practice is completed by caseworkers, whereas our study used self-report measures. Therefore, a subsequent multi-informant approach may provide convergent and divergent information on risk indicator responses.

Limitations

This study had several limitations. First, it was limited to child welfare–involved children and adolescents and captured only a subpopulation of the total population of young people at high risk for CSE, although many are also involved with other child-serving systems (eg, juvenile justice, schools), exposed to other risks such as experiencing homelessness, or not in contact with child welfare. In addition, the CSE reference standard we used was limited by a single outcome question about payment for sexual relations that could be interpreted narrowly by respondents. Future research should replicate our findings among representative samples from other child-serving systems and community-based samples to assess for similarities and differences in risk and protective factors. Future research should also seek to further understand how and why children and adolescents with similar risk profiles are not victimized, given that the PPV of the screener in our study was only 7% despite identification of >250 children and adolescents being at risk for CSE. This finding likely means that about 90% of young people identified as being at high risk for CSE did not self-report CSE based on our narrow reference standard. Future work should include a more robust reference standard for confirmed CSE (eg, CSE proxy that also includes other forms of nonmonetary exchange-based CSE that may not necessarily be considered survival sex) that would allow a more nuanced examination of possible differences in these risk trajectories (ie, risk and protective factors) or whether the reference standard in the outcome was not capturing actual CSE.

Second, our study was focused on sex trafficking rather than human trafficking broadly (ie, labor trafficking), limiting our understanding of shared and unique risk factors to understand the nuanced needs of children and adolescents. For example, child welfare–involved children and adolescents at risk for labor trafficking tended to be younger and more frequently Black and Hispanic when compared with young people identified for sex trafficking.⁴² Therefore, future instrument development and validation should include the broader construct of human trafficking and include labor trafficking and sex trafficking dimensions, further assessing structural validity.

Third, findings from our study were limited to a sample of young people aged 11-17 years, and because CSE risk can begin at a younger age,⁴³ younger cohorts should be examined. Relatedly, subpopulations that are disproportionately represented in child welfare (eg, young people in foster care or other out-of-home care, girls/women, African American children) are known to be at high risk for CSE.^40,44 In addition, we were limited to a binary categorization of sex rather than the full spectrum of gender identities, which limited our ability to assess for gender differences. Potential differences in risk for subpopulations should be assessed in future research to identify differences in risk prevalence and to assess whether the same risk indicators are similarly predictive for these subpopulations.

Finally, the composite score we used assumed equal weighting of items and, therefore, should not be shortened. Given evidence of differential item weighting in other studies,⁴⁵ however, future work should use item response theory–derived weights to calculate composite scores. Doing so will provide evidence on discrimination and difficulty/risk at the item level that can inform which items are substantially contributing to risk and which items can be removed to shorten the screener. Relatedly, results from our study were based on a previously developed screener and mapped onto an existing dataset using a proxy outcome measure for CSE. These results need to be replicated using data that are collected prospectively and across multiple informants.

Conclusions

Our study provides important contributions to our understanding of CSE risk for children and adolescents in child welfare. Specifically, results of our study provided evidence of validity and reliability for a screener, guided by a tool in practice, that can distinguish among child welfare–involved children and adolescents who are at high versus low risk for experiencing CSE. Given the absence of accurate prevalence rates of CSE risk in the population, our study also demonstrated how a theoretical cutoff index using an established method can provide an objective decision on where to draw the line distinguishing risk levels and further assess the screener’s validity index. Finally, using the theoretical cutoff of 6 points, our study showed that more than one-quarter of the child welfare–involved children and adolescents in our nationally representative sample were at high risk for CSE, providing preliminary prevalence estimates for risk of CSE. Such estimates highlight the need for systematic, validated screening efforts in child welfare to identify young people at risk for CSE so that appropriate prevention and intervention efforts can be identified.

Footnotes

Acknowledgements

The data in this study were made available (in part) by the National Data Archive on Child Abuse and Neglect, Cornell University, Ithaca, New York.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Eunice Kennedy Shriver National Institute of Child Health and Human Development (grant Nos. P2CHD041025, P50HD089922) and the Social Science Research Institute at The Pennsylvania State University.

ORCID iD

Carlomagno C. Panlilio, PhD

References

Polaris. The U.S. National Human Trafficking Hotline: 2019 Data Report. Published 2020. Accessed March 28, 2021. https://polarisproject.org/wp-content/uploads/2019/09/Polaris-2019-US-National-Human-Trafficking-Hotline-Data-Report.pdf

National Academies of Sciences, Engineering, and Medicine. Estimating the Prevalence of Human Trafficking in the United States: Considerations and Complexities: Proceedings of a Workshop–in Brief. National Academies Press; 2019.

Cole

Sprang

Lee

Cohen

. The trauma of commercial sexual exploitation of youth: a comparison of CSE victims to sexual abuse victims in a clinical sample. J Interpers Violence. 2016;31(1):122-146. doi:10.1177/0886260514555133

Naramore

Bright

Epps

Hardt

. Youth arrested for trading sex have the highest rates of childhood adversity: a statewide study of juvenile offenders. Sex Abuse. 2017;29(4):396-410. doi:10.1177/1079063215603064

Felitti

Anda

Nordenberg

, et al. Relationship of childhood abuse and household dysfunction to many of the leading causes of death in adults: the Adverse Childhood Experiences (ACE) Study. Am J Prev Med. 1998;14(4):245-258. doi:10.1016/s0749-3797(98)00017-8

Victims of Trafficking and Violence Protection Act, Pub L No 106-386, 2 USC §7102 (2000).

Justice for Victims of Trafficking Act, Pub L No 114-22 (2015).

Child Abuse Prevention and Treatment Act, Pub L No 93-247, 42 USC §5101-51116i (2017).

Child Welfare Information Gateway. Definitions of Child Abuse and Neglect. US Department of Health and Human Services, Children’s Bureau; 2019.

10.

Child Welfare Information Gateway. Definitions of Human Trafficking. US Department of Health and Human Services, Administration for Children and Families, Children’s Bureau; 2020.

11.

Dierkhising

Geiger

Hurst

Panlilio

Schelbe

. Preventing adolescent maltreatment: a focus on child welfare, juvenile justice, and sexual exploitation. In: Daro

Cohn Donnelly

Huang

Powell

, eds. Advances in Child Abuse Prevention Knowledge. Springer International; 2015:43-65.

12.

Bounds

Julion

Delaney

. Commercial sexual exploitation of children and state child welfare systems. Policy, Polit Nurs Pract. 2015;16(1-2):17-26. doi:10.1177/1527154415583124

13.

Dierkhising

Walker-Brown

Ackerman-Brimberg

Newcombe

. Commercially Sexually Exploited Girls and Young Women Involved in Child Welfare and Juvenile Justice in Los Angeles County: An Exploration and Evaluation of Placement Experiences and Services Received. National Center for Youth Law, California State University; 2018. Accessed October 4, 2021. https://youthlaw.org/wp-content/uploads/2018/11/CSEC-Research-Report_Placement-Exp-Svcs-Recd__NCYL_Cal-State.pdf

14.

Preventing Sex Trafficking and Strengthening Families Act, Pub L No 113-183 (2014).

15.

Social Security Act, 42 USC §438 (1935).

16.

Lederer

Wetzel

. The health consequences of sex trafficking and their implications for identifying victims in healthcare facilities. Ann Health Law. 2014;23:61-91.

17.

Judge

Murphy

Hidalgo

Macias-Konstantopoulos

. Engaging survivors of human trafficking: complex health care needs and scarce resources. Ann Intern Med. 2018;168(9):658-663. doi:10.7326/M17-2605

18.

Ross

Dimitrova

Howard

Dewey

Zimmerman

Oram

. Human trafficking and health: a cross-sectional survey of NHS professionals’ contact with victims of human trafficking. BMJ Open. 2015;5(8):e008682. doi:10.1136/bmjopen-2015-008682

19.

Varma

Gillespie

McCracken

Greenbaum

. Characteristics of child commercial sexual exploitation and sex trafficking victims presenting for medical care in the United States. Child Abuse Negl. 2015;44:98-105. doi:10.1016/j.chiabu.2015.04.004

20.

Gies

Bobnis

Cohen

Malamud

. Safe Harbor Laws: Changing the Legal Response to Minors Involved in Commercial Sex, Phase 1. The Legal Review. Development Services Group, Inc; 2018. Accessed March 28, 2021. https://www.ojp.gov/pdffiles1/ojjdp/grants/253146.pdf

21.

Ryan

Marshall

Herz

Hernandez

. Juvenile delinquency in child welfare: investigating group home effects. Child Youth Serv Rev. 2008;30(9):1088-1099. doi:10.1016/j.childyouth.2008.02.004

22.

Ocen

. (E)racing childhood: examining the racialized construction of childhood and innocence in the treatment of sexually exploited minors. UCLA Law Rev. 2015;62:1586-1640.

23.

Families First Prevention Services Act, Pub L No 115-123 (2018).

24.

Basson

. Validation of the Commercial Sexual Exploitation–Identification Tool (CSE-IT): Technical Report. WestCoast Children’s Clinic; 2017.

25.

Greenbaum

Dodd

McCracken

. A short screening tool to identify victims of child sex trafficking in the health care setting. Pediatr Emerg Care. 2018;34(1):33-37. doi:10.1097/PEC.0000000000000602

26.

Simich

Goyen

Powell

Mallozi

. Improving Human Trafficking Victim Identification: Validation and Dissemination of a Screening Tool. VERA Institute of Justice; 2014. Accessed March 28, 2021. https://www.ncjrs.gov/pdffiles1/nij/grants/246712.pdf

27.

US Department of Justice. Awards: listing of funded projects. Accessed December 1, 2020. https://nij.ojp.gov/funding/awards/list

28.

Basson

Langs

Acker

Katz

Desai

Ford

. Psychotherapy for Commercially Sexually Exploited Children: A Guide for Community-Based Behavioral Health Practitioners and Agencies. WestCoast Children’s Clinic; 2018.

29.

Epstein

Edelman

. Blueprint: A Multidisciplinary Approach to the Domestic Sex Trafficking of Girls. Center on Poverty and Inequality at Georgetown Law School; 2014.

30.

Muraya

Fry

. Aftercare services for child victims of sex trafficking: a systematic review of policy and practice. Trauma Violence Abuse. 2016;17(2):204-220. doi:10.1177/1524838015584356

31.

Richardson

. Providing a Path: Exploratory Promising Practices for Working With Sexually Exploited/Trafficked Children and Youth. Masters Thesis, University of Manitoba; 2017. Accessed March 28, 2021. https://mspace.lib.umanitoba.ca/bitstream/handle/1993/32804/Richardson%20Jennifer.pdf;jsessionid=7EF3D64CDED6B1D4B73259FA94692C39?sequence=2

32.

Walker

. Ending the Commercial Sexual Exploitation of Children: A Call for Multi-system Collaboration in California. California Child Welfare Council; 2013. Accessed March 28, 2021. http://www.youthlaw.org/fileadmin/ncyl/youthlaw/publications/Ending-CSEC-A-Call-for-MultiSystem_Collaboration-in-CA.pdf

33.

Tracia’s Trust. Collaboration and Best Practices to End Sexual Exploitation and Sex Trafficking in Manitoba. Manitoba Government; 2019. Accessed October 6, 2021. https://www.gov.mb.ca/fs/traciastrust/pubs/tracias_trust_report_2019.pdf

34.

Protection of Communities and Exploited Persons Act, Bill C-36; SC 2014, c. 25 (2014).

35.

American Educational Research Association, American Psychological Association, National Council on Measurement in Education, Joint Committee on Standards for Educational and Psychological Testing. Standards for Educational and Psychological Testing. American Educational Research Association; 2014.

36.

Manitoba Families, Child and Family Services Division. StreetReach Program. Accessed October 6, 2021. https://www.gov.mb.ca/fs/traciastrust/street_reach.html

37.

US Department of Health and Human Services, Children’s Bureau, National Data Archive on Child Abuse and Neglect. National Survey of Child and Adolescent Well-Being (NSCAW). Accessed October 6, 2021. https://www.ndacan.acf.hhs.gov/datasets/datasets-list-nscaw.cfm

38.

Krzanowski

Hand

. ROC Curves for Continuous Data. Monographs on Statistics and Applied Probability: Vol. 111. Taylor & Francis; 2009.

39.

Fluss

Faraggi

Reiser

. Estimation of the Youden Index and its associated cutoff point. Biom J. 2005;47(4):458-472. doi:10.1002/bimj.200410135

40.

Dierkhising

Walker Brown

Ackerman-Brimberg

Newcombe

. Recommendations to improve out of home care from youth who have experienced commercial sexual exploitation. Child Youth Serv Rev. 2020;116:105263. doi:10.1016/j.childyouth.2020.105263

41.

Hickle

Hallett

. Mitigating harm: considering harm reduction principles in work with sexually exploited young people. Child Soc. 2016;30:302-313. doi:10.1111/chso.12145

42.

Cohen

Mannarino

Kinnish

. Trauma-focused cognitive behavioral therapy for commercially sexually exploited youth. J Child Adolesc Trauma. 2017;10(2):175-185. doi:10.1007/s40653-015-0073-9

43.

Gibbs

Aboul-Hosn

Kluckman

. Child labor trafficking within the US: a first look at allegations investigated by Florida’s child welfare agency. J Hum Traffick. 2020;6(4):435-449. doi:10.1080/23322705.2019.1594551

44.

Development Services Group Inc. Commercial Sexual Exploitation of Children and Sex Trafficking: Literature Review. Office of Juvenile Justice and Delinquency Prevention; 2014. Accessed March 28, 2021. https://www.ojjdp.gov/mpg/litreviews/CSECSexTrafficking.pdf

45.

Panlilio

Miyamoto

Font

Schreier

HMC

. Assessing risk of commercial sexual exploitation among children involved in the child welfare system. Child Abuse Negl. 2019;87:88-99. doi:10.1016/j.chiabu.2018.07.021

Evaluating and Validating the Classification Accuracy of a Screening Instrument to Assess Risk for Commercial Sexual Exploitation of Child Welfare–Involved Children and Adolescents

Abstract

Objectives:

Methods:

Results:

Conclusions:

Keywords

Methods

Participants

Measures

Indicators of risk

Child sexual exploitation

Analysis

Results

Discussion

Limitations

Conclusions

Footnotes

Acknowledgements

Declaration of Conflicting Interests

Funding

ORCID iD

References