Organizational Capacity for Youth Engagement: A Multi-Site Child Welfare Training-and-Coaching Model Evaluation

Abstract

Purpose

To assess the effects of a child welfare workforce training-and-coaching model on workers’ perceptions of individual and organizational capacity to engage youth in case/permanency planning.

Method

Five state pilot sites—Michigan, Missouri, Montana, Nebraska, and Rhode Island—selected intervention and comparison regions based on population density and characteristics. Intervention groups received the training-and-coaching model; comparison groups received no intervention. Pre–post surveys were completed (N = 703) between January 2023 and September 2024. Analysis of covariance models were used to assess outcomes.

Results

Intervention groups showed small, significant improvements in caseworkers’ self-perceptions of confidence, skills, and agency culture supportive of youth–adult partnership; in supervisors’ and caseworkers’ commitment to meaningful youth–adult participation; and in supervisors’ perceptions of organizational capacity for change. No significant changes were observed in worker well-being or job satisfaction.

Conclusions

Training paired with supervisor coaching may strengthen workforce capacity for youth engagement, though attention to workforce well-being during implementation is warranted.

Keywords

training-and-coaching model youth engagement child welfare workforce multi-site evaluation workforce capacity-building

Authentic youth engagement in child welfare is a strengths-based practice that centers youth as experts of their own experiences and active partners in decisions about their lives, which has been shown to support their autonomy, resilience, and more effective permanency planning (Administration for Children and Families, 2019; Annie E. Casey Foundation, 2019; Williams-Butler et al., 2018). For youth in care, permanency extends beyond physical placement to include legal (Annie E. Casey Foundation, 2019; Winokur et al., 2018), relational (Fowler et al., 2025; Williams-Butler et al., 2018), and cultural aspects (Fowler et al., 2025; Julien-Chinn & Lietz, 2016), together supporting long-term well-being and positive identity formation (Bell & Romano, 2017; Child Welfare Information Gateway, 2019). Sustainable permanency across all three dimensions is best achieved through authentic youth partnerships. Child welfare practices that emphasize youth voices have been shown to be more likely to lead to permanency plans that better align with youth needs and realities, making them more durable and relevant (Annie E. Casey Foundation, 2019; Williams-Butler et al., 2018). Consequently, increased youth involvement in permanency planning is associated with more stable placements, stronger social networks, improved compliance with case plans, and greater perceived fairness in legal proceedings (Administration for Children and Families, 2019; Williams-Butler et al., 2018). Genuine engagement has been shown to foster critical developmental gains that can help prepare youth for adulthood and support successful transitions in employment, education, and independent living (Annie E. Casey Foundation, 2019; Williams-Butler et al., 2018).

While evidence supports these practices, individual- and system-level barriers affect the extent to which youth are engaged in planning processes. For example, child welfare workers sometimes restrict youth participation in aspects of case planning due to concerns about emotional impact, doubts about the decision-making abilities of younger children, or in high-stakes situations (Toros, 2021). Without proper support or preparation, youth with out-of-home care (OOHC) experiences have described unrealistic expectations that they would advocate for themselves (Simmel et al., 2025). There is an ongoing need for practices and policies that promote youth agency and address implicit biases (Administration for Children and Families, 2019; Annie E. Casey Foundation, 2019; Peters et al., 2024) as youth participation remains constrained by skill gaps, overprotective attitudes, and organizational barriers, such as heavy caseloads and limited time for meaningful conversations (Peters et al., 2024; Toros, 2021). Structural reforms, such as providing ongoing training for workers, have been shown to foster meaningful youth engagement (Kim et al., 2024; Toros, 2021). Organizational contexts are vital for facilitating this engagement. Research has shown that supportive cultures, policies, and training within child welfare organizations may foster environments where youth–adult partnerships can flourish (Annie E. Casey Foundation, 2019; Julien-Chinn & Lietz, 2016). However, a critical gap remains due to the lack of tangible, evidence-based approaches to effectively translate this evidence on youth engagement into system-wide child welfare practice (see Wollenhall et al. (2025) for a comprehensive review of youth engagement practices in child welfare settings). Building an evidence base for workforce development requires first establishing whether training and coaching shift worker perceptions—a necessary precursor to changes in practice behavior and, ultimately, youth outcomes.

Given this need, the current study examines the extent to which the Quality Improvement Center on Engaging Youth in Finding Permanency (QIC-EY) child welfare workforce training and coaching model promotes shifts in the workforce's perspectives on individual and organizational capacity to engage youth in case and permanency planning. Three primary outcomes were evaluated in the current study, including pre–post changes in perceptions of: child welfare workforce preparedness and capacity to engage youth in case and permanency planning; organizational philosophy and culture related to youth engagement; and professional well-being and job satisfaction. We provide background on each, followed by an overview of the QIC-EY initiative.

Workforce Preparedness and Capacity to Engage Youth in Case/Permanency Planning

Child welfare workers’ confidence and skill in engaging youth in permanency planning may influence how youth in care experience their child welfare services and long-term outcomes. Higher self-efficacy among child welfare workers has been associated with higher-quality permanency decisions, reduced placement disruptions, and foster stability (Julien-Chinn & Lietz, 2016; Winokur et al., 2018). Conversely, low worker confidence and resource constraints have been shown to limit youth involvement, especially for younger or more vulnerable children (Kim et al., 2024; Williams-Butler et al., 2018). However, enhanced communication training tailored to youths’ developmental stages has been shown to promote inclusive engagement across age groups, and confidence can be further bolstered through targeted supervision and the integration of trauma-informed practices (Toros, 2021). Collectively, the literature indicates the need for comprehensive workforce and organizational development efforts to enhance competencies in youth engagement and permanency planning within supportive organizational cultures (Julien-Chinn & Lietz, 2016; Kim et al., 2024; Toros, 2021).

Organizational Philosophy and Culture Related to Youth Engagement

Organizational culture, a set of learned beliefs that become norms, influences how an organization translates its mission into action (Osborne et al., 2022; Schein, 2010) and arises from the interplay among individuals’ beliefs, organizational practices, and external forces (Osborne et al., 2022). Research has shown that social service workers’ attitudes often align with those of organizational leadership (Reingold & Liu, 2009), and individual beliefs can impact organizational practices and policies (Al-Hawari et al., 2021; Brieger et al., 2021; Rahn et al., 2023). Employees’ values, beliefs, and attitudes (VAB) have been shown to influence the VAB of other staff, employee satisfaction, management practices, and the organizational agenda (Stokes et al., 2016), underscoring the multilevel influences on the adoption of quality improvement initiatives.

The Organizational Readiness Framework (Weiner, 2009) suggests two key factors of change readiness: employees’ level of change commitment and change efficacy. Change efficacy, workers’ beliefs in the organization's ability to implement change, is influenced by aspects of infrastructure, including guidance from supervisors and access to training and resources (Schulman, 2006). When change commitment and change efficacy align, they create a culture conducive to enduring, meaningful change.

In youth-serving organizations, dedication to youth engagement is reflected in funding streams, training opportunities, and roles designated for youth within programs (Butler, 2025; Carter, 2021; Conrad, 2023). Powers (2024) expanded the youth–adult partnership framework by introducing the youth–adult coproduction model. In this model, every conversation involving youth includes youth representation in the room, such as youth participation in departmental leadership meetings. In legal and court contexts, youth engagement advances through practices such as early rapport-building, transparency, and the inclusion of youth voices in hearings and permanency decisions (Annie E. Casey Foundation, 2019; Kim et al., 2024). These youth-centered strategies can improve youth well-being by supporting stable relationships, boosting self-esteem, and preparing youth for independent adult roles, while addressing structural inequalities (Toros, 2021; Williams-Butler et al., 2018).

Professional Well-Being and Job Satisfaction

Facilitating system-wide change in any organization can be challenging, particularly in child welfare systems; organizational challenges in child welfare agencies include high staff turnover, competing professional demands, and resource limitations (Nalani et al., 2021; Salazar et al., 2021). Staff turnover is especially disruptive to continuity in youth–adult relationships; each time a youth is assigned a new caseworker, the trust-building cycle restarts. Over time, ongoing disruptions make it increasingly difficult to engage youth with OOHC experience, who may already take longer to build trust in a youth–adult partnership (Lee et al., 2026; Nalani et al., 2021). Further, staff are expected to balance competing work demands while adhering to procedural timelines and working within resource limitations; pressure to perform amid less-than-ideal working conditions can lead staff to revert to adult-centric relationships (Elsen and Ord, 2021). Rates of burnout and secondary traumatic stress remain high among the child welfare workforce, who are regularly exposed to trauma experienced by children and families. In a systematic review of resilience and burnout among the child welfare workforce, McFadden et al. (2015) identified both individual and organizational factors influencing professional well-being. Individual factors identified in the review included child welfare workers’ own trauma histories—which can be a source of motivation for entering the field while placing workers at increased risk of burnout—and experiencing symptoms of secondary traumatic stress. However, adequate preparation for the job through training, education, and ongoing professional development has been linked to decreased burnout and increased retention. Organizational factors identified by the authors included commitment to their work and agency, adequate support from supervisors, peers, and the organization, and time and resource demands (McFadden et al., 2015). Consequently, there is a crucial need for additional strategies to support the child welfare workforce in their practice, for their own well-being, and for the well-being of the young people they serve.

The Quality Improvement Center on Engaging Youth in Finding Permanency

The QIC-EY is a national initiative funded by the Children's Bureau and organized by Spaulding for Children to improve how child welfare systems in the United States involve youth aged 12 and older in permanency planning and decision-making. Historically, the focus of the child welfare system has been to protect children from harm. While important, this singular goal has often led the system to overlook other dimensions of well-being, thereby affecting the quality and types of services offered, as well as the genuine involvement of youth in planning and decision-making. The QIC-EY aims to change this so that youth are treated as capable partners in decisions that impact their lives. The project centers on the idea that achieving legal, relational, and cultural permanency should involve and empower children from the earliest stages of the permanency process.

In pursuit of these goals, the QIC-EY partners conducted an environmental scan, including a scoping review of available evidence on youth engagement practices (Wollenhall et al., 2025), focus groups with individuals with lived experience, and a nationwide survey of over 100 young adults (Day et al., 2024; Peters et al., 2024; Quality Improvement Center on Engaging Youth in Finding Permanency, n.d.; Wollenhall et al., n.d.). The results of these efforts indicated that people recently involved in the child welfare system often had negative experiences related to finding permanency. Children reported feeling disempowered, not understanding their options or rights, and not having a voice in permanency-related discussions with caseworkers or in court proceedings, underscoring the need for transformational change within the child welfare system.

These findings informed the design of the core components of the QIC-EY that were implemented and evaluated in eight pilot state and tribal child welfare systems across the United States: a workforce training and coaching model to address identified concerns regarding youths’ lack of voice in permanency-related discussions and decisions; an asynchronous training program for court personnel focused on youth engagement in child welfare legal proceedings; site-specific youth program models; and site-specific capacity-building activities aimed at supporting system-level change. The integrated workforce training and coaching model was the first of these core components to be implemented and is the focus of the current paper.

Workforce Training and Coaching Model

Grounded in Empowerment theory (Zimmerman, 2000) and solution-focused case management (Blundo & Simon, 2015), the workforce training and coaching model was developed to guide case-carrying and supervisory child welfare staff in engaging youth to take an active and collaborative role in permanency planning and related decision-making. Caseworkers and supervisors in the QIC-EY groups participated in workforce training. Supervisors in the QIC-EY groups also participated in coaching training and facilitated coaching sessions with their staff.

Workforce Training. The original workforce training was delivered by Spaulding for Children (https://spaulding.org/), a private, non-profit child welfare agency based in Michigan. They used live in-person or virtual classroom model to deliver 12 1-h modules: (1) Art of Engaging Children and Youth; (2) Benefits of Child and Youth Engagement; (3) Characteristics and Competencies that are Important to Authentically Engage with Children and Youth; (4) Respecting, Responding, and Maintaining Connections to Cultural Strengths; (5) Permanency Planning and Engagement; (6) Inform and Prepare Children and Youth of Transitions; (7) Trauma's Impact on Child and Youth Engagement; (8) Child and Youth–Adult Partnership and Shared Decision Making; (9) Sexual Orientation, Gender Identity and Expression Affirming Engagement; (10) Having Difficult and Sensitive Conversations; (11) Interpersonal Communication Skills; and (12) Supporting Child and Youth Voice.

Coaching. The original coaching training, delivered by Families Rising (https://wearefamiliesrising.org/), an organization based in Minnesota that provides advocacy and education, combined sessions facilitated by a trainer to equip participating supervisors with tools, provide ongoing peer support for coaching, and support the use of a Transfer of Learning Guide. The 12-h, five-module training topics included: (1) Coaching 101; (2) Coaching to Engage Children and Youth in Permanency Conversations; (3) Coaching Skill Building Module: Trauma's Impact on Child and Youth Engagement; (4) Coaching for Finding Creative Solutions and Looking for the Yes; (5) Coaching to Engage Children and Youth in Difficult Conversations. After the coaching training, supervisors facilitated structured check-ins with caseworkers to reinforce youth engagement practices, provide feedback, advise caseworkers on empowering youth decision-making, and support the translation of learning into practice. Participating supervisors also attended at least three 90-min group sessions with a trainer to provide support for their coaching experiences.

Final Intervention Model. The final version of the training materials differs from the version evaluated, as the curriculum and delivery approaches were modified in response to pilot-site feedback and evaluation findings. The workforce training was consolidated to an 11-h training with 7 modules: (1) Using Engagement to Move Toward Permanency; (2) Having Difficult and Sensitive Conversations; (3) Informing and Preparing Children and Youth for Transitions; (4) Trauma's Impact on Engagement; (5) Supporting Cultural Connections; (6) Partnership and Shared Decision Making; and (7) Supporting Child and Youth Voice. The coaching component was substantially revised. In lieu of training, coaching guidebooks were developed to accompany six of the seven child welfare workforce training modules, helping coaches support skill development among frontline workers. Each guide includes (1) a summary of key workforce training concepts, (2) a coaching framework to guide them in supporting skill development, and (3) tools coaches can use to help workers engage with youth on their caseload. This final version, titled Transforming Engagement: Keeping Children at the Center of Our Work (hereinafter referred to as Transforming Engagement), is free of charge and publicly available at https://qic-ey.org/curriculum.

QIC-EY Evaluation and Current Study

The QIC-EY evaluation team assessed the model's effectiveness through a rigorous evaluation across pilot sites. A theory of change and logic model were developed, positing that participation in the training and coaching model would improve various outcomes related to youth engagement in finding permanency, relative to an equivalent comparison group that did not receive the training and coaching model. Based on prior literature and the environmental scan, expected outcomes included greater staff preparedness to engage children and youth in permanency and case planning, as well as a shift in organizational culture toward youth engagement. Additional outcomes assessed as part of the workforce evaluation are presented in a separate paper.

The current study evaluates the impact of the QIC-EY Transforming Engagement training and coaching model on workforce perceptions of individual and organizational capacity to engage children and youth in case/permanency planning, compared to “business as usual.” Using a quasi-experimental design with non-randomized but equivalent comparison groups across five state pilot sites in the United States, outcomes were measured using pre- and post-survey data to assess workers’ perceptions of (1) workforce preparedness and capacity to engage youth in case planning and permanency planning, (2) organizational philosophy and culture related to youth engagement, and (3) professional well-being and job satisfaction. Based on theory, lived expertise, and empirical literature, we developed the following hypotheses:

Intervention group participants will have increased perceptions from pre- to post-test of their capacity to engage youth in case/permanency planning. Specifically, we predict higher scores on the Workforce Confidence Scale and the Self-Rated Skills in Engaging Youth in Case/Permanency Planning Scale, and lower scores on the Barriers to Youth Involvement Scale relative to the comparison group.

Intervention group participants will show improvements in their perceptions of an organizational philosophy and culture conducive to youth engagement, including increased scores on the Case-Level YE Philosophy and Culture Scale, on the Youth Voice Agency Level Assessment (Y-VAL), on the Y-VAL Overall Vision and Commitment Subscale, and on the Texas Christian University Organizational Climate (TCU ORC-D4) Change Subscale relative to the comparison group.

Intervention group participants will report greater improvements in professional well-being and job satisfaction, including reductions in scores on the Professional Quality of Life (ProQOL-5) Burnout and Secondary Traumatic Stress Subscales, alongside increases in ProQOL-5 Compassion Satisfaction Subscale scores, TCU ORC-D4 Efficacy Subscale, Supervision for Frontline Staff Scale, and the Intent to Stay Scale relative to the comparison group.

Method

The University's Institutional Review Board determined this study to be exempt. All participants provided their consent electronically to participate in the evaluation.

Pilot Sites and Participant Recruitment

The QIC-EY partners employed a two-phase process in recruiting pilot sites. The first phase involved an assessment of site-level data for each potential site, including demographic characteristics such as the child welfare system's operational structure, permanency rates for children in foster care, and overall population density. These data were also used to help sites select comparable intervention and comparison regions. The second phase included site-level interviews assessing the alignment of the QIC-EY project goals with the site's broader objectives, the site's interest in implementing the required evaluation components, and its capacity to execute the project. At the end of this process, two tribal communities and six states were selected as QIC-EY pilot sites. The two tribal communities included the Yakama Nation, Nak-Nu-We-Sha, in the state of Washington, and the Southern Plains Consortium in Oklahoma, which includes the Caddo Nation, Cheyenne and Arapaho Indian Child Welfare, Comanche Nation Social Services, Delaware Nation, Ft Sill Apache, Kiowa Tribe, and Wichita Affiliated. The six participating states included Hawai’i, Nebraska, Michigan, Missouri, Montana, and Rhode Island. Hawai’i, the Southern Plains Consortium, and the Yakama Nation piloted a culturally relevant version of the workforce and coaching training model. The current paper evaluates the five states that piloted the standard version of the training: Michigan, Missouri, Montana, Nebraska, and Rhode Island.

For the current analysis, each of the five sites had an intervention area where the QIC-EY training and coaching model was implemented and a comparison area that did not receive the intervention. Pilot sites aimed to recruit at least 35 supervisors and 70 case-carrying staff per condition. Each trainer, participant, and evaluator was aware of the participants’ assigned condition in the study. Based on historical staff turnover rates, attrition across sites was assumed to be high. Ideally, additional participants would have been recruited at each site to account for this. However, capacity constraints stemming from existing site initiatives, short staffing, and the relatively small size of some agencies prevented such over-recruitment. Despite these limitations, each supervisor and case-carrying staff member in the intervention sites was required by their state to complete their assigned training; however, participation in the evaluation was optional. The comparison regions received no intervention. Site implementation teams and agency staff invited caseworkers and supervisors in the predetermined intervention and comparison regions to consider participating in the study.

Implementation and Data Collection

The workforce training and coaching model was implemented from January 2023 to September 2024. At each pilot site, training was delivered in person and synchronously online across multiple cohorts, depending on the needs and preferences of sites. Collection of baseline survey data began in January 2023 and continued through March 2024 as sites rolled out the training cohorts. The evaluation team used REDCap, a secure online data-collection system, for participant registration, consent, and survey administration. For the intervention groups, baseline surveys were initially completed during the allocated time on the morning of the training start date, when possible. By May 2023, however, we had moved to having participants complete the survey on a different day from the training due to feasibility issues. Site implementation teams and agency staff contacted workers in the intervention and comparison regions to support the completion of the baseline and follow-up surveys. The evaluation team also built in automatic reminders for the surveys to increase completion rates. The follow-up survey became available 180 days after the initial training start date at all sites, except in Rhode Island, where it became available 150 days after the training start date. The survey closed for all sites on September 9, 2024.

Implementation Fidelity

Data on the fidelity of implementation of the training and coaching models were collected to assess whether the models were being implemented as designed. At the end of each training session, facilitators reported the training duration and which activities from that module were successfully included that day. For workforce training, facilitators reported that the average duration of each module was 59 min, compared with the expected 60 min. The average percentage of activities completed by the facilitators for the workforce training was 96%. For the supervisor training, facilitators reported an average duration of each coaching module of 184 min per module, compared with the expected average of 144 min. The average percentage of activities completed by the facilitators for the coaching training was 95%. At each of the three 90-min group sessions with a facilitator, supervisors were asked whether they had delivered coaching to their staff. On average, supervisors reported delivering coaching 75% of the time since their last meeting with a facilitator.

Measures

Each of the three primary outcomes is listed below, along with the associated constructs and measures. Due to the many factors that needed to be assessed, alongside practical limitations on the acceptable length of the outcome survey, several validated measures had to be shortened from their original forms. Items less central to the goals of the current intervention were excluded. Several of the measures were developed by the researchers due to a lack of existing measures assessing the constructs of interest in this population. To increase the validity, feasibility, and acceptability of these newly developed measures, the survey underwent several iterations of refinement following review and feedback from child welfare professionals and administrators, as well as a team with lived experience in child welfare. All the following measures were administered via REDCap at baseline and follow-up in the intervention and comparison groups.

Outcome 1: Workforce Preparedness and Capacity to Engage Youth

Confidence to Support Youth in Case/Permanency Planning. The evaluation team developed the 4-item Workforce Confidence Scale to assess child welfare workers’ preparedness for, and confidence in, supporting youth in achieving permanency goals and plans. Participants were asked to rate their level of confidence in engaging youth in case planning and permanency planning that includes (a) legal permanency, (b) relational permanency, and (c) cultural permanency on a scale ranging from (0) = Not at All Confident to (10) = Very Confident, for example, “How confident do you feel that you can engage children and youth in their case planning?” “How confident do you feel that you can engage children and youth in permanency planning that includes legal permanency?” Cronbach's alpha calculated from baseline data in the current sample was somewhat low, but within an acceptable range (α = 0.69).

Skills in Engaging Youth in Case Planning and Permanency Planning. The Self-Rated Skills in Engaging Youth in Case/Permanency Planning Scales include two measures that assess child welfare workforce self-perceptions of skill in engaging youth in (a) case planning (7 items) and (b) permanency planning (4 items). Questions were developed by the evaluation team based on the National Association of Social Workers Standards for Social Work Practice in Child Welfare (2013), Standard 11: Youth Engagement, which states that: “Social workers in child welfare shall actively engage older youths in addressing their needs while in out-of-home care and as they prepare to transition out of foster care” (p. 22). Participants were asked to rate their level of skill in engaging youth in case planning and permanency planning on a scale ranging from (0) = I have no skill to (10) = I have a high level of skill., for example, “Supporting children and youth in developing decision-making skills, achieving goals, and celebrating successes;” “Understanding adolescent development and how it impacts children's and youth's perspectives on permanency.” Cronbach's alpha, calculated from baseline data in the current sample, indicated good reliability for the case planning scale (α = 0.94) and the permanency planning scale (α = 0.91).

Barriers to Youth Involvement in Case/Permanency Planning. The 15-item Barriers to Youth Involvement Scale was used to assess possible agency-level challenges and barriers to authentic youth engagement in case/permanency planning. The evaluators adapted the “Barriers to Youth Involvement” items from Youth Involvement in Systems of Care: A Guide to Empowerment (Matarese et al., 2005) and added an additional item to create this scale. Participants were instructed to rank how much each item has been a barrier to youth engagement on a scale ranging from (0) = Never a Challenge to (3) = Always a Challenge, for example, “Children/youth have ideas, but don’t always know how to implement them;” “There is distrust between children/youth and adults.” Cronbach's alpha calculated from baseline data from the current sample yielded an adequate reliability coefficient (α = 0.87).

Outcome 2: Organizational Philosophy and Culture Related to Youth Engagement

Personal and Agency-Wide Beliefs on Engaging Youth. The Case-Level YE Philosophy and Culture Scale was developed by QIC-EY partners at the University of Lincoln, Nebraska, to measure child welfare workforce beliefs about eliciting youth perspectives and youth–adult partnership. There were 16 items measuring workers’ personal beliefs and 16 measuring their beliefs about the work being done in their agencies overall. Participants were asked to rate their agreement with a series of statements on a scale ranging from (1) Strongly Disagree to (5) Strongly Agree, for example, “Child welfare professionals should make active efforts to elicit children and youth's perspectives and preferences around permanency;” “It is not developmentally appropriate to involve children and youth in permanency planning and decision-making.” Cronbach's alphas were calculated from baseline data from the current sample. They showed adequate reliability for the personal beliefs scale (α = 0.88) and good reliability for the agency-wide beliefs scale (α = 0.91).

Agency Commitment to Meaningful Youth–Adult Participation. The Y-VAL (Walker et al., 2018; Youth MOVE National, 2016) measures adult support for authentic youth engagement and participation at the agency level. Five of the six items from the Y-VAL Overall Vision and Commitment subscale were utilized to assess agency-level commitment to the meaningful involvement between youth and adults. Evaluators added an item to the subscale to meet the current study's evaluation needs. Each item describes what fully developed agency support for youth voice looks like in a specific area. Participants were asked to rate how their own agencies on a scale that ranged from (1) = Least developed to (5) = Fully developed, for example, “The agency has created a formal statement affirming its commitment to promoting child/youth engagement within the agency;” “The agency and its leaders clearly communicate this vision to stakeholders—including young people, management and staff.” Cronbach's alpha, calculated from baseline data in the current sample, demonstrated excellent reliability (α = 0.91).

Agency-Wide Culture of Youth Partnership in Permanency Planning. A 5-item subset of the Y-VAL Overall Vision and Commitment subscale, the Culture of Partnership items were used to evaluate the organizational-level partnership between the agency and youth regarding permanency planning. Each item describes what fully developed agency support for youth voice entails in a specific area. Participants were asked to compare their agencies with the descriptions, with answer choices ranging from (1) = Least Developed to (5) = Fully Developed, for example, “Leaders and staff throughout the agency genuinely view children/youth as having valuable experience and ideas that can contribute to improving case/permanency planning outcomes.” Cronbach's alpha calculated from baseline data from the current sample yielded an excellent reliability coefficient (α = 0.93).

Organizational Change Related to Youth Engagement. The TCU ORC-D4 Change Subscale (Institute of Behavioral Research, 2011), is a 6-item, validated measure (α = 0.76; Lehman et al., 2002) that was used to assess organizational change related to youth engagement. The researchers added one question to meet the evaluation needs of the current study. Participants were asked to rate their level of agreement to a series of statements, with answer choices ranging from (1) = Disagree Strongly to (5) = Agree Strongly, for example, “You frequently hear good ideas from other staff for improving child/youth engagement in permanency planning;” “The general attitude in your program is to accept new and changing technology in relation to child/youth engagement (ex., social media).”

Outcome 3: Professional Well-Being and Job Satisfaction

Professional Well-Being, Burnout, and Secondary Traumatic Stress. Three subscales from the ProQOL-5 (Stamm, 2009) were used: Compassion Satisfaction, Burnout, and Secondary Traumatic Stress. The language was slightly adapted to be specific to child welfare workers and working with children and youth. For each subscale, participants were asked to consider statements within the context of their current work situation over the past 30 days and rate each on a scale ranging from (1) = Never to (5) = Very Often, for example, “I get satisfaction from being able to help children and youth;” “I feel trapped in my job as a child welfare worker;” “I am preoccupied with more than one child/youth that I work with.” The Compassion Satisfaction subscale is an 11-item validated measure (α = 0.88; Stamm, 2010) of workers’ professional well-being and the satisfaction they derive from their work. Researchers added one item to assess supervisors’ satisfaction with their engagement with staff. The Burnout subscale is a validated measure (α = 0.75; Stamm, 2010) that was used to assess burnout among workers. The Secondary Traumatic Stress subscale is a validated measure (α = 0.81; Stamm, 2010) assessing secondary traumatic stress responses among workers.

Professional Efficacy. The Texas Christian University Organizational Readiness for Change (TCU ORC-D4), Efficacy Subscale of Staff Attributes Scale (Institute of Behavioral Research, 2011), is a 6-item validated measure (α = 0.68; Lehman et al., 2002) assessing professional efficacy. The language was slightly adapted to be specific to child welfare workers. Participants were asked to rate their level of agreement with a series of statements on a scale ranging from (1) = Disagree strongly to (5) = Agree strongly, for example, “You have the skills needed to conduct effective child welfare work;” “You are effective and confident in doing your job.” The evaluators added one additional question to assess supervisors’ perceived efficacy in supporting their staff in child and youth engagement.

Supervisor Support. The Supervision for Frontline Staff Scale (Butler Institute for Families, 2014) is an unpublished 10-item scale that assesses frontline staff's perceived supervisor support. The evaluators added two questions that are specific to supporting child and youth engagement practices. Participants were asked to rate their level of agreement with a series of statements on a scale ranging from (1) = Strongly Disagree to (4) = Strongly Agree, for example, “My immediate supervisor genuinely cares about me;” “My immediate supervisor supports me in difficult case situations.” Cronbach's alpha calculated from baseline data from the current sample indicated good reliability (α = 0.94).

Intent to Stay. A 4-item subset of the Intent to Stay Scale (Butler Institute for Families, 2014) was used to assess employees’ retention and their intention to remain with the same agency. Participants were asked to indicate the extent to which they agree with a series of statements on a scale ranging from (1) = Strongly Disagree to (4) = Strongly Agree, for example, “I plan to leave this agency as soon as possible,” “I am committed to staying at this agency.” Cronbach's alpha calculated from baseline data in the current sample indicated acceptable reliability (α = 0.72).

Analysis

Baseline equivalence between groups was confirmed using propensity score matching (PSM) as a diagnostic check. PSM was used to estimate the likelihood that any participant would be in the intervention group, based on several role-based and demographic characteristics (Rosenbaum & Rubin, 1983; Starks & Garrido, 2014; Stuart, 2010). This process helps confirm baseline equivalence between the comparison and intervention groups, accounting for underlying participant characteristics as potential confounders in data analyses. Participant characteristics used as propensity-score matching control variables included role (i.e., supervisor or case-carry staff), years of experience in the role, age, sex, education level, and aggregated race and ethnicity. For each of these characteristics, baseline equivalence was assessed using Variance Ratios and Absolute Standard Mean Differences (ASMDs). The Variance Ratio applies only to continuous variables, and baseline equivalence is indicated when the calculated value is near 1. The ASMD measure is an effect-size calculation; baseline equivalence is indicated if the calculated value is 0.25 or less (Wilson et al., 2024). The PSM technique was conducted in R. The ASMD for the full sample of participant data was measured for each characteristic, with values less than 0.13. Age, the only continuous variable, had a Variance Ratio of 0.9 in the full sample. These ASMD and Variance Ratio values indicate baseline equivalence across the full participant sample, with no need for reduction. Accordingly, the PSM method was employed only as a diagnostic check and not in the analysis, and the full sample of participants was used for all statistical models.

SPSS version 29.02 was used to conduct statistical comparisons for each outcome measure. Missing data were managed using regression imputation within SPSS's Missing Value Analysis tool; thus, the sample size for each model remained consistent. Between-group differences were assessed using analysis of covariance (ANCOVA) models. The difference in scale scores between baseline and follow-up was the dependent variable, and study condition (intervention or comparison group) was the independent variable. In each model, we controlled for baseline scale score, state, race, ethnicity, years of experience in the child welfare system, and the worker's age. To address the elevated risk of Type I error across the multiple ANCOVA models, we applied Šidák corrections to adjust for multiple comparisons within each outcome domain and report the adjusted p-values.

Results

Participant Demographics

Among workers participating in workforce training across all pilot sites, 957 consented to participate in the evaluation and complete the baseline survey. However, records were suspended for 143 baseline participants, including 83 intervention group participants and 60 comparison group participants, who no longer met eligibility criteria due to changes in role or leaving their jobs. The number of suspended intervention group participants is likely higher due to increased implementation team contact with this group, which facilitated awareness of these workers’ employment changes (but does not capture all cases where attrition was due to employment). Consequently, the baseline sample comprises 813 participants who were eligible to take the follow-up survey, including 109 intervention supervisors, 100 comparison supervisors, 370 intervention caseworkers, and 256 comparison caseworkers. Overall, 87% of eligible participants who took the baseline survey (N = 703) completed the follow-up survey, including 94% of intervention supervisors, 97% of comparison supervisors, 80% of intervention caseworkers, and 88% of comparison caseworkers. Among eligible participants who did not complete the follow-up survey, analysis of the intervention and comparison groups showed no statistically significant differences in demographic characteristics; however, among the caseworkers, there were differences in attrition between conditions by pilot site, X²(4) = 20.80.63, p < .001, with one site accounting for over 50% of the intervention group attrition. All intervention group participants included in the current sample completed the baseline survey and at least 80% of the training, while comparison group participants received no intervention.

Participant demographics are included in Table 1. Of the 703 participants, approximately 17% (n = 117) were from Michigan, 22% (n = 151) from Montana, 24% (n = 169) from Nebraska, 9% (n = 64) from Rhode Island, and 29% (n = 202) from Missouri. For supervisors, the only demographic that differed significantly between intervention and control groups was highest educational degree obtained, with the intervention group having higher education levels than the comparison group, X²(2) = 7.63, p = .02. For caseworkers, no demographics were statistically significantly different between conditions.

Table 1.

Participant Demographics, N = 703.

	Child Welfare Supervisors		Child Welfare Caseworkers
	N = 198		N = 505
	QIC-EY	Comparison	QIC-EY	Comparison
	n = 101	n = 97	n = 281	n = 224
	N (%)	N (%)	N (%)	N (%)
Gender ^a
Female	89 (88.1%)	90 (92.8%)	241 (85.8%)	194 (86.6%)
Male	8 (7.9%)	2 (2.1%)	34 (12.1%)	21 (9.4%)
Other or prefer not to answer	4 (4.0%)	5 (5.2%)	6 (2.2%)	9 (3.9%)
Ethnicity
Hispanic or Latino	2 (2%)	4 (4.1%)	18 (6.4%)	10 (4.5%)
Not Hispanic or Latino	91 (90.1%)	86 (88.7%)	252 (89.7%)	203 (90.6%)
Prefer not to answer	8 (7.9%)	7 (7.2%)	11 (3.9%)	11 (4.9%)
Race ^b
Am. Indian/Alaska Native	0 (0%)	1 (1%)	5 (1.8%)	6 (2.7%)
Black or Afr. Amer.	7 (6.9%)	4 (4.1%)	23 (8.2%)	24 (10.7%)
Asian or Asian Amer.	0 (0%)	0 (0%)	2 (0.7%)	1 (0.4%)
White	88 (87.1%)	86 (88.7%)	246 (87.5%)	181 (80.8%)
Other	0 (0%)	1 (1%)	2 (0.7%)	5 (2.2%)
Prefer not to answer	8 (7.9%)	6 (6.2%)	9 (3.2%)	12 (5.4%)
Field of Study of Highest Degree
Social Work	25 (24.8%)	25 (25.8%)	62 (22.1%)	40 (17.9%)
Psychology	19 (18.8%)	18 (18.6%)	56 (19.9%)	38 (17%)
Education	9 (8.9%)	8 (8.2%)	26 (9.3%)	12 (5.4%)
Law	2 (2.0%)	0 (0%)	2 (0.7%)	3 (1.3%)
Public Health or Admin.	3 (3.0%)	1 (1.0%)	6 (2.2%)	6 (2.6%)
Human Development	4 (4.0%)	5 (5.2%)	17 (6%)	10 (4.5%)
Criminal Justice	15 (14.9%)	21 (21.6%)	46 (16.4%)	38 (17%)
Sociology	6 (5.9%)	7 (7.2%)	12 (4.3%)	19 (8.5%)
Other or Missing	18 (17.8%)	12 (12.4%)	54 (19.2%)	58 (25.9%)
Highest Level of Education
Less than bachelor's degree	0 (0.0%)	0 (0.0%)	2 (0.7%)	2 (0.8%)
Bachelor's degree	61 (60.4%)	75 (77.3%)	242 (86.1%)	190 (84.8%)
Graduate degree	40 (39.6%)	22 (22.7%)	37 (13.2%)	31 (13.8%)
Missing	0 (0.0%)	0 (0.0%)	0 (0.0%)	1 (0.4%)
	Mean (SD)	Mean (SD)	Mean (SD)	Mean (SD)
Age	44.8 (8.4)	43.2 (8.9)	38.3 (10.3)	38.8 (11.5)
Years worked in child welfare field	13.5 (8.0)	13.3 (8.1)	5.6 (6.6)	6.0 (6.9)
Years worked in current agency	11.6 (6.7)	10.8 (8.0)	3.9 (5.0)	3.8 (5.1)
Years in current position	4.7 (4.5)	5.3 (5.7)	3.3 (3.9)	3.3 (4.1)

Due to US Federal Executive Order 14168 limiting reporting of gender identities to male and female in federally funded research reports, participants choosing a different gender identity, along with those who selected prefer not to answer, are included in the “other or prefer not to answer” category for this variable.

Participants were able to choose more than one race.

Descriptive and Bivariate Analyses

We conducted preliminary analyses, including descriptive statistics and bivariate correlations, to examine relationships between demographic control variables and baseline scale scores and to verify assumptions for subsequent ANCOVA models. Table 2 presents descriptive statistics for all study variables at baseline and follow-up by group. For continuous demographic variables, bivariate correlations were calculated; for categorical variables, independent-samples t-tests or ANOVAs with Bonferroni pairwise comparisons were used. Statistical comparisons of baseline scores and control variables are presented in Table 3.

Table 2.

Descriptive Statistics for Study Variables for the QIC-EY Intervention and Control Groups.

	Supervisors				Caseworkers
	QIC-EY		Control		QIC-EY		Control
	M(SD)		M(SD)		M(SD)		M(SD)
	Baseline	Follow-up	Baseline	Follow-up	Baseline	Follow-up	Baseline	Follow-up
Outcome 1: Workforce Preparedness and Capacity to Engage Youth
Workforce Confidence (0–10)	8.23 (1.43)	8.84 (1.25)	8.38 (1.29)	8.65 (1.39)	7.33 (1.93)	8.32 (1.71)	7.76 (1.83)	8.13 (1.76)
Skill—Case Planning (0–10)	7.79 (1.19)	8.13 (1.02)	7.91 (.98)	8.03 (.92)	7.39 (1.33)	8.06 (1.17)	7.81 (1.24)	7.92 (1.18)
Skill—Permanency Planning (0–10)	7.46 (1.35)	8.07 (.98)	7.78 (1.13)	7.94 (1.14)	6.77 (1.66)	7.81 (1.38)	7.30 (1.50)	7.57 (1.45)
Barriers to Youth Involvement (0–3)	1.38 (0.26)	1.27 (0.34)	1.44 (0.36)	1.39 (0.45)	1.44 (0.39)	1.32 (0.41)	1.32 (0.38)	1.28 (0.40)
Outcome 2: Organizational Philosophy and Culture Related to Youth Engagement
Case-Level YE Philosophy and Culture
Personal Beliefs (1–5)	4.29 (0.40)	4.28 (0.47)	4.22 (0.41)	4.27 (0.51)	4.16 (0.51)	4.18 (0.52)	4.09 (0.50)	4.10 (0.55)
Agency-Wide Beliefs (1–5)	4.14 (0.51)	4.10 (0.53)	4.06 (0.54)	4.11 (0.59)	3.96 (0.57)	4.04 (0.59)	3.96 (0.52)	3.99 (0.59)
Commitment to Youth–Adult Participation (1–5)	3.50 (0.81)	3.90 (0.73)	3.59 (0.90)	3.68 (0.82)	3.67 (0.72)	3.97 (0.74)	3.75 (0.87)	3.77 (0.82)
Culture of Partnership (1–5)	3.50 (0.82)	3.86 (0.74)	3.50 (0.82)	3.71 (0.78)	3.63 (0.76)	3.92 (0.77)	3.68 (0.86)	3.78 (0.77)
Organizational Change (1–5)	3.47 (0.54)	3.62 (0.52)	3.37 (0.51)	3.41 (0.52)	3.43 (0.50)	3.51 (0.49)	3.37 (0.52)	3.47 (0.56)
Outcome 3: Professional Well-Being and Job Satisfaction
Professional Satisfaction (1–5)	4.10 (0.52)	4.01 (0.55)	4.06 (0.55)	4.07 (0.54)	3.97 (0.63)	3.96 (0.66)	3.99 (0.64)	3.98 (0.72)
Burnout (1–5)	2.47 (0.52)	2.45 (0.54)	2.55 (0.50)	2.50 (0.57)	2.52 (0.57)	2.62 (0.58)	2.60 (0.60)	2.61 (0.61)
Secondary Traumatic Stress (1–5)	2.25 (0.57)	2.31 (0.68)	2.41 (0.65)	2.41 (0.72)	2.38 (0.73)	2.48 (0.79)	2.35 (0.70)	2.38 (0.75)
Professional Efficacy (1–5)	4.17 (0.37)	4.20 (0.38)	4.11 (0.40)	4.14 (0.40)	3.94 (0.46)	4.08 (0.48)	4.08 (0.51)	4.09 (0.57)
Supervisor Support (1–4)	3.26 (0.62)	3.32 (0.56)	3.40 (0.51)	3.42 (0.52)	3.33 (0.65)	3.42 (0.59)	3.56 (0.57)	3.57 (0.57)
Intent to Stay in Job (1–5)	3.23 (0.57)	3.22 (0.58)	3.19 (0.55)	3.07 (0.54)	3.06 (0.54)	2.96 (0.60)	2.98 (0.54)	2.98 (0.63)

Table 3.

Statistical Comparisons of Scale Scores and Demographic Control Variables—Full Baseline Sample, N = 703.

	State		Role		Education		Ethnicity		Race		Years in Child Welfare	Age
	F	η²	t	d	F	η²	F	η²	t	d	r	r
Outcome 1: Workforce Preparedness and Capacity to Engage Youth
Workforce Confidence	2.86*	0.01	6.16***	0.45	9.97**	0.02	1.98	0.01	−1.28	−0.17	0.25***	0.10**
Skill—Case Planning	2.93*	0.02	2.84**	0.22	6.28***	0.03	2.37	0.00	0.47	0.06	0.15***	0.06
Skill—Permanency Planning	1.60	0.01	5.32***	0.40	3.92**	0.03	0.60	0.00	−0.95	−0.12	0.22***	0.07
Barriers to Youth Involvement	2.16	0.01	1.32	0.10	1.73	0.01	0.42	0.00	−0.65	−0.09	0.03	0.00
Outcome 2: Organizational Philosophy and Culture Related to Youth Engagement
Case-Level YE Philosophy and Culture
Personal Beliefs	7.21***	0.04	3.00**	0.23	2.64*	0.02	0.40	0.00	−6.22***	−0.80	0.04	0.10**
Agency-Wide Beliefs	6.48***	0.04	1.72	0.14	1.93	0.01	1.55	0.00	−3.13**	−0.40	−0.00	0.06
Commitment to Youth–Adult Participation	0.25	0.00	−2.07*	−0.18	1.51	0.01	3.75*	0.01	0.37	0.05	−0.14***	−0.08*
Culture of Partnership	0.97	0.01	−2.10*	0.18	1.65	0.01	5.32**	0.02	−0.38	−0.05	−0.11**	−0.08*
Organizational Change	1.05	0.01	0.41	0.04	0.78	0.01	5.18**	0.02	5.18	0.02	−0.10**	−0.11**
Outcome 3: Professional Well-Being and Job Satisfaction
Professional Satisfaction	1.83	0.01	2.15*	0.17	2.23	0.02	9.18***	0.03	0.52	0.07	−0.07	0.02
Burnout	3.24*	0.02	−0.84	−0.07	1.11	0.01	3.07*	0.01	−0.32	−0.04	0.08*	−0.07
Secondary Traumatic Stress	2.60*	0.02	−0.70	−0.06	1.25	0.01	1.48	0.00	−1.07	−0.14	0.06	−0.08*
Professional Efficacy	1.44	0.01	3.50***	0.29	3.23**	0.02	1.89	0.01	0.42	0.05	0.16***	0.09*
Supervisor Support	0.66	0.00	−1.87	−0.16	3.70**	0.03	5.82**	0.02	0.06	0.01	−0.11**	−0.11**
Intent to Stay in Job	3.35*	0.02	5.29***	0.44	0.46	0.00	7.26***	0.02	−2.22*	−0.29	0.13***	0.19***

Note. η² effect size interpretation: small = 0.01, medium = 0.06, large = 0.14; Cohen's d: 0.2 = small, 0.5 = medium, and 0.8 = large; Pearson r: 0.1 = small, 0.3 = medium, and 0.5 = large.

*p < .05, **p < .01, ***p < .001.

Several demographic characteristics were significantly associated with baseline scores across outcome domains. State of employment and staff role were associated with baseline scores across workforce preparedness, organizational culture, and professional well-being outcomes. Education level was associated with workforce confidence, skills in engaging youth, and professional efficacy. Race and ethnicity were associated with several organizational philosophy and professional well-being outcomes. Years working in child welfare and worker age showed consistent associations across domains, with more experienced and older workers generally reporting higher confidence and skills but lower commitment to youth–adult participation and organizational change. No demographic variables were significantly associated with Barriers to Youth Involvement at baseline. Given these associations, state, role, education, race, ethnicity, years in child welfare, and age were retained as covariates in subsequent ANCOVA models, and all analyses were separated by supervisor and caseworker roles.

ANCOVAS

ANCOVA models were used to test the hypotheses corresponding to each of the following outcomes. A p-value less than .05 was considered statistically significant. Partial Eta Squared was used to measure effect size, using Cohen's (1988) interpretation: η² = 0.01 is a small effect, η² = 0.06 is a medium effect, and η² = 0.14 is a large effect. Table 4 presents ANCOVA model results across outcomes.

Table 4.

Statistical Comparisons of QIC-EY Intervention and Control Groups Pre–Post Changes.

	Supervisors (N = 198)					Caseworkers (N = 505)
	SS	df	F	p	η2	SS	df	F	p	η2
Outcome 1: Workforce Preparedness and Capacity to Engage Youth
Workforce Confidence	1.28	1	0.91	.340	0.005	11.66	1	4.82	.028*	0.010
Skill—Case Planning	0.34	1	0.45	.503	0.002	9.03	1	8.89	.003**	0.018
Skill—Permanency Planning	2.88	1	3.36	.068^	0.018	14.74	1	9.24	.002**	0.019
Barriers to Youth Involvement	0.37	1	2.92	.089^	0.017	0.03	1	0.19	.660	0.000
Outcome 2: Organizational Philosophy and Culture Related to Youth Engagement
Case-Level YE Philosophy and Culture
Personal Beliefs	0.16	1	0.97	.0326	0.006	0.13	1	.62	.431	0.001
Agency-Wide Beliefs	0.25	1	1.21	.272	0.007	0.26	1	1.02	.314	0.002
Commitment to Youth–Adult Participation	3.10	1	6.62	.011*	0.038	5.40	1	11.73	<.001***	0.026
Culture of Partnership	0.76	1	1.79	.183	0.010	2.83	1	6.30	.012*	0.014
Organizational Change	1.60	1	7.61	.006**	0.040	0.003	1	0.01	.913	0.000
Outcome 3: Professional Well-Being and Job Satisfaction
Professional Satisfaction	0.20	1	1.01	.317	0.006	0.06	1	.193	.661	0.000
Burnout	0.02	1	0.12	.735	0.001	0.48	1	2.17	.141	0.004
Secondary Traumatic Stress	0.03	1	0.13	.724	0.001	0.93	1	2.81	.094^	0.006
Professional Efficacy	0.16	1	1.26	.262	0.007	0.27	1	1.35	.246	0.003
Supervisor Support	0.17	1	.098	.32	0.006	0.13	1	0.501	.479	0.001
Intent to Stay in Job	0.66	1	3.38	.068^	0.018	0.78	1	3.33	.069^	0.007

Note. η² effect size interpretation: small = 0.01, medium = 0.06, large = 0.14.

^p < .10 (approaching statistical significance), *p < .05, **p < .01, ***p < .001.

Outcome 1: Workforce Preparedness and Capacity to Engage Youth

Hypothesis 1: Intervention group participants will have increased perceptions from pre- to post-test of their capacity to engage youth in case/permanency planning. Specifically, we predict higher scores on the Workforce Confidence Scale and the Self-Rated Skills in Engaging Youth in Case/Permanency Planning Scale, and lower scores on the Barriers to Youth Involvement Scale relative to the comparison group.

Supervisors. There were no statistically significant differences between supervisor groups in workforce confidence, skill at engaging youth in case or permanency planning, or in barriers to youth involvement relative to the comparison group supervisors. For the supervisor group, our hypothesis was not supported.

Caseworkers. The caseworkers in the QIC-EY group showed a statistically significant improvement in workforce confidence relative to the comparison group, F(1, 487) = 4.82, p = .028, η² = .010, adjusted mean difference (AMD) = 0.31, 95% CI [0.03, 0.59]. The QIC-EY group also showed a statistically significant improvement in their skill at engaging youth in case planning, F(1, 487) = 8.89, p = .003, η² = .018, AMD = 0.28, 95% CI [0.09, 0.46], and in their skill at engaging youth in permanency planning, F(1, 487) = 9.24, p = .002, η² = .019, AMD = 0.35, 95% CI [0.13, 0.58]; the magnitude of these effects was small. There were no statistically significant differences between groups in their perceived barriers to youth involvement. Among the caseworker group, the hypothesis was partially supported with small, but statistically significant improvements in three of the four sub-outcomes.

Outcome 2: Organizational Philosophy and Culture Related to Youth Engagement

Hypothesis 2: Intervention group participants will show improvements in their perceptions of an organizational philosophy and culture conducive to youth engagement, including increased scores on the Case-Level YE Philosophy and Culture Scale, on the Y-VAL, on the Y-VAL Overall Vision and Commitment Subscale, and on the TCU ORC-D4 Change Subscale relative to the comparison group.

Supervisors. The supervisors in the QIC-EY group showed a statistically significant improvement relative to the comparison group supervisors in their commitment to meaningful youth–adult participation, F(1, 181) = 6.62, p = .011, η² = .038, AMD = 0.23, 95% CI [0.03, 0.43], and in their capacity for organizational change related to youth engagement, F(1, 181) = 7.61, p = .006, η² = .040, AMD = 0.19, 95% CI [0.05, 0.32]; the magnitude of these effects was small. There were no statistically significant differences between groups in personal beliefs or agency-wide beliefs related to youth engagement or in changes in an agency-wide culture of youth–adult partnership. The hypothesis was partially supported for the supervisor group, with statistically significant improvements in two of the six sub-outcomes.

Caseworkers. The caseworkers in the QIC-EY group showed a statistically significant improvement relative to the comparison group in their commitment to meaningful youth–adult participation, F(1, 487) = 11.73, p < .001, η² = .026, AMD = 0.23, 95% CI [0.11, 0.35], and in a culture of partnership, F(1, 487) = 6.30, p = .012, η² = .014, AMD = 0.16, 95% CI [0.04, 0.28]; the magnitude of these effects was small. There were no statistically significant differences between groups in personal or agency-wide beliefs regarding youth engagement or organizational change. Among the caseworker group, the hypothesis was partially supported with small, but statistically significant improvements in two of the six sub-outcomes.

Outcome 3: Professional Well-Being and Job Satisfaction

Hypothesis 3: Intervention group participants will report greater improvements in professional well-being and job satisfaction, including reductions in scores on the ProQOL-5 Burnout and Secondary Traumatic Stress Subscales, alongside increases in ProQOL-5 Compassion Satisfaction Subscale scores, TCU ORC-D4 Efficacy Subscale, Supervision for Frontline Staff Scale, and the Intent to Stay Scale relative to the comparison group.

Supervisors. Among supervisors, there were no statistically significant differences between groups in professional satisfaction, burnout, secondary traumatic stress, intention to stay in their current jobs, or professional efficacy. For the supervisor group, our hypothesis was not supported.

Caseworkers. Among caseworker groups, there were no statistically significant differences in secondary traumatic stress, professional satisfaction, burnout, workers’ intent to stay in their jobs, or professional efficacy. Among the caseworker group, the hypothesis was not supported.

Discussion and Applications to Practice

We conducted a multi-site quasi-experimental evaluation of the QIC-EY Transforming Engagement training and coaching model. The current study investigated the influence of this model on child welfare workforce perceptions of individual and organizational capacity to engage children and youth in case and permanency planning, relative to a comparison group. While the magnitude of effects across outcomes was small, the findings provide preliminary evidence that structured workforce training, paired with supervisor coaching, can strengthen child welfare caseworkers’ perceived preparedness to engage youth in case and permanency planning and to support organizational change related to youth engagement. However, some findings were mixed, including the lack of meaningful change in workforce well-being and job satisfaction.

For the first outcome—workforce preparedness and capacity to engage youth in case and permanency planning—the intervention's most beneficial effects were among caseworkers: those who participated in the QIC-EY Transforming Engagement training and coaching model showed statistically significant improvements in their confidence and skills. This provides promising evidence of the training's effects on caseworkers’ perceptions of their capacity to meaningfully engage youth. Because workers’ confidence and perceived skill have been linked to permanency outcomes among youth in care (Julien-Chinn & Lietz, 2016; Kim et al., 2024; Williams-Butler et al., 2018; Winokur et al., 2018), future research should explore how these positive shifts in perceptions may influence practice behaviors and youth outcomes. While supervisors did not show any statistically significant changes, this first outcome relates to experiences in direct practice with children and youth, so the lack of impact on supervisors’ perceptions in this area is understandable.

The second outcome—organizational philosophy and culture related to youth engagement—showed improvements in constructs related to change commitment and change efficacy among the intervention groups, both of which, according to the Organizational Readiness Framework, are necessary conditions for supporting enduring, meaningful change (Weiner, 2009). Regarding change commitment, both the supervisor and caseworker intervention groups showed statistically significant improvements in their commitment to meaningful youth–adult participation. Supporting shifts in change efficacy, caseworkers in the intervention group showed statistically significant improvements in an agency-wide culture supportive of youth–adult partnership, while supervisors in the intervention group showed statistically significant improvements in organizational capacity to implement changes in youth engagement practices. Collectively, findings for the first two outcomes align with and extend prior work, indicating that ongoing worker training, supportive organizational cultures, and policies can foster meaningful youth–adult partnerships (Annie E. Casey Foundation, 2019; Julien-Chinn & Lietz, 2016; Kim et al., 2024; Toros, 2021).

Contrary to our expectations, there were no statistically significant changes in the third outcome—professional well-being and job satisfaction. Although prior literature indicates better child welfare workforce retention and reduced burnout when supervisors and organizations provide adequate support, training, and greater workforce preparedness for their roles (McFadden et al., 2015), the intervention group did not show improvements in job satisfaction. It is possible that participation in the program model increased workers’ expectations without reducing their workload in systems already facing high turnover, competing demands, and pressure. Other factors beyond those controlled for in our models could influence these findings. It is important to consider that any positive impacts of new initiatives may take time to be realized. While the long-term impacts on professional well-being were beyond the scope of this study, the lack of change in this outcome suggests a need for added organizational support when rolling out child welfare workforce development initiatives.

These findings provide insights into strategies that may support meaningful change in child welfare practice and policy. A key practice implication is the use of structured supervisory coaching to reinforce transfer of learning. The intervention's design illustrates a feasible pathway for translating training into day-to-day practice through supervisor reinforcement and guided application tools. Because the observed improvements concentrated in practice-proximal outcomes among caseworkers, agencies implementing similar models should emphasize consistency in coaching routines (e.g. regular supervisor-caseworker check-ins focused on youth engagement strategies). It is also important to distinguish between the types of support required for supervisors and caseworkers. In practice, agencies may consider tailoring supervisor-facing components toward (a) coaching skill, (b) implementation leadership, and (c) creating conditions for sustained change, consistent with the observed supervisor improvement in organizational change capacity.

At the organizational level, these evaluation findings offer preliminary evidence supporting child welfare agency investment in training-plus-coaching as a workforce strategy for youth engagement. Because the strongest impacts were observed among caseworkers with direct responsibility for engaging youth in practice, workforce development policies should prioritize training models that explicitly build youth-engagement competencies and reinforce them through supervision/coaching structures. Similarly, pairing practice expectations with organizational infrastructure for implementation is essential. Beyond individual skill gains, the intervention was associated with statistically significant improvements in perceptions of agency commitment to meaningful youth–adult participation and improvements in organizational culture constructs. These findings provided some evidence supporting state and agency policies that not only articulate expectations for youth engagement but also invest in the organizational conditions that support implementation (e.g. coaching structures, shared vision and commitment), because organizational culture and implementation capacity are measurable and can shift within the study period. Finally, the Supporting America's Children and Families Act (P.L. 118-258), signed Jan. 4, 2025, which reauthorizes Title IV-B programs through 2029, emphasizes the need for states, tribes, and territories to improve workforce retention and recruitment via increased attention to workforce supports. Additionally, this law requires states to implement practices that promote high-quality caseworker visits. While our work does not evaluate the effects of this legislation, we anticipate that workers trained in authentic engagement would deliver higher-quality visits, leading to improved retention in the child welfare workforce. Future research is needed to assess the implications of the QIC-EY on these policy requirements. In sum, evaluation findings provide promising evidence to support policy and administrative decisions to fund, require, and sustain youth-engagement capacity-building as an ongoing practice change, especially when the model includes both didactic learning and structured coaching supports.

There are also several notable limitations to this study. This evaluation focused on efforts in five states; findings may differ in other contexts. While the non-randomized, quasi-experimental design of this study reflects a feasible, ethical approach to implementation and evaluation in real-world child welfare settings, it also has implications for the generalizability of the findings. First, given the real-world context of this evaluation, contamination between intervention and comparison regions is possible. Next, the implementation fidelity data collected and reported related to time and activity completion of the trainings, along with data on supervisors’ delivery of coaching, generally indicated high levels of adherence to the models as designed. However, these data were not collected in a way that allowed them to be tied to outcome measurements for specific supervisors and caseworkers. In addition, the dosage data collected on the frequency and quality of coaching delivered by supervisors to caseworkers were too generic to fully understand the model's coaching component in detail. Furthermore, while this study offers important insights into shifts in child welfare workers’ perceptions of youth engagement practices following program participation, the findings should be interpreted keeping in mind that all assessments in this study were self-report data from the perspectives of the participants rather than external assessments of behavior or practice change, and thus could be impacted by inaccurate self-assessment or social desirability. The study timeline also did not allow for more intensive psychometric assessment of newly created and/or adapted survey instruments beyond assessments of internal consistency. Future evaluation should consider additional observable outcome measures and perspectives of young people whom caseworkers aim to better engage. Finally, an extensive follow-up period was not feasible to assess whether the gains resulting from the intervention were maintained or whether the observed strain on the workforce declined over the long term.

Conclusion

Although substantial evidence indicates improved outcomes associated with centering authentic youth engagement throughout case and permanency planning processes (Administration for Children and Families, 2019; Annie E. Casey Foundation, 2019; Williams-Butler et al., 2018), practical strategies are needed to systematically embed these values in practice and policy. The QIC-EY Transforming Engagement training and coaching model begins to fill this resource gap by offering an adaptable set of tools to support multilevel change across diverse child welfare settings. This study provides promising preliminary evidence of a shift in workers’ perspectives on their individual and organizational capacity to engage young people as active participants in their own lives, a critical step toward improving the lives of young people in care.

Footnotes

ORCID iDs

Jenna N. Thompson

Amy M. Salazar

John Fowler

Lexi West

Angelique Day

Ethics Considerations

The University of Washington's IRB reviewed this study and determined it to be exempt from full review.

Consent to Participate

All study participants provided electronic consent prior to participation.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by the U.S. Department of Health and Human Services (HHS), Administration for Children and Families (ACF), Children's Bureau. Quality Improvement Center: Engaging Youth in Finding Permanency. HHS-2021-ACF-ACYF-CO-1911. The contents are those of the authors and do not necessarily represent the official views of, nor an endorsement by, ACF/HHS or the U.S. government.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Data Availability

The data that has been used is currently confidential.

References

Administration for Children and Families. (2019). Engaging, empowering, and utilizing family and youth voice in all aspects of child welfare to drive case planning and system improvement (ACYF-CB-IM-19-03). U.S. Department of Health and Human Services. https://acf.gov/sites/default/files/documents/cb/im1903.pdf

Al-Hawari

M. A.

Quratulain

Melhem

S. B.

(2021). How and when frontline employees’ environmental values influence their green creativity? Examining the role of perceived work meaningfulness and green HRM practices. Journal of Cleaner Production, 310, 127598. https://doi.org/10.1016/j.jclepro.2021.127598

Annie E. Casey Foundation. (2019). A framework for effectively partnering with young people. https://www.aecf.org/resources/a-framework-for-effectively-partnering-with-young-people

Bell

Romano

(2017). Permanency and safety among children in foster family and kinship care: A scoping review. Trauma, Violence, & Abuse, 18(3), 268–286. https://doi.org/10.1177/1524838015611673

Blundo

R. G.

Simon

J. K.

(2015). Solution-focused case management. Springer Publishing.

Brieger

S. A.

De Clercq

Meynhardt

(2021). Doing good, feeling good? Entrepreneurs’ social value creation beliefs and work-related well-being. Journal of Business Ethics, 172, 707–725. https://doi.org/10.1007/s10551-020-04512-6

Butler

K. D.

(2025). Engaging youth in philanthropic practice: Field-building strategies and the promise of youth-possible philanthropy™. The Foundation Review, 17(1), 9. https://doi.org/10.9707/1944-5660.1725

Butler Institute for Families. (2014). Supervision for frontline staff [Unpublished measure]. University of Denver.

Carter

A. J.

(2021). Foster care nonprofit leaders’ perspectives on funding sources and partnerships: Qualitative study [Doctoral dissertation. Capella University]. ProQuest.

10.

Child Welfare Information Gateway. (2019). Belonging matters—Helping youth explore permanency. U.S. Department of Health and Human Services, Administration for Children and Families, Children’s Bureau. https://www.childwelfare.gov/resources/belonging-matters-helping-youthexplore-permanency/

11.

Cohen

. (1988). Statistical power analysis for the behavioral sciences. Lawrence Erlbaum Associates.

12.

Conrad

(2023). Engagement, authenticity, and advocacy in “youth uncensored”: Ethics in applied theater research with street-involved youth. Qualitative Inquiry, 29(2), 365–373. https://doi.org/10.1177/10778004221099567

13.

Day

Abrahamson-Richards

Grinnell Davis

Roundtree-Swain

(2024). Youth engagement in permanency planning within tribal child welfare systems: A qualitative study of current practice. Journal of Public Child Welfare, 20(1), 205–235. https://doi.org/10.1080/15548732.2024.2434968

14.

Elsen

Ord

(2021). The role of adults in “youth-led” climate groups: Enabling empowerment. Frontiers in Political Science, 3, 641154. https://doi.org/10.3389/fpos.2021.641154

15.

Fowler

Day

Zachry

(2025). Conceptualizations of permanency in an online foster care community. Families in Society, 106(1), 25–44. https://doi.org/10.1177/10443894231209799

16.

Institute of Behavioral Research. (2011). Texas Christian University organizational readiness for change (TCU ORC-D4). Texas Christian University. https://ibr.tcu.edu/wp-content/uploads/2014/01/ORC-D4-sg-Rev-v11.pdf

17.

Julien-Chinn

F. J.

Lietz

C. A.

(2016). Permanency-focused supervision and workers’ self-efficacy: Exploring the link. Social Work, 61(1), 37–44. https://doi.org/10.1093/sw/swv043

18.

Kim

Pierce

Park

T. K.

(2024). Secondary traumatic stress and public child welfare workers’ intention to remain employed in child welfare: The interaction effect of job functions. Human Service Organizations: Management, Leadership & Governance, 48(2), 136–149. https://doi.org/10.1080/23303131.2023.2263518

19.

Lee

S. R.

Kim

Hong

Kovarie

G. M.

Patel

Kazaki

(2026). Turnover intentions among child welfare caseworkers: A qualitative analysis of pre-and during-COVID-19 experiences. Child Welfare, 103(5), 35–62.

20.

Lehman

W. E.

Greener

J. M.

Simpson

D. D.

(2002). Assessing organizational readiness for change. Journal of Substance Abuse Treatment, 22(4), 197–209. https://doi.org/10.1016/S0740-5472(02)00233-7

21.

Matarese

McyGinnis

Mora

(2005). Youth involvement in systems of care: A guide to empowerment . Technical Assistance Partnership for Child and Family Mental Health. https://nwi.pdx.edu/NWI-book/Chapters/App-6e.3-Youth-Involvement-In-Systems-Of-Care.pdf

22.

McFadden

Campbell

Taylor

(2015). Resilience and burnout in child protection social work: Individual and organizational themes from a systematic literature review. The British Journal of Social Work, 45(5), 1546–1563. https://doi.org/10.1093/bjsw/bct210

23.

Nalani

Yoshikawa

Godfrey

E. B.

(2021). Theorizing organizational learning to enhance youth–adult partnerships in community‐based youth serving organizations. American Journal of Community Psychology, 68(3–4), 513–530. https://doi.org/10.1002/ajcp.v68.3-4

24.

National Association of Social Workers. (2013). NASW Standards for Social Work Practice in Child Welfare. National Association of Social Workers. https://www.socialworkers.org/Practice/NASW-Practice-Standards-Guidelines/NASW-Standards-for-Social-Work-Practice-in-Child-Welfare

25.

Osborne

S. P.

Powell

Cui

Strokosch

(2022). Value creation in the public service ecosystem: An integrative framework. Public Administration Review, 82(4), 634–645. https://doi.org/10.1111/puar.13474

26.

Peters

Vesneski

Day

(2024). Youth engagement in courtroom dependency proceedings: Findings from interviews with national legal experts. Child Welfare, 102(4), 29–52. https://doi.org/10.2307/4880987

27.

Powers

(2024). Moving beyond a youth-adult partnership and youth advisory boards: Maine’s innovative model of foster youth empowerment and collaboration. SSRN. https://dx. https://doi.org/10.2139/ssrn.4687933

28.

Quality Improvement Center on Engaging Youth in Finding Permanency. (n.d.). State survey analysis . United States Children’s Bureau. https://qic-ey.org/wp-content/uploads/2022/08/QIC-EY-StateSurvey-08262022.pdf

29.

Rahn

O. G.

Soutar

G. N.

Lee

J. A.

(2023). Perceived values-congruence and employees’ change beliefs. Journal of Management & Organization, 29(6), 991–1009. https://doi.org/10.1017/jmo.2020.4

30.

Reingold

D. A.

Liu

H. K.

(2009). Do poverty attitudes of social service agency directors influence organizational behavior? Nonprofit and Voluntary Sector Quarterly, 38(2), 307–332. https://doi.org/10.1177/0899764008316967

31.

Rosenbaum

P. R.

Rubin

D. B.

(1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 70(1), 41–55. https://doi.org/10.1093/biomet/70.1.41

32.

Salazar

A. M.

Spiers

S. S.

Pfister

F. R.

(2021). Authentically engaging youth with foster care experience: Definitions and recommended strategies from youth and staff. Journal of Youth Studies, 24(8), 1015–1032. https://doi.org/10.1080/13676261.2020.1792863

33.

Schein

E. H.

(2010). Organizational culture and leadership (Vol. 2). John Wiley & Sons.

34.

Schulman

(2006). Terms of engagement: Aligning youth, adults, and organizations toward social change. Journal of Public Health Management and Practice, 12, S26–S31. https://doi.org/10.1097/00124784-200611001-00007

35.

Simmel

Bowden

C. F.

Todd

S. N.

Thorp

Hyde

Crystal

(2025). Youth in foster care: Highlighting the importance of youth engagement in mental health treatment decisions. Children and Youth Services Review, 169, 108094. https://doi.org/10.1016/j.childyouth.2024.108094

36.

Stamm

B. H.

(2009). Professional quality of life: Compassion, satisfaction, and fatigue version 5 (ProQOL). Center for Victims of Torture. https://proqol.org/ProQol_Test.html

37.

Stamm

B. H.

(2010). The concise manual for the professional quality of life scale. http://proqol.org/

38.

Starks

Garrido

M. M.

(2014, October). Observational & quasi-experimental research methods [PowerPoint slides]. 8th Annual Kathleen Foley Palliative Care Retreat Methods Workshop, Park City, UT. http://npcrc.org/files/NPCRC.Observational-PropensityScoreMethodsWkshop.10-20-14.pdf

39.

Stokes

Baker

Lichy

(2016). The role of embedded individual values, belief and attitudes and spiritual capital in shaping everyday postsecular organizational culture. European Management Review, 13(1), 37–51. https://doi.org/10.1111/emre.12065

40.

Stuart

(2010). Matching methods for causal inference: A review and a look forward. Statistical Science, 25(1), 1–21. https://doi.org/10.1214/09-STS313

41.

Toros

(2021). Children’s participation in decision making from child welfare workers’ perspectives: A systematic review. Research on Social Work Practice, 31(4), 367–374. https://doi.org/10.1177/1049731520984844

42.

Walker

J. S.

Masselli

B. M.

Blakeslee

Baird

Thorp

(2018). Development and testing of an assessment of youth/young adult voice in agency-level advising and decision making. Children and Youth Services Review, 94, 598–605. https://doi.org/10.1016/j.childyouth.2018.08.040

43.

Weiner

B. J.

(2009). A theory of organizational readiness for change. Implementation Science, 4(1), 112. https://doi.org/10.1186/1748-5908-4-67

44.

Williams-Butler

Ryan

J. P.

McLoyd

V. C.

Schulenberg

J. E.

Davis-Kean

P. E.

(2018). Relational permanence and psychological well-being among African American adolescents in foster care. Journal of Child and Family Studies, 27(10), 3277–3287. https://doi.org/10.1007/s10826-018-1155-8

45.

Wilson

S. J.

Brown

S. R.

Kerns

S. E. U.

Dastrup

S. D.

Hedberg

Schachtner

Jackson

Norvell

Campbell

Wall

(2024). Title IV-E prevention services clearinghouse handbook of standards and procedures, Version 2.0. Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.

46.

Winokur

M. A.

Holtan

Batchelder

K. E.

(2018). Systematic review of kinship care effects on safety, permanency, and well-being outcomes. Research on Social Work Practice, 28(1), 19–32. https://doi.org/10.1177/1049731515620843

47.

Wollenhall

Feltner

Vanderwill

Fowler

Day

(2025). Scoping review of worker characteristics and competencies that support youth engagement in child welfare case planning. Families in Society, 107(2). https://doi.org/10.1177/10443894251321346

48.

Wollenhall

Vanderwill

Peters

Day

(n.d.) Barriers to Authentic Youth Engagement in Permanency Planning. DC: US Children’s Bureau. https://qic-ey.org/wp-content/uploads/2025/06/QIC-EY-Barriers-Report-06022025.pdf

49.

Youth MOVE National & Regional Research Institute. (2016). Youth/young adult voice at the agency level: A self-assessment (YVAL Self-Assessment). Youth MOVE National, Portland State University. https://youthmovenational.org/resources/

50.

Zimmerman

M. A.

(2000). Empowerment theory: Psychological, organizational and community levels of analysis. In Handbook of community psychology (pp. 43–63). Springer.