Abstract
This article examines organizational-level outcomes achieved during a technical assistance (TA) initiative designed to increase the capacity of local health departments (LHDs) to prevent youth violence (YV) via a multisectoral approach. This effort was designed to address the knowledge gap regarding how to provide effective TA to LHDs, specifically in YV. Twelve communities with high rates of YV were selected to participate using a multistage process. TA provided to LHD representatives (and other community partners) included monthly calls with TA specialists, group online learning events, community-of-practice calls, and access to an online portal offering additional resources. Data sources were used to determine the extent to which communities achieved the initiative’s intended outcomes, including increased infrastructure to support youth violence prevention (YVP) at LHDs and the creation of community-wide violence prevention plans. Data sources included an online TA tracking system, annual semistructured group interviews, and point-of-contact surveys. While results indicated variation in TA uptake across sites, several target outcomes were achieved including increased representation and engagement of diverse perspectives in local YVP efforts and strengthened infrastructure and integration of YVP at LHDs. Findings highlight the importance of supporting LHDs to align YVP work with other priorities and funded activities, building a larger role for leadership in providing organizational support for YV, supporting the development of multisector coalitions or partnerships to decrease silos among different sectors focused on similar populations or communities. Implications from this initiative suggest that LHDs can be supported to convene local, multisector YVP efforts, which can be sustained if local YVP infrastructure is enhanced.
Keywords
Young people continue to be affected by violence in their homes, neighborhoods, and schools, even though violence was declared a public health concern 39 years ago (Dahlberg & Mercy, 2009). In 2018, there were an estimated 404,691 nonfatal violence-related assault injuries for youth ages 10 to 24 (Centers for Disease Control and Prevention [CDC] & National Center for Injury Prevention and Control, 2019b), and homicide was the third leading cause of death for all youth aged 10 to 24 years (CDC & National Center for Injury Prevention and Control, 2019a). Youth violence is the intentional use of physical force or power to threaten or harm others by young people aged 10 to 24 years, which includes behaviors such as homicide, nonfatal violence-related assault injuries, gang violence, and bullying (David-Ferdon et al., 2016).
In 1993, the CDC recommended using a public health approach to mobilize communities to prevent youth violence (YV), as described in The Prevention of Youth Violence: A Framework for Community Action (CDC & National Center for Injury Prevention and Control, 1993). This four-step approach applies scientific and programmatic expertise to preventing violence, including (1) collecting data and coordinating multiple data systems to understand and monitor the problem of violence and to identify the populations and communities at greatest risk, (2) identifying risk and protective factors that influence the likelihood of violence, (3) developing and testing violence prevention strategies, and (4) widely implementing violence prevention strategies that are based on the best available evidence. The World Health Organization has also recommended using a public health approach to address violence, under the leadership of local health departments (LHDs; Krug et al., 2002). LHDs are uniquely suited to lead or colead efforts to foster community implementation of the public health approach, given their traditional role as convening multiple partners and sectors within a community. In 2011, a CDC commissioned report developed by the National Association of County and City Health Officials (NACCHO) identified nine roles for LHDs to play in preventing YV: (1) building coalitions and partnerships; (2) conducting needs assessments and strategic planning; (3) identifying and supporting effective policy approaches to violence prevention, (4) seeking sustainable financial resources; (5) implementing evidence-based policies, programs and practices to prevent violence; (6) enhancing public awareness; (7) conducting surveillance; (8) building LHD practitioner capacity and skills to prevent violence; and (9) supporting the medical community to assess and respond to violence (Safe States, 2011).
In the past two decades, the field of violence prevention has established what works to prevent YV, created and tested model programs, and examined how to establish community partnerships and build communities’ capacity to design, implement, and evaluate youth violence prevention (YVP) efforts (David-Ferdon et al., 2016). Despite the large body of literature and registries listing evidence-based YVP programs, practices, and policies, communities struggle to implement them in a coordinated manner and on a large-enough scale to realize community-level impacts. Barriers to implementation include the absence of a unifying philosophy and local data, the challenge of selecting a program within a theory of change, the use of untested programs, and challenges in implementing programs with fidelity (Cawood, 2010). LHDs, schools, juvenile justice centers, and other sectors often select and implement strategies in silos to address YV with the populations they serve. Coordination across LHDs and other community organizations can improve the effective use of resources (e.g., by addressing duplication and enhancing complementarity in programming) and result in greater community-level impacts (Lawal et al., 2013).
Furthermore, while LHDs have served as conveners and leaders to mobilize communities in addressing public health issues, such as infectious diseases and emergency preparedness, they do not always consider violence prevention (VP) a priority. In 2016, only 22% of LHDs surveyed by NACCHO reported the provision of VP services (NACCHO, 2016). LHDs may require support to build critical capacities such as community assessment, strategic planning, data-informed decision making, and evaluation (Lawal et al, 2013). Training and technical assistance (TTA) is one strategy for enhancing LHD’s readiness to implement community-wide efforts. However, there is limited consensus regarding the essential features of TTA and how to provide it with quality. A systematic review indicated that an explicit model or organizing framework is rarely used to plan, implement, and/or evaluate TTA (Katz & Wandersman, 2016). Given the rarity of an explicit model/framework, TTA tasks (e.g., conducting a needs assessment) vary widely across efforts designed to affect the same outcomes, and TTA is often not delivered systematically (Katz & Wandersman, 2016).
Background
The American Institutes for Research (AIR) and CDC partnered on a 5-year Youth Violence Prevention Training and Technical Assistance (YVP TTA) Initiative to bring these different bodies of knowledge together. This initiative was designed to (1) prevent violence using a public health approach, (2) use available data to inform and evaluate YVP efforts, and (3) engage key stakeholders in sharing responsibility to prevent YV, including public health, justice, education, police, social services, and youth-serving organizations (CDC & National Center for Injury Prevention and Control, 2018). We developed a framework identifying the specific capacities and outcomes the initiative wanted to target (Figure 1) and defined explicit steps in the initiative’s TTA approach. The framework was grounded in the premise that strengthening certain capacities (Table 1) through TTA would lead to achieving targeted short- and long-term outcomes. It was developed by (a) reviewing relevant literature (e.g., evidence-based TTA, successful LHD-led YVP efforts), (b) conducting nine interviews with LHD directors, and (c) interviewing eight LHD staff involved in another relevant CDC-funded initiative. To develop annual benchmarks for each capacity area, we asked sites what they needed each year to have the necessary capacity at the end of the project and had expert consultants review the decision points/criteria and offer suggestions.
Logic Model for the Youth Violence Prevention Training and Technical Assistance Initiative
Youth Violence Prevention 1 and 2 Sites’ Benchmarks in the Final Year, and Technical Assistance Specialists’ Ratings of Sites on Those Benchmarks
Note. “NA” means that the indicator is YVP1- or YVP2-specific. YVP = youth violence prevention; LHD = local health department; YV = youth violence; TTA = training and technical assistance.
In early stages, LHDs convened a coalition (or brought together an existing coalition) representing diverse multisector perspectives to discuss the problem of YV (who was affected, under what circumstances, and in what settings). The coalition began by discussing goals (the vision) for the coalition work as well as identifying and looking at data (e.g., youth homicide, school bullying rates) to determine the scope of the YV problem and related risk and protective factors. Many also determined what existing programs, practices, or policies were already in place in their community and began to develop a community-wide violence prevention plan while continuing to recruit and engage coalition members and other key stakeholders. As part of the initiative, LHDs received TTA in a number of areas such as developing and enhancing coalitions; engaging community members (including youth); developing comprehensive, data-informed strategic plans; selecting, implementing, and evaluating programs, practices, and policies based on the best available evidence; using communication strategies to change community norms; and engaging in systems change to support the long-term sustainability of prevention activities. TTA was free of charge, and no grants were provided to LHDs to conduct YVP activities. The focus of this article is to understand both the communities’ participation in the TTA and the associated outcomes.
Method
Recruitment
A multiphase process was used to recruit sites (which were all LHDs), with the goal of selecting 12, as specified in CDC’s request for proposals. First, CDC identified the top 100 violent cities based on violent crime and homicide rates from the Uniform Crime Reporting data (United States Department of Justice, Federal Bureau of Investigation, 2015). Information about each city’s geographic location (urban, suburban, rural), size, and previous involvement in CDC initiatives was added to ensure diversity. After reviewing these cities, 19 sites were invited to participate (Phase 1) and four applications were received. As a result, 45 additional sites were invited to participate (Phase 2) followed by 34 more sites (Phase 3), which resulted in 17 interested candidates. Reasons given for declining AIR’s invitation included: lack of interest in unfunded initiatives and other health issues being a higher priority than YVP.
Interested sites completed application materials to assess their readiness to engage in this specific initiative, based on research suggesting the importance of using this type of information when selecting sites (Dymnicki et al., 2014). They were asked to provide information about the following readiness criteria: (1) their past and current activities and motivation/capacity to engage in YVP work, (2) their local needs/problems and existing VP infrastructure, and (3) key partners involved in this effort. They were also asked to provide letters of support from their LHD and community partners. Three members of the AIR team scored (up to 100 points) each application using predetermined readiness criteria. The team averaged these ratings and discussed any applications for which raters’ scores varied by more than 10 points.
Site Selection
The goal was to identify eight low capacity (referred to as Youth Violence Prevention Community 1 [YVP1]) and four high capacity sites (referred to as YVP2 sites) to participate. The AIR team discussed each applicant and unanimously agreed on the final group of 12 sites, who all accepted. Across the three raters, the average rating for the 12 invited sites was 86.2 (ranging from 80.1 to 90.5).
Participants
Eight LHDs (YVP1) had a lower capacity of YVP work (e.g., did not have a formal infrastructure for multisectorial YVP) and did not yet have a strategic plan. Four LHDs (YVP2) were at higher capacity of YVP work (e.g., had a formal infrastructure on which to build in the current initiative). The criteria for growth in the initiative (defined as changes in capacity) was relative to each group’s starting point. Each site identified an LHD point of contact (POC) to serve as the primary TTA recipient.
TTA Activities
AIR’s TTA specialists, who have expertise in YVP and community-based initiatives, and content experts delivered several modes of TTA to LHD representatives and other community partners for 4 years. This included site-specific and cross-site TTA, as well as in-person and virtual TTA. TTA focused on topics such as communicating a YV public health approach to partners, recruiting coalition members, and developing a YVP plan that community members supported. Site-specific TTA included monthly phone calls between LHD POCs and TTA specialists, as well as annual site visits (for YVP2 sites). Cross-site TTA included an annual in-person convening, as well as online learning events (OLEs), community-of-practice (CoP) calls during Years 3 and 4, and access to an online portal with additional resources and opportunities for peer dialogue.
The initiative’s TTA approach consisted of the following steps (based on activities, specific capacities, and outcomes listed in the initiative’s framework):
Sites completed an initial capacity and readiness assessment that they discussed the results of with their TTA specialist.
Each site’s TTA specialist took time to learn about a site’s culture, context, values, goals, needs, and desires, with the goal to build trust, mutual respect, and strong rapport. Sites and their TTA specialists collaboratively developed a TTA plan that emphasized the public health approach to YVP, aligned with the initiative’s capacity framework. Each site’s plan included TTA goals and objectives, activities and staff responsibilities for each objective, and a timeline.
TTA specialist and site met monthly to discuss progress, successes, and challenges (which were documented in TTA plans and a TTA tracker).
TTA specialists and sites engaged in continuous evaluation to track progress toward achieving their TTA plan goals and updated TTA plans appropriately.
Data Sources
The AIR team applied a mixed method approach, collecting and analyzing quantitative (e.g., TTA tracker, TTA Activity Checklist, and LHD POC Quantitative Survey) and qualitative (e.g., primarily group interview) data. Before recruiting sites, AIR and CDC identified short- and long-term outcomes for YVP1 and YVP2 sites. These outcomes were reflected in the initiative’s logic model, which described TTA activities, inputs, and outputs, as well as short- and long-term outcomes (see Figure 1).
TTA Tracker
AIR’s TTA tracker tracked each site’s progress in YVP capacity areas (see Table 1) and identified the most common TTA requested and the most common format of TTA offered. The TTA tracker also helped TTA specialists monitor and improve the TTA offered and monitor sites’ responses to that TTA.
TTA Activity Checklist
Participants who attended OLEs, recorded presentations, and/or in-person meetings were invited to complete an online checklist. This checklist solicited feedback on whether a TTA session achieved its learning objectives, increased participants’ content area knowledge, and was delivered effectively. It also asked how participants planned to use the information. The checklist was completed by most site representatives after annual in-person meetings and 76 participants after 23 OLEs (average of four participants per event, ranging from two to 10, with representation from, on average, seven of 12 sites).
LHD POC Quantitative Survey
AIR administered an annual online survey (lasting approximately 30 minutes) to the LHD POC at each site. The survey solicited information on progress toward key initiative outcomes, and on the role of TTA in that process. Survey response rates were consistently high (all 12 sites completed the survey in Years 1–3; 11 sites completed it in Year 4).
Qualitative Group Interview
AIR conducted an annual group telephone interview with LHD POCs, LHD leaders, other staff, and community partners, using a semistructured protocol. Interviews generally occurred after each site’s survey closed, so that survey findings could inform interview questions. Interviewees were asked about the overall functioning of their YVP coalition and about site-specific benchmarks. Only one site did not participate in the interview during Years 1 and 2 because of competing demands.
Design and Data Analysis
Descriptive analysis of quantitative data sources (TTA tracker, TTA checklist, and annual survey) was conducted in SPSS version 16.0. Two team members analyzed qualitative interview data using content analysis procedures (Krippendorff, 1980). A modified grounded theory approach (Kloos et al., 2005) allowed coders to uncover major themes and patterns within and across sources (Denzin & Lincoln, 2003; Dey, 1993; LeCompte, 2000). Team members developed an initial list of codes based on the research questions; identified early patterns in the data (revising the initial list of codes as appropriate); and then coded the data systematically based on the refined coding scheme. The team drew conclusions by identifying and interpreting coding patterns such as high-frequency codes and coding clusters. To test for interrater reliability, each team member coded the same transcript in NVivo 11, and the team met to compare coding and reconcile disagreements. After coding, the team organized the data according to the research questions. AIR’s Institutional Review Board provided human subject review and study oversight.
Results
TTA Initiative Participation
AIR offered several types of cross-site and site-specific TTA activities (Figures 2 and 3). There was considerable variation in TTA uptake by site, and the total number of site-specific and cross-site events for both YVP1 and YVP2 sites decreased over time (Figures 2 and 3). 1 According to annual interviews and surveys, the decline in TTA requests over time was typically indicative of sites’ increasing capacity.
TTA Events Delivered Annually in the YVP TTA Initiative to the Eight YVP1 Sites Reflecting Site Specific and Cross-site Activities, As Entered by the Technical Assistance Specialists Into an Online TTA Tracking System
TTA Events Delivered Annually to the Four YVP2 Sites Reflecting Site Specific and Cross-site Activities, As Entered by the Technical Assistance Specialists Into an Online TTA Tracking System
Experience With TTA Specialists
POCs met with their TTA specialists an average of nine times per year (ranging from three to 15 times). Seven sites had POC turnover. The majority of POCs (including those new to the position) reported each year that they had a collaborative, trusting relationship with their TTA specialists. They also reported that YVP TTA Center staff demonstrated cultural competence and an understanding of sites’ local context and history.
CoP Calls
Fourteen CoP calls were held during the initiative, with five sites participating in each call on average (ranging from two to eight sites per call). Topics included engaging public health directors and community members in YVP efforts, strategic alignment, and increasing local resources for YVP. Nearly all sites—but particularly YVP1 sites—reported that CoP calls encouraged positive interactions among members and provided a structure for a learning community.
Site Visits
YVP2 sites participated in three site visits, on average. POCs from these sites indicated that these visits had significant benefits because they focused attention on TTA needs that were important for their LHDs and communities and identified specific action steps for follow-up during monthly TTA calls. The only suggestion for improvement, shared by one site, was to offer more site visits.
In-Person Meetings
Ten LHD POCs, on average, attended the three in-person, 2-day meetings for YVP1 and YVP2 sites. The majority of learning objectives for each meeting were accomplished. For example, all but one person agreed or strongly agreed that the learning objectives had been met for all five sessions of the final in-person meeting. Each year, all respondents indicated that they intended to use or apply information gained from the sessions in their professional work. They felt that sessions were clear and increased their familiarity with the topics, that presenters were knowledgeable about the content area, that information was presented effectively, and that teaching methods were effective. Respondents suggested that meetings could be improved by slowing down the pace, increasing the duration, and providing more discussion time. This feedback was addressed (in later meetings) by providing breakout groups to increase learning time.
Online Learning Events
Seventeen OLEs were held: five each year in 2015 through 2017 and two in 2018. Topics included communicating data effectively; engaging diverse audiences; resource mapping; community violence and the built environment; strategic planning; and selecting YVP programs, practices, and policies based on the best available evidence. The majority of learning objectives were accomplished for the 17 OLEs. Participants indicated that OLE elements were clear, effective, and helpful, and all participants agreed or strongly agreed that session learning objectives were met in 10 of 17 OLEs. Scheduling (e.g., competing priorities) prevented full attendance, but sites could access archived OLEs at any time through the YVP portal.
YVP TTA Initiative Outcomes
Desired community project outcomes included (among others): enhancement of infrastructure in participating communities to implement YV prevention plans, increased role for the LHD and its partners in YVP, and implementation of at least one YVP program, practice, or policy (only for YVP2 sites). Achievement of outcomes at project end was determined based on several data sources (e.g., site-specific capacity ratings by technical assistance specialists, Table 1), quantitative survey, interview data collected). Themes emerging from this analysis, outlined below, reflect the major topics and capacity areas targeted by the YVP TTA Initiative.
Facilitating Diverse and Inclusive Participation in Local YVP Efforts. Site feedback indicated increased local YVP participation after the initiative began, including partnerships with other services, small businesses, family organizations, and faith-based institutions. Most sites (11 of 12) also described engagement with youth as active partners, not just recipients of YVP programs. TTA Tracker data documented that these topics were frequently addressed in individual and group TTA activities.
Engaging Coalition Members in the Public Health Approach. Across the initiative, sites increasingly engaged coalition members (e.g., staff across different youth-serving sectors including education and juvenile justice) in the public health approach. This included collecting and analyzing data (e.g., youth violence surveillance data and risk/protective factor data), mapping resources, understanding what evidence-based strategies are, and knowing how to identify these strategies. For example, six of eight YVP1 sites reported progress in engaging coalition members in data collection and/or analysis tasks and in increasing coalition members’ understanding and awareness of the public health approach.
Strengthening LHD Infrastructure for YVP. LHD staffing devoted to YVP increased after the initiative began and was sustained throughout. For seven sites that reported the number of full-time LHD staff devoted to YVP in their applications, the median number of staff at initiative start was 1 (interquartile range = 3) and at initiative end was 4 (interquartile range = 7). LHD site leadership (6 of 8) were also generally supportive of YVP, particularly in advocating for LHD strategic plans to mention reducing violence.
Integrating YVP Into LHDs. Sites initially made tactical/procedural changes to integrate YVP into LHDs (e.g., adding YVP as a metric to a LHD’s strategic plan). Nine sites later reported more wide-reaching efforts to integrate YVP into other aspects of LHDs. For example, one site reported integrating YVP into its LHD’s health equity initiative.
Selecting YVP Programming Based on the Best Available Evidence. Seven of eight YVP1 sites used a data-informed process to consider the fit between strategy and community (according to TTA specialist’s ratings) and selected a strategy for implementation. Only one site moved from selecting a strategy to implementation during the initiative. This strategy focused on addressing increased violence at local parks and the resulting fear among local youth about using the parks. Three of four YVP2 sites conducted an outcome evaluation of an evidence-based strategy that was being implemented in the local community.
Discussion
Implications for Practice
AIR’s evaluation of the YVP TTA Initiative generated findings that could inform the work of other LHD practitioners who would like to successfully engage in similar multisectoral YVP efforts. First, LHDs can advance YVP work, without using new funding as a catalyst, as long as activities align with other health department priorities. The TTA was provided to participating LHDs free of charge but we did not provide direct funding to LHDs. Therefore, most LHD POCs quickly found ways to leverage existing LHD resources. POCs linked YVP to their other work responsibilities and partnered with other LHD staff so that YVP work aligned with other priorities and programs, such as health equity, adolescent health, school health, maternal and child health, bicycle and pedestrian safety, and obesity prevention. In this way, LHD POCs were able to launch and lead coalitions, develop or update strategic plans, and work with partners to implement YVP strategies, with the support of their LHD leaders.
Second, findings highlight the importance of LHDs intentionally trying to decrease silos among different sectors focused on similar populations or communities through multisector coalitions or partnerships. For example, the TTA was designed to create infrastructures to build new partnerships (including reaching out to partners who may not have been included previously but who had a stake in the issue). The TTA also highlighted redundancies in terms of multiple programs being implemented that were designed to target the same outcomes or the same populations. In one site, the LHD was able to strengthen its partnership with schools, including charter schools, which brought neighborhoods closer together. These types of partnerships that focused on violence prevention broadly, were maintained after the initiative ended and are critical mechanisms through which LHDs can effect change.
Third, future efforts to effect communitywide change should encourage LHD leaders (i.e., senior health officials) to play a larger and more defined role in YVP. While all LHD leaders involved in the initiative expressed general support for a public health approach to YVP, some felt their only role was to authorize a staff person to work part-time on YVP. Others served as champions (e.g., Damschroder et al., 2009). Champions regularly requested progress updates; educated stakeholders in the community (e.g., elected officials, leaders of agencies) about the public health approach to VP; found ways to increase funding for YVP (e.g., one site successfully obtained several Department of Justice youth violence focused grants); supported the recruitment and hire of additional short-term staff (e.g., youth leaders, CDC public health associates) and sometimes even established a permanent position (such as a YVP coordinator) in the LHD. Having members of the LHD convene or participate in community-wide YVP efforts is not sufficient; LHD leaders need to be involved in these types of efforts from the outset and be engaged throughout.
Limitations
First, while the AIR team used a mixed-methods approach to evaluate the TTA and the achievement of outcomes, it employed a case study design. While case studies can generate rich and holistic accounts of program implementation and evaluation efforts, limitations of this design include not being able to generalize results to the wider population, difficulty in replication, and a researcher’s own subjective feelings affecting the interpretation of data and findings presented (Flyvberg, 2006). Future work may include comparison sites to understand differences in outcomes between sites that did and did not participate in the TTA. Second, most collected information was self-reported by sites, introducing the possibility of social desirability bias. This limitation was addressed by triangulating self-reported site data with data collected from technical assistance specialist. Third, due to space limitations, this study does not discuss or evaluate the outcomes achieved by implementing YV interventions and programs in the YVP2 communities, but this could be the topic of a future paper.
Conclusions
LHDs have the potential to convene community stakeholders in implementing the public health approach to YVP and use this approach to effect community-wide change. Quantitative and qualitative data from multiple sources indicate that LHDs can, with TTA support, facilitate progress in implementing multisector YVP efforts (through developing multisector coalitions, increasing LHD infrastructure, and selecting appropriate YVP programming). However, these efforts take time to develop, and there is variation in progress by site characteristics.
