Abstract
It is well known that perceptions of intervention characteristics (e.g., cost, source, evidence strength and quality) are a critical link from dissemination to implementation. What is less known is the process by which researchers understand the characteristics most valued by key intermediaries (i.e., real-world decision-makers), particularly in the federal system of Cooperative Extension. In Extension, university-based specialists are available to assist county-based agents in program selection, delivery, and evaluation. For this work, a sequential explanatory mixed-methods design was used to conduct surveys and semi-structured interviews, informed by the Diffusion of Innovations theory and Consolidated Framework for Implementation Research. Educators and specialists were recruited across 47 states to identify characteristics of health promotion interventions that facilitate the adoption decision-making process. Analysis of intervention attribute importance survey data was conducted through a one-way ANOVA with Bonferroni post hoc test to determine individual variable differences between responses. Interviews underwent a conventional content analysis. In total, 121 educators and 47 specialists from 33 states completed the survey. Eighteen educators and 10 specialists completed interviews. Educators and specialists valued components such as the community need for the intervention, and potential reach compared with other components including previous delivery settings and external funding of the intervention (p < .05). Qualitative data indicated divergence between educators and specialists; educators valued understanding the intervention cost (time and training) and specialists valued the evidence base and external funding available. Intervention developers should communicate information valued by different stakeholders to improve the adoption of evidence-based interventions.
Keywords
Health educators face the challenging task of choosing from the myriad of interventions available to meet specific community needs (e.g., youth nutrition education or older adult physical activity promotion). Adding to the complexity of this task is that to determine the best intervention for a given need and setting, educators need to discern both the level of evidence (e.g., from practice-tested to peer-reviewed; Brownson et al., 2009) and key characteristics of the intervention that determine whether it will be a good fit with their specific delivery setting (Damschroder et al., 2009). Delivery personnel’s awareness of intervention characteristics significantly influences the adoption decision-making process (i.e., seeking and processing information to decide whether to implement an intervention; Damschroder et al., 2009; Feldstein & Glasgow, 2008; Rogers, 2003). When this information is not readily available, delivery personnel may perceive a lack of fit with their needs and create “commonsense” programs (Hansen et al., 2017) rather than adopting evidence-based interventions. Overall, there are challenges across the field of public health in the adoption and implementation of evidence-based interventions, with an average translational lag time of 17 years from original research to implementation in intended delivery settings (Balas & Boren, 2000; Morris et al., 2011).
One such delivery system is the national Cooperative Extension Service (Extension). Extension is housed within land-grant universities located in every state and territory and reaches over 6 million Americans per year through “bringing the university to the people” (Rasmussen, 2002). A national Extension priority is promoting health throughout the life span through delivering chronic disease prevention programs (Braun et al., 2014), such as statewide walking programs (Balis & Harden, 2021; Estabrooks et al., 2008; Harden et al., 2019); nutrition and physical activity policy, systems, and environment interventions (Holston et al., 2020; Murriel et al., 2020); and low-income nutrition education (Auld et al., 2015; Dollahite et al., 2014).
In Extension, programs are delivered by county-level educators, who are supported by state-level specialists. Educators (typically master’s degree level) identify community needs and then search for, adopt, and implement programming to address these needs. Extension state health specialists (typically doctoral level) serve as intermediaries between researchers and educators and provide program training and support (Strayer et al., 2020). Both educators and specialists have high autonomy to select interventions to deliver or support (Balis et al., 2021). Evidence-based practice is new to Extension (Dunifon et al., 2004; Fetsch et al., 2012), and increasing uptake of evidence-based programs has been a challenge. Previous work identifying adoption decision-making characteristics valued by educators found that an intervention’s effect, compatibility within Extension, high reach, and ease of delivery most influenced educators’ adoption. However, this work only explored perceptions related to a specific intervention and only included educator perspectives (Downey et al., 2012). The work presented here aimed to understand overall program characteristics that guide educators and specialists in developing their program portfolio.
To improve the adoption decision-making process and increase the uptake of evidence-based chronic disease prevention programs in Extension—and other community-based systems—there is a need to determine which intervention characteristics are most valued by potential program adopters. That is, there is a need to understand how public health professionals seek and process information to decide whether to initiate the delivery of interventions. Therefore, the purpose of this study was to identify intervention characteristics that are valued and considered in the adoption decision-making process for Extension educators and specialists across the nation.
Method
Study Design
A sequential explanatory mixed-methods approach was used, in which surveys were followed by individual semi-structured interviews (Creswell & Plano Clark, 2011). This approach (quantitative followed by qualitative) was selected to allow the research team to interpret and expand on the survey findings through in-depth inquiry (Ivankova et al., 2006). Roger’s seminal Diffusion of Innovations theory (DOI; Rogers, 2003) and the Consolidated Framework for Implementation Research (CFIR; Damschroder et al., 2009) were used to develop both the survey and the semi-structured interviews. The DOI posits that key intervention characteristics include the intervention’s relative advantage compared with common practice, compatibility to the system, testability, and potential for reinvention (Rogers, 2003). CFIR highlights the importance of these constructs as well as other intervention characteristics: the intervention’s cost, source, design quality and packaging, and adaptability (Damschroder et al., 2009). This study was conducted in tandem with work on Extension specialists’ and educators’ information-seeking practices (Strayer et al., 2020; Strayer et al., n.d.). All study procedures were approved by the Institutional Review Board of Virginia Tech.
The methods and results of this study were developed and interpreted within an Integrated Research-Practice Partnership (IRPP). An IRPP is a participatory approach that equally values researchers’ and educators’ priorities (Estabrooks et al., 2019), and had led to co-produced interventions with improved adoption and delivery (Harden et al., 2017). This IRPP—consisting of the research team and Extension educators—provided unique insights on specialists’ and educators’ perceptions of intervention characteristics. Regular meetings with the IRPP to discuss survey responses from educators and specialists were an integral part of our interpretation, as they aided in the construction of the semi-structured interview guide. Notably, the research team conducted the educator portion of the research prior to the specialist portion. Results from the educator portion of the work were discussed with the IRPP, and the research team used this input to design the specialist survey.
Participants and Recruitment
Quantitative
First, the research team identified specialists in each state through searching individual state Extension websites. Specialists were considered eligible if they held a statewide leadership position in health promotion. Second, specialists were contacted and asked to identify eligible educators in their state. Educators were eligible if they were employed as an Extension health educator and currently delivering health promotion programming within communities (Strayer et al., 2020; Strayer et al., n.d.). Educator participant accrual spanned 1 month, with an initial email sent to specialists, then follow-up emails sent 1 week apart for a total of five emails or until a response was received. The educator survey was distributed in 2017.
Qualitative
Following survey data collection, the research team recruited a subsample of educators and specialists to complete semi-structured interviews. A total of 20 randomly selected educators, five from each of the four national regions (Northeast, North Central, Southern, and Western) were chosen, and 18 were interviewed in April and May of 2017. A total of 12 specialists were invited and 10 (83%) were interviewed in July of 2018. Specialists were nationally distributed: three each from Sothern and North Central regions and two each from the Western and Northeastern regions. The rationale for selecting these sample sizes to reach thematic saturation was (a) the majority of themes are typically identified within 6–12 interviews, with little new information gained as sample sizes reach 20 interviews (Guest et al., 2020), and (b) there are fewer specialists than educators.
Data Collection
Quantitative
The survey assessed intervention characteristics important to adopting health promotion programs through an item with 11 characteristics ranked on a 5-point Likert-type scale (1—not at all important; 5—extremely important). The survey was created by the researchers with input from the IRPP. Characteristics included cost, feasibility, sustainability of the program, level of comfort delivering the program, and other factors important to dissemination and implementation (Barnidge et al., 2013; Damschroder & Hagedorn, 2011; Nelson et al., 2007; Rogers, 2003). The survey also included demographic items based on standard variables described in highly cited research methodology texts (Kumar & Phrommathed, 2005) and previous work (Harden et al., 2015), including race, ethnicity, sex, state of employment, official role title within Extension, duration of employment within Extension, educational degree, and the field in which the degree was obtained. Of note, specialists were asked about their role within research, Extension, and teaching (standard academic faculty time equivalent), while educators were asked about their role within youth, adult, and older adult programming. Surveys were distributed via Qualtrics (Provo, UT).
Qualitative
The semi-structured interview guide was developed based on the DOI theory, CFIR, and feedback from members of the IRPP. Questions included what programs educators currently delivered, what was most important when selecting a program to deliver (with prompts of cost, time, equipment, location, previous experience, evidence base), as well as steps taken to understand community needs and select programs to deliver. Interviews were conducted by the lead author under supervision of the senior author. The lead author had received formal interview training through doctoral studies. Interviews were conducted over the phone, audio recorded, and transcribed by trained research assistants.
Analysis
Quantitative
The study used ANOVA testing to analyze Likert-type scale measures and rank questions and presented in Likert-type response frequencies for each of the intervention characteristics; see Figures 1 and 2 (Alwin & Krosnick, 1985). For this study, a one-way ANOVA was used in conjunction, if significant, with a Bonferroni post hoc test (Hommel, 1988) which determined the individual variable differences between responses. Alphabetical indicators were used to identify differences in Figures 1 and 2 . Statistical analyses were conducted using SPSS (v. 25.0 for Windows, SPSS Inc., Chicago, Illinois).
Importance of intervention characteristics for educators’ adoption decision-making process.
Importance of intervention characteristics for specialists’ adoption decision-making process.
Qualitative
A conventional content analysis (Hsieh & Shannon, 2005) was applied to analyze the qualitative data. That is, the coding categories were gleaned directly from the text of the interview transcripts (Hsieh & Shannon, 2005). This approach was selected to allow the data to speak for itself rather than placing it within a particular framework. Trained qualitative researchers independently separated each interview into meaning units (interview words or phrases that represent a singular meaning and are considered units of analysis; Castro et al., 2010) and then determined categories, subthemes, and themes from these determined data (Castro et al., 2010; Creswell & Plano Clark, 2011). Meaning unit quotes are direct and represent the authentic voice of participants (including filler words, repeats, and pauses) to give the reader the opportunity to feel the total tone and impression of the response (Tracy, 2013). Discrepancies that the research team could not reconcile were referred to the coordinating supervisor, who then assisted in reconciliation (Castro et al., 2010). The inter-rater reliability for the interviews was greater than 90%. Audit trails were maintained, including audio recordings, transcripts, and coding documents (Cutcliffe & McKenna, 2004).
Results
Quantitative
Educators
One hundred thirty-six survey responses were received, and 121 (89%) met the eligibility criteria. These responses were received from 33 of the potential 36 responding states (92% response rate). Educators who responded were predominantly female (91%), Caucasian (80%), and had worked for Extension for 5 or more years (68%). In addition, 71% of educators had master’s degrees within 5 years of being hired. See Table 1 for additional information.
Extension Educator and Specialist Demographics Table.
Figure 1 represents educators’ preferences of the intervention characteristics important in the adoption decision-making process. There were a few statistical differences (p < .05) between characteristics. The cost of ongoing delivery, external funding of an intervention, and the number of similar delivery settings were valued less than other factors. The need for an intervention, and reach were more highly valued.
Specialists
A total of 94 specialists were identified in 47 states in 2018. A total response rate of 50% was achieved from the original 94 identified specialists with responses from 31 (66%) states. Forty-seven (77%) of the 61 responses were completed and met eligibility requirements. The remaining 14 responses submitted were incomplete and thus excluded. Specialists were predominantly female (89%), Caucasian (70%), possessed a doctoral degree (62%), and had worked in Extension for five or more years (62%). See Table 1 for additional information.
Figure 2 represents the specialists’ preferences of intervention characteristics important for the adoption decision-making process. Specialists were similar to educators as they also valued both the scientific evidence that a program is efficacious and the community’s continued need for the intervention; these characteristics were both statistically different (p<.05) in importance from the other factors (the last five items in Figure 2). This outcome is not necessarily unexpected, as the original purpose of Cooperative Extension is to bring the university to the people, hence a direct link for having interventions linked to scientific support. The ability of the program to be sustained and the cost of ongoing delivery were the next highest reported intervention characteristics, though they were not significantly different from other characteristics with the exception of the number of similar settings that have delivered the program. Characteristics with lower values of importance were the number of similar settings that have delivered the program followed by external funding for the intervention.
Qualitative
In total, there were three main themes from both specialists and educators (presented in Tables 2 and 3). Themes, subthemes, categories, and example meaning units are presented in the tables. Frequencies (i.e., the number of meaning units) are included for each theme, subtheme, and category. The number and proportion of educators (Table 2) or specialists (Table 3) who contributed to each subtheme are also included. This approach (including both the number of meaning units and the number/proportion of interviewees contributing) was used to demonstrate data saturation. Finally, meaning unit examples by category were included to give insight into the coding process (Graneheim & Lundman, 2004). Full results are presented in the table, and the most prevalent subthemes (i.e., described by >50% of respondents) are described in the following section.
Extension Educator Qualitative Results.
Specialists’ Qualitative Results.
Educators
Table 2 represents the themes of (a) educator adoption process, (b) educators’ perceptions of program adoption factors, and (c) characteristics that influence program adoption. Three subthemes emerged from the educator adoption process theme: (a) strategies for need assessment, (b) educators perform research, and (c) level of autonomy to choose programming. Need assessments, mentioned by all 18 interviewees, were the most common way for educators to determine a health promotion need in the community. Meetings with community groups were a common method of assessing needs. As for educators perform research, interviewees shared that they perform their own research to determine programming to meet the need of a community.
The second theme, educators’ perceptions of program adoption factors, includes the subthemes of (a) definition of evidence-based interventions, (b) role in Extension, (c) funding, and (d) educator opinions. Educators’ definition of evidence-based interventions centered around evidence of effectiveness. This included the presence of the intervention in a peer-reviewed journal and testing of the intervention in a research setting. Considering Educators’ role in Extension, leadership and helping community members improve health were considered most important.
The final theme, characteristics that influence program adoption, reflects factors that Extension educators consider when adopting a program for delivery. Subthemes were (a) cost of programming, (b) funding, (c) program features, (d) location for delivery, (e) participants, (f) sustainability, (g) program creation, and (h) educator previous experience. Considering cost of programming, Educators stated that delivery costs—including their time—were important considerations for program adoption. They also reported having to determine the cost on their own with little information available from the program materials. As for funding, both fee-based programming and grants were commonly used sources. Regarding program features, effectiveness and having a standard curriculum were considered most important. Educators were divided on delivery location influencing program adoption, as some shared that location is a barrier and others shared that it is not. Concerning participants, the target audience of a program was an important consideration. Finally, interviewees were also mindful of program sustainability, and shared that funds to continue program delivery (e.g., beyond the duration of a grant) were important.
Specialists
The three themes from the specialist interviews were (a) specialist perceptions, (b) characteristics that influence program adoption, and (c) system-level factors, as seen in Table 3. Specialist perceptions consisted of six subthemes: (a) educator factors, (b) definition of an evidence-based intervention, (c) importance of evaluation, (d) role in Extension, (e) purpose of Extension, and the (f) dissemination of programming. As for educator factors, Specialists were concerned about whether program duration was feasible for educators as well as educators’ level of comfort delivering programming. Considering the definition of evidence-based interventions, specialists considered whether a program has been evaluated for effectiveness or is evidence-informed. Regarding the importance of evaluation, specialists shared that programs must have an evaluation component, and this should ideally consider long-term outcomes. Concerning role in Extension, specialists mentioned avoiding program duplication and judging program adaptability. As for the purpose of Extension, specialists primarily viewed it as an educational delivery system. Finally, specialists shared several processes for dissemination of program, including conferences, open-access materials, and other specialists.
The second theme identified through the specialists’ interviews was characteristics that influence program adoption, with subthemes of (a) program features and (b) cost of programming. Program features included the program’s goal aligning with the need of the community to be considered for adoption. In addition, it was mentioned that programs need to be adaptable to educators’ unique environments. The preference for the program to be previously implemented in Extension or similar settings was also important. Concerning cost of programming, specialists mentioned time for training, training cost, and curriculum cost as important considerations.
The final theme was system-level factors, which includes three subthemes of (a) funding, (b) organization structure, and (c) organizational requirements. Funding considerations included, specialists were considered with whether funders’ needs were met and shared that funding sources are important for program adoption. Finally, organizational structure also influenced program adoption, with considerations of sustainability and programming support through staffing.
Discussion
Comparing Educator and Specialist Responses
Similarities and differences between educators and specialists can be seen in Figure 3. Educators and specialists differed in their perceptions of two key intervention characteristics defined in the CFIR: funding and intervention source. First, related to funding, the survey results showed that specialists were more concerned over the sustainability of programming than educators. In the interviews, educators mentioned the need for programming to be fee-based to sustain delivery and that grant funding was important but sometimes a barrier to program delivery options. Specialists viewed this barrier instead as a condition to meet funder’s needs with programming to ensure sustainment and delivery. These distinctions are important to consider for research–practice partnerships. Educators have strong communities ties, and may be leery of partnering on grants that require delivering specific programs for specified amounts of time. Rather, educators may prefer to charge program fees to sustain delivery of programs valued by community members. Overall, maintenance of programming beyond the duration of grants should be considered to ensure Extension reaches populations most in need (who may not be able to pay for programming) and can implement policy, systems, and environment level interventions that are not fee-based (e.g., safe routes to school or point of decision prompts to encourage physical activity; Balis & Strayer, 2019; Spear et al., n.d.).
Relationship between educators’ and specialist’ perceptions of intervention characteristics.
Second, related to the intervention source, educators considered multiple local resources (e.g., local department of public health, community health centers, etc.) when searching for potential programming to adopt and deliver; this was not mentioned by specialists when searching for potential programming. This distinction may be related to the location of educators within communities versus specialists typically housed within university settings (without direct access to community partners). Along with funding, both parties discussed the cost of programming in their considerations for program adoption, including costs for training, implementation, and sustainment. Specifically, educators and specialists both discussed the time required for training, implementing the program (educators), and providing training for educators (specialists), along with cost of curriculum, travel for training, and training materials.
In addition, educators reported that they perform research to identify programming or use existing literature to create new programming, indicating that often, educators do not see current reported intervention characteristics as a complete fit with their needs. However, specialists mentioned that educators have no formal adoption process and that educators’ interests inform their choices; one specialist mentioned that a disconnect between the science and educators exists. In addition, specialists worried about educators’ understanding of programming adaptation, though educators did not mention their use of adaptation in interviews. These perceptions likely are a result of educators’ and specialists’ primary responsibilities: Educators deliver programs while specialists connect research to practice and support educators in delivering evidence-based programs.
Discrepancies Between Surveys and Interviews
Evidence strength and quality, as described in the CFIR, were important to educators and specialists. Both survey and interview data suggest that program goals must align with the needs of the community. Thus, program goals should be easily accessible for potential adopters, that is, educators or specialists, to facilitate adoption. In addition, both groups reported the importance of using evidence-based programming, but the meaning of evidence-based to these groups was unclear. When asked, educators and specialists often described evidence-based as programming being “effective”; specialists also described Extension programming as being evidence-informed in general. Program evaluation and demonstrated effectiveness were mentioned by both parties as sufficing to meet the evidence-based criteria. Specialists also mentioned that evidence-based could be from credible institutions (e.g., National Institute of Health) or peer-reviewed resources (e.g., academic journal). Taken together, more clarification on the meaning of evidence-based is needed. To remedy this, training and technical assistance within Extension should include levels of evidence as well as the need to select the best fit program from the available evidence.
In addition, along with program goals, the program’s duration and feasibility were discussed by both parties; they mentioned the time commitments required as a barrier to program implementation. Also, the level of comfort delivering the program was not highly ranked in the survey portion of the study, but specialists often expressed the need for educator comfort to be addressed (through methods such as training) to promote adoption and implementation. Finally, specialists and educators were divided on the idea of avoiding program duplication and program creation. Within each group, some respondents believed it necessary to continuously develop new programming, while others believed it is something to be avoided. This may be a result of Extension’s history as an educational delivery system valuing the creation of programming (Balis et al., 2019, 2020). However, this perception is critical to address to increase the use of evidence-based programming. Suggestions include changing Extension evaluation metrics (e.g., from listing “programs created” to “evidence-based programs adopted.”)
Program Adoption Barriers
The work presented here did not identify a standardized process by which educators and specialists learn of intervention characteristics, but does highlight key characteristics considered, by role within Extension, when adopting an intervention. Specialists reported focusing on academic and scholarly processes along with funding requirements, while educators’ focus was on individual program adoption and implementation methods needed for their unique environments. However, several intervention characteristics were deemed important for program adoption by most specialists and educators, including the program goals and duration as well as the cost of training (including time and training components), curriculum, and materials. In addition, the time needed for both specialists and educators to implement programming, the evidence-based programs, and the evaluation tool used to continue showing effectiveness in new settings were also mentioned.
The characteristics mentioned by both educators and specialists are similar to program translation barriers found in previous work. For example, specialists and educators both mentioned time as a barrier to program training and implementing. In previous research, time constraints have been found to be a barrier to program scaling in different health promotion systems (Gravel et al., 2006; Norton & Mittman, 2010). Communicating the time required to deliver an intervention—and whether or not the time requirements meet or exceed expectations—is imperative. Furthermore, communicating the most valued intervention characteristics may lead to collective impact (i.e., more states, specialists, and educators delivering the same evidence-based programs rather than duplicating similar program components). Another important issue was the resources surrounding program adoption, such as curriculum, equipment, materials, and funding, that are needed to facilitate the adoption process.
Also, researchers need to consider likely adaptations at the onset of program development to be clear on the evidence-based principles and what can be adapted (Klesges et al., 2005). That is, an intervention developer needs to specify the key features of the intervention (that should not be altered) and the adaptable components (that can and should be altered; Chambers & Norton, 2016; Kirk et al., 2020). For example, a program was research-tested to be delivered twice a week for eight weeks, but this might not be feasible for every delivery setting (Balis et al., 2018). Changes in the dose of the intervention may require additional comparative effectiveness testing to determine the flexibility of the programming. Essentially, being explicit about what can and cannot be adapted is imperative to improve translation, adaptation, sustainability, and two-way trust and communication between developers and adopters. As well, training and technical assistance on program adaptation may increase the uptake of evidence-based programs. For example, Physical Activity in Cooperative Extension is a training based on the Interactive Systems Framework that includes modules on program selection, adaptation, and evaluation (Daniels et al., n.d.; Dysart et al., 2021).
Limitations
This work is not without limitations. First, there are limited data on Extension educators’ and specialists’ knowledge and preferences for health promotion intervention characteristics, thus the data collected and analyzed for this study are not readily comparable or generalizable at a national level. This study also may be affected by response bias, or the tendency for participants to answer questions untruthfully or misleadingly, even with the usage of a mixed-methods approach (Furnham, 1986). Finally, the study may have been affected by differential selection bias between educator and specialist respondents. It is unknown why 89% of educators but only 50% of specialists responded to the survey invitation. A lower response rate among specialists has been noted in other research (Balis et al., 2021.) and may reflect survey fatigue among the approximately 100 specialists serving the country.
Implications for Policy and Practice
To increase the adoption of evidence-based programs across Extension, understanding the intervention characteristics that are valued and considered by educators and specialists is necessary. The results here revealed pertinent information that developers should include to facilitate program adoption. These characteristics include the program goals and duration; details on the time associated with training and implementation; the necessary components to implement a program (e.g., curriculum and equipment); the intervention’s evidence-based and history of effectiveness; and evaluation tools needed to report impact and scholarly reports. Future research should investigate the impacts of specifying these intervention characteristics on program adoption rates. In addition, from other research related to specialists’ and educators’ adoption and implementation practices (Balis et al., 2021; Strayer et al., 2020; Strayer et al., n.d.), it may also be in the interest of developers to consider adaptations and the impact they may have on programming fidelity prior to program dissemination.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
