Introducing a New Intervention: An Overview of Research Phases and Common Challenges

Challenges of and Progress in Behavioral Intervention Research

One of the most critical challenges in introducing a new intervention is that development and testing transpire over a long period of time, estimated at 17 yr or more. Even then, most interventions are not integrated into practice. Only 14% of new scientific discoveries ever enter real-world contexts; Americans receive only about 50% of recommended evidence-based preventive, acute, and long-term health care services; and minority populations are at the greatest disadvantage, receiving recommended, proven evidence-based care only 35% or less of the time (IOM, 2001, 2008; McGlynn et al., 2003).

The need to shorten the time span from idea inception to translation has increasingly become a concern of funders, health systems, researchers, and the public (Glasgow, Lichtenstein, & Marcus, 2003). In response to this research-to-practice crisis, the National Institutes of Health (NIH) launched its Roadmap for Medical Research in 2004 and the Common Fund in 2006 to develop and support Roadmap initiatives, yet these opportunities have favored knowledge transfer from laboratory understandings of disease mechanisms to the development of diagnostic therapies (T1 research; Bear-Lehman, 2011). The translation of results from clinical studies into everyday practice and decision making (T2 research) has reflected only less than 5% of the NIH budget, and research support for dissemination and implementation of interventions (T3 research) is rarely funded (Westfall, Mold, & Fagnan, 2007; Woolf, 2008).

Another challenge in introducing a new intervention concerns the complexity of practice environments (Burke & Gitlin, 2012). Embedding scientific knowledge into practice requires its own set of implementation steps and evaluative processes (Fixen, Naoom, Blasé, Friedman, & Wallace, 2005; Wilson, Brady, & Lesesne, 2011). Unfortunately, most researchers lack a clear understanding of the contexts in which their interventions may eventually be located. Understanding organizational factors affecting knowledge uptake and potential barriers to intervention adoption up front in the intervention development process may better inform trial designs that in turn advance integration of the intervention into practice if it is proven effective (Cochrane et al., 2007; Gitlin, Jacobs, & Earland, 2010; Glasgow, 2002; Lavis, Robertson, Woodside, McLeod, & Abelson, 2003).

Another important challenge concerns the level of responsiveness of tested interventions to the needs of diverse populations. Because efficacy trials are typically dependent on study volunteers and focus on internal versus external validity, inclusion criteria usually delimit a homogeneous population. Thus, generalizing to nonvolunteers may be hindered, necessitating further testing of an intervention’s acceptability and effectiveness for a broader group of potential users. Tailoring intervention protocols to the needs and preferences of diverse populations and testing cultural modifications may require additional research steps and further testing before rolling out and scaling up for a particular intervention.

Thus, length of time from discovery to integration in a practice setting, complexity of developing evidence for diverse populations, and insufficient funding for developing and introducing behavioral interventions in real-world contexts continue to fuel the knowledge-to-application gap. New strategies are needed to address these challenges, and researchers are called on to more fully understand the trajectory of intervention development and implementation hurdles.

Despite challenges, great strides have been made in introducing new interventions (Campbell et al., 2000). Single and multisite interventions such as the NIH-supported Resources for Advancing Caregiver Health studies (REACH I, Gitlin et al., 2003; REACH II, Belle et al., 2006); ACTIVE to improve cognitive well-being (Jobe et al., 2001); primary care practice–based interventions to improve a variety of conditions such as dementia care (Callahan et al., 2006) or geriatric syndromes (Counsell et al., 2007); and well-constructed occupational therapy–based interventions to enhance older adult well-being (Clark et al., 1997), reduce functional disability (Gitlin et al., 2006), and increase quality of life in people with dementia and their caregivers (Gitlin, Winter, Dennis, Hodgson, & Hauck, 2010a, 2010b) are only a few of the recent studies exemplifying progress in efficacy trials that address multifaceted and complex public health challenges. Nevertheless, although these interventions have been shown to be effective, they are mostly out of reach to the public, primarily because of the complexities of workforce preparatory needs and the fiscal constraints of care settings.

Advances in principles and practices of community-based participatory research (Clinical and Translational Science Awards Consortium, Community Engagement Key Function Committee, 2011; Viswanathan et al., 2004) and use of embedded trial designs (Cooper, Hill, & Powe, 2002) provide researchers the theory base and tools to systematically integrate stakeholders and end users early on in developing an intervention to enhance relevance and the potential for generalizability. Similarly, advances in implementation science (Brownson, Colditz, & Proctor, 2012) and conceptual frameworks for guiding translational efforts (see, e.g., Glasgow, 2002; Murray et al., 2010) have contributed to more nuanced understandings of practice contexts and strategies for moving interventions into practice. Concomitantly, new pipelines are evolving, such as hybrid designs that combine developmental steps to more efficiently and quickly move interventions from test phases to implementation phases (Curran, Bauer, Mittman, Pyne, & Stetler, 2012). These developments are positively changing the conduct of behavioral intervention research.

Intervention Development Phases

Figure 1 presents an adaptation of the traditional four-phase sequence for developing an intervention: discovery; Phase 1, feasibility; Phase 2, exploratory; Phase 3, efficacy; and Phase 4, dissemination and implementation (Kleinman & Mold, 2009). The figure suggests that developing and testing health-related behavioral interventions involve an incremental, iterative process of building a robust body of evidence (discovery, proof of concept, feasibility, safety, and efficacy, similar to the traditional phased approach) followed by three additional phases—translation, implementation, and maintenance and sustainability—to embed a proven intervention and normalize it within a practice setting.

OPEN IN VIEWER

Figure 1.

The randomized trial-to-translation continuum.

Regardless of the pipeline followed, intervention development must begin with discovery (preclinical phase). Discovery is guided by theory and involves identifying a clinical problem, target population, and existing evidence (e.g., epidemiological record, previous trials) that supports the potential benefit of the intervention being developed. Identifying a base in theory (or theories) is critical to inform an understanding and evaluation of the mechanisms by which an intervention may have its desired effect. An essential question in this phase is, Why would the intervention work?

Several challenges are encountered at the discovery stage. First, the epidemiological record may be incomplete or inadequate to substantiate the scope of a health problem. Second, theories often lack a strong empirical foundation, and their application to an intervention can be difficult. Theory may suggest what needs to be changed and why but not how to induce change. Behavior change and health behavior theories tend to explain behavioral intentions or motivation but do not explain or predict actual behavior or behavior change, which is typically the intent of an intervention. Multicomponent interventions may require combining complementary theories. Similarly, an intervention targeting multisystems (individual, family, community) may require a broad ecological model supplemented with theories specific to the planned intervention activities at each level. Another challenge is the difficulty in identifying a potential practice or service context and payment mechanism; these may not even exist for the proposed intervention. Still, consideration of where and how an intervention will be embedded in the future, if proven effective, is important even at this early stage and helps establish feasibility.

Funding for the discovery stage is scarce. Intramural funds and support from a professional association may be available. If this stage is combined with Phase 1, an NIH planning grant (R34) sponsored by some institutions is a potential funding source. Also, investigators located in research-intensive universities may find pilot funds available for discovery as part of large initiatives such as the Clinical and Translational Science Award grant activities or NIH P30 grant initiatives, all of which support pilot funding, particularly for developmental projects and new investigators.

Phase 1 testing involves identifying and evaluating intervention components and determining acceptability, feasibility, and safety. In this phase, case studies, pre–post study designs, or focus groups, or a combination of these methodologies, can help define and refine intervention delivery characteristics (e.g., dose, intensity, treatment elements). Qualitative research can help investigators evaluate the acceptability and utility of intervention components and potential barriers to adherence and behavioral change. Knowledge generated from activities in this phase can advance a working intervention prototype and treatment manual. If there is a clear implementation site for the intervention if it is proven to be effective, then involving stakeholders at this stage is important. Involving stakeholders at this early stage can help identify the potential facilitators and barriers to inform development of the intervention protocol. As in discovery, funding for Phase 1 is dependent primarily on intramural support or planning grant initiatives occasionally sponsored by foundations or NIH mechanisms such as the R34 or K for new investigators.

Phase 2 involves an initial test of the intervention in comparison with an appropriate alternative. A small randomized trial (with a sample size, e.g., of 30 to 60) can be used to identify outcomes, evaluate whether measurement is sensitive to the expected change, and generate effect sizes. The latter is particularly important because it informs sample size calculations and other design elements of a Phase 3 efficacy trial. In Phase 2, monitoring of feasibility, acceptability, and safety continues, and investigators examine the theoretical basis for observed changes. Yet another important task is evaluating fidelity and refining a monitoring plan and measures for ensuring that intervention groups are implemented as intended. This phase should also yield preliminary evidence that the intervention is efficacious and well-defined treatment manuals. In addition to the funding sources discussed above, activities for this phase can be supported through the NIH R21 mechanism. Also, intramural support through pilot study mechanisms and foundation support are feasible funding alternatives.

Phase 3 represents the definitive randomized controlled trial that compares a fully developed intervention with an appropriate alternative. The most robust efficacy trials are double-blinded: Research team members as well as study participants remain unaware of group allocation. Double-blinding can be difficult to achieve in behavioral intervention research; typically, a single-blind approach is used in which only assessors remain masked to participants’ group assignment. Intention-to-treat (ITT) analysis, in which all data are used regardless of study completion, is considered the most definitive approach, in contrast to a per protocol analysis, in which only participants who complete the entire clinical trial are included in the final analyses of results. ITT may require that interpolation methodologies be applied because behavioral trials always experience attrition. Also, a modified ITT can be used in which data only of participants available at the follow-up are used, regardless of their level of intervention participation.

Determining an appropriate alternative for comparison with the intervention is an important challenge that needs to be addressed in Phase 3. Standard care, usual care, attention control, or active alternative interventions are potential comparison groups, each with its own pros and cons. No consensus exists, and researchers need to offer justification for any type of control group. An active control group is often favored for comparative purposes because it is designed to control for attention, time, and empathy, factors that are afforded the treatment group and that may account for or confound treatment benefits. An active intervention, however, should not include any active ingredients being tested in the treatment group. Active ingredients must be identified a priori and be grounded in the theoretical basis of the proposed intervention. Another consideration in this phase is monitoring fidelity in both the intervention and control groups.

Also of importance in designing an efficacy trial is attention to potential mediator and moderator factors, again according to the theory guiding the trial. Mediational analyses seek to examine the underlying mechanisms or latent variables responsible for treatment benefit, whereas moderator analyses seek to evaluate whether differential treatment effects are obtained on the basis of factors of interest (e.g., participant characteristics such as age, gender, health, cognitive status). Funding for Phase 3 trials is typically through the NIH R01 mechanism. A competitive application for an efficacy trial must be supported by pilot test results, demonstration of feasibility, expected effect size, treatment manuals detailing treatment and control group protocols, tested recruitment and enrollment procedures, and a strong theory base and hypotheses. Before conducting an efficacy trial, formal registration is required at ClinicalTrials.gov, a registry of federally and privately supported clinical trials conducted in the United States and internationally (http://clinicaltrials.gov/). Also, in addition to institutional review board approval, a Phase 3 trial requires a data safety and monitoring board that provides oversight of recruitment and accrual progress, adverse events, interim analyses if part of the trial design, and stopping rules.

In reporting efficacy trial results, investigators must address the design elements detailed by the Consolidated Standards of Reporting Trials (CONSORT) group (www.consort-statement.org/home/). Many journals require submission of a standard study flowchart and CONSORT checklist indicating that listed design elements (e.g., randomization scheme, sample size calculations) are included in the manuscript. It is helpful to become familiar with the CONSORT checklist before designing an efficacy trial and writing a grant application because it reflects a comprehensive list of necessary considerations.

Trials are increasingly costly to conduct because of the need for diverse and large samples, repeated measures, skilled personnel intervening in both treatment and control groups, and extensive monitoring. Thus, balancing costs with funding levels and necessary design elements is an ongoing challenge. Because of the budgetary constraints of funders, it is virtually impossible to design a trial involving long-term follow-up (>12 mo); thus, long-term treatment effects are difficult to evaluate, even though they are important to understand.

Demonstrating efficacy is only the first requirement on the long road to introducing an intervention into standard care. Although scientific reporting of trial results is critical, publications alone do not lead to adoption of the evidence. Hence, Phases 4, 5, and 6 reflect processes in moving from evaluating efficacy to normalizing an intervention and sustaining it in practice. Phase 4 can be either an effectiveness or a replication trial in which the intervention is evaluated within the practice or service context and with a broader group of study participants. Phase 5 consists of a wide range of translational processes, including identifying facilitators of and barriers to implementation, evaluating fidelity, fully manualizing the intervention (Gitlin, Jacobs, & Earland, 2010), developing training programs for instruction in its delivery, and scaling up for full implementation (Glasgow, 2010). Critical to this phase is ensuring that the active ingredients that make the intervention work, or its immutable components (e.g., use of a client-directed approach), are not modified. As to sustainability, of interest is evaluating whether and how the intervention is embedded in a setting such that it becomes “normal” practice.

Determining ways to monitor quality of and fidelity in delivery, receipt, and enactment of the intervention remains the primary focus of these three latter phases. Also, new evidence may emerge from practice or additional research testing the intervention, and this new evidence may lead to enhancements of the original tested intervention. Thus, mechanisms for infusing new evidence to enhance the existing intervention need consideration.

Key Considerations in Designing an Intervention

Table 1 outlines domains and specific elements to consider in constructing an intervention. Each of these considerations should be informed by theory, best evidence, practice guidelines, and clinical know-how and knowledge of the implementation goal and site. For example, evidence shows that education can enhance knowledge of a content area but not skill or behavior change. If skill enhancement is the desired intervention goal, a more interactive approach in intervention delivery would be necessary, such as using structured role-play or simulation. Similarly, evidence suggests that tailoring an intervention to specific needs or characteristics is more effective than taking a one-size-fits-all approach if the desired outcome is behavior change (Richards et al., 2007).

In addition to these considerations, conducting a task analysis of an intervention is a useful exercise. This analysis entails breaking down the intervention by detailing its theory base, treatment goals, objectives and specific activities, and hypothesized primary (proximal) and secondary (distal) outcomes to ensure alignment of elements. Figure 2 graphically displays a task analysis for a depression intervention protocol as an example (Gitlin et al., 2012). On the basis of this analysis, a detailed description of the specific content and logical sequence of activities in each intervention session can be constructed in a treatment manual.

OPEN IN VIEWER

Figure 2.

Task analysis for a depression intervention protocol.

Emerging Models

Given the elongated process of building and testing a behavioral intervention as shown in the six test phases, recent efforts have been directed at developing hybrid models to accelerate the discovery process (Curran et al., 2012). One approach is inclusion of economic analyses in Phase 2 or Phase 3, or both, to evaluate the implementation potential of an intervention from a cost perspective. Economic evaluations inform the investment necessary at the individual, agency, and societal levels.

In addition, “practical” or embedded trials combining efficacy and effectiveness or effectiveness and implementation and that involve the practice setting as the test site for which the intervention is intended may shorten the developmental trajectory. Other hybrid models involve conducting implementation trials that secondarily evaluate treatment benefits (Curran et al., 2012) or embedding the test of dissemination strategies early on in the development process.

Yet another approach is involvement of stakeholders and end users as research team members in early phases. Using principles of community-based participatory research, involvement of agency personnel, clinicians, or community members may facilitate development of interventions that are more responsive and implementation ready. Nevertheless, strategies to shorten the developmental trajectory and enhance implementation need to be evaluated to establish evidence that they indeed improve knowledge transfer.

Conclusion

Introducing a new behavioral intervention is an important and exciting endeavor but is not without challenges. It takes many years, from discovery to efficacy testing and evaluation of implementation, for interventions to be fully developed and subsequently integrated in practice. Introducing a new behavioral intervention has typically followed a traditional medical and pharmacological linear pipeline. This pathway must, however, be tailored to the behavioral intervention context and accelerated. New hybrid models that blend test phases and involve a team approach consisting of stakeholders and end users up front in developing and testing interventions may shorten the timeframe and enhance adoption. Knowledge of setting exigencies and implementation challenges may inform intervention protocol development and facilitate more rapid and efficient translation into practice. Integrating cost analyses early on and using hybrid models also hold promise for closing the research–practice gap.

An iterative developmental model for building evidence for novel behavioral interventions and training the next generation in such approaches is critical. Although interventions to improve the public’s health are complex and funding lags, introducing new interventions remains a critical and most worthy pursuit.