Abstract
The manualization of a complex occupational therapy intervention is a crucial step in ensuring treatment fidelity for both clinical application and research purposes. Toward the latter end, intervention manuals are essential for ensuring trustworthiness and replicability of randomized controlled trials that aim to provide evidence of the effectiveness of occupational therapy. In this article, we review the literature on the process of intervention manualization. We then illustrate the prescribed steps through our experience in implementing the University of Southern California/Rancho Los Amigos National Rehabilitation Center’s collaborative Pressure Ulcer Prevention Project. In this research program, qualitative research provided the initial foundation for manualization of a multifaceted occupational therapy intervention designed to reduce the incidence of medically serious pressure ulcers in adults with spinal cord injury.
Keywords
As occupational therapy confronts the challenge of providing evidence-based practice, the need to design empirically based interventions capable of withstanding scientific scrutiny is growing. Manualization of an intervention is a key step in conducting successful evaluation studies, including randomized controlled trials (RCTs), because it enables the research team to monitor treatment fidelity, defined as the extent to which the intervention as actually delivered adheres to the program described in the research protocol (Lichstein, Riedel, & Grieve, 1994; Moncher & Prinz, 1991). If such fidelity is not established, study results may be ambiguous, incapable of future replication, or difficult to apply in other treatment contexts.
Most psychosocial interventions outside occupational therapy rely on manualization as the key mechanism for ensuring treatment fidelity and guaranteeing success (Chorpita, Taylor, Francis, Moffitt, & Austin, 2004; Lopata, Thomeer, Volker, Nida, & Lee, 2008; Moretti & Obsuth, 2009; Smith et al., 2007). However, the significance of treatment fidelity is not well recognized within the occupational therapy field. For example, a literature review of 34 studies reporting on the effects of sensory integration interventions revealed that 29 of the studies did not include measures of adherence to or quality of the intervention, making it difficult to draw conclusions about the form of the intervention under study (Parham et al., 2007).
The concept of treatment fidelity has evolved and expanded over time. Initially, treatment fidelity was conceptualized simply as treatment integrity and required merely that the treatment be delivered as intended (Moncher & Prinz, 1991). A more recent report from the Treatment Fidelity Workgroup of the National Institutes of Health Behavior Change Consortium recommended five areas in which fidelity could be enhanced during clinical trials: study design, training of providers, delivery of treatment, receipt of treatment, and enactment of treatment skills (Bellg et al., 2004). Specific suggestions for avoiding threats to fidelity include the following: development of a treatment manual that includes information about treatment dose (length and number of contacts) and the specific content of each contact, standardization of therapist training, monitoring of the intervention with fidelity checklists, and inclusion of strategies to measure the client’s comprehension and enactment of the intervention principles addressed (Bellg et al., 2004; Borrelli et al., 2005; Nelson & Mathiowetz, 2004). In this more contemporary framework, the development of a treatment manual is the first step in ensuring fidelity. The process of manualizing a treatment is the focus of this article.
In addition to promoting treatment fidelity, manuals perform other functions in the conduct of both RCTs and clinical practice. In particular, manuals provide structure for an intervention’s delivery, enable the intervention to be delivered consistently by different therapists, facilitate staff training, and allow treatment replication in different contexts (McMurran & Duggan, 2005). Typically, manuals accomplish those aims by describing overarching treatment principles and phases, providing specific session-by-session guidelines for the intervention, specifying strategies for administering intervention content, presenting case studies that illustrate intervention principles, and detailing a plan for training therapists (Carroll & Nuro, 2002; Chambless & Ollendick, 2001; Bellg et al., 2004; Borrelli et al., 2005; Nelson & Mathiowetz, 2004).
Intervention manuals are not widely used in occupational therapy clinical settings. In their place, a variety of intervention guidelines are used, most of which are not sufficiently rigorous to be used in RCTs. Guidelines are like manuals in that they provide an outline of the intervention, identify the goals of the program, and describe procedures to address the goals, but they are less detailed and specific than manuals (Marshall, 2009). Manuals developed as structured protocols in preparation for or as a result of RCTs are less commonly available to practitioners, in part because of the paucity of such research in occupational therapy. Thus, an urgent need exists for occupational therapy researchers and practitioners to understand the role of manualized interventions in RCTs and their subsequent use in clinical practice.
Issues Concerning Manualization of an Intervention
A key challenge in the manualization of an intervention is to provide sufficient structure and uniformity while preserving the flexibility and potential for individualization that typify occupational therapy practice. Accordingly, the ability of manualization to allow for responsiveness to the unfolding life story of clients as a way of prioritizing treatment goals is frequently in doubt. In this article, we address those concerns by describing the advantages and disadvantages of manualization, the process of development and refinement of treatment manuals, and the functions of a feasibility study. Finally, to provide a concrete example, we describe in detail the process of manualization used in developing an intervention designed to prevent pressure ulcers in adults with spinal cord injury (SCI).
Advantages and Disadvantages of Manualization
The advantages and disadvantages of manuals vary according to their purpose. The use of manuals in RCTs is considered essential to the success of the study, but in clinical practice they are viewed as difficult to apply and are less popular. The resistance toward manuals in clinical practice is unproductive, however, because research manuals and clinical practice are mutually reliant. Intervention manuals linked to RCTs offer several advantages, including the promotion of evidence-based practice, the enhancement of treatment integrity, the facilitation of staff training and quality assurance, and the potential for treatment replication (Mann, 2009; McMurran & Duggan, 2005; Wilson, 1998).
In clinical practice, manuals also help clinicians focus on what is important, specify intervention procedures, delineate the theoretical rationale behind treatment, and contribute to the evolution of the intervention by explicating the reasoning process necessary to solve clinical dilemmas (Mann, 2009; McMurran & Duggan, 2005). In the psychosocial literature, studies comparing the effectiveness of manualized versus individualized treatments have revealed that manualized interventions are associated with better treatment outcomes, especially when they are flexible and when the content is easily translated into action (Mann, 2009; Shapiro, Youngstrom, Youngstrom, & Marcinick, 2009; Vande Voort, Svecova, Jacobsen, & Whiteside, 2010).
Several disadvantages of using treatment manuals have been identified, however. For example, manuals typically contain a single approach to intervention, which downplays the eclectic and improvisational nature of clinical practice. In this vein, manuals tend to reduce the opportunities for independent clinical judgment and can emphasize therapeutic technique and structure over process (Beutler, 1999). The use of manuals has also been reported to negatively affect the fluidity of intervention delivery and, hence, the quality of the interpersonal relationship between the client and the intervener (Beutler, 1999), an important factor in the success of interventions (Elvins & Green, 2008). For occupational therapy practitioners, whose interventions are frequently focused on the unfolding narrative of the individual (Mattingly, 1991), this disadvantage can make the use of manuals particularly problematic.
The conflicting strengths and limitations of using manualized interventions have resulted in ambivalence among clinicians toward their use. This topic has been studied intensively in the field of psychotherapy, a discipline that shares with occupational therapy the concern that treatment manuals potentially minimize the role of clinical judgment and professional experience in shaping therapy and in which practitioners have resisted adding structure to client-based, flexible intervention practices (Beutler, 1999, 2002; Chorpita, 2002; Henggeler & Schoenwald, 2002; McMurran & Duggan, 2005; Najavits, Weiss, Shaw, & Dierberger, 2000; Westen, 2002). At the same time, psychotherapists have been found to value treatment manuals’ inclusion of specific techniques, descriptions of frequently encountered problems and possible solutions, clear articulation of the theoretical rationale for a treatment approach, provision of a structured approach that includes session-to-session plans, and review of empirical support for a treatment approach (Najavits et al., 2000). To a large degree, these mixed perspectives may also be present among occupational therapy practitioners.
The demand for scientific legitimacy requires researchers to develop repeatable interventions. Given the pros and cons of intervention manualization and clinicians’ ambivalence toward their use, the optimal solution may be to create manuals that, although structured, allow for individualization and for responsiveness to ever-changing life circumstances and narratively described client experiences.
Development and Refinement of Treatment Manuals
Several models that depict the development of treatment manuals have been described (e.g., Carroll & Nuro, 2002; Schnyer & Allen, 2002). The models differ from one another in the number of steps of manual evolution they identify and the items they list for inclusion. The process of developing a treatment manual for research purposes has been described from at least two different perspectives. First, Schnyer and Allen (2002) described the manualization process as a two-step process, beginning with the identification of a conceptual model and then moving to the development of a structure to enhance treatment fidelity. The establishment of a conceptual model requires taking the needs of the discipline and the populations under study into consideration. The conceptual model is based on a survey of the literature, consultation with a panel of experts, and a review of existing treatment protocols. After establishing the conceptual model, the next step is to develop a structure for maintaining fidelity. Fidelity structures must address the specifications of the research project, incorporate a review of previous research to identify factors that could compromise adherence, synthesize the results of prior case studies, specify evaluation tools, and attend to specific clinical issues that influence intervention delivery (Schnyer & Allen, 2002).
A second model, developed by Carroll and Nuro (2002), focuses on the connection between treatment manuals developed as part of a research project and clinicians’ subsequent application of the intervention. In this model, treatment manuals evolve through three stages. Stage 1 manuals are developed for feasibility and pilot studies. These preliminary manuals specify the intervention for initial evaluation, describe the problem to be addressed, indicate the intervention format and session content, explicate treatment goals, and note similarities and differences with other approaches. Stage 2 manuals are then developed to evaluate the efficacy of an intervention in an RCT. In addition to the content developed in Stage 1, such manuals include specific guidelines for managing clinical issues; a plan for training therapists; and attention to other aspects of treatment, such as guidelines for developing client–practitioner relationships. Finally, Stage 3 manuals are produced for generalized use by clinicians in the field and, relative to the previous two stages, add specifications for treating various types of clients so that the program can be applied to diverse populations in multiple settings. For our study, we chose to follow Carroll and Nuro’s model of manual construction because it provides a comprehensive sequence of manual development that starts before its use in an RCT and concludes in the application of the manual to clinical practice.
Functions of a Feasibility Study
Researchers frequently evaluate the soundness of a manualized intervention by conducting a feasibility pilot study before undertaking a full-scale RCT. Feasibility pilot studies are necessary when a planned RCT requires testing of a new intervention or procedure (Grady & Hulley, 2007; van Teijlingen & Hundley, 2002). In such studies, scaled-down versions of the proposed intervention as described in the manual are delivered to a small sample of recipients to determine the viability of intervention delivery. Other purposes of feasibility studies include assessing the success of the participant recruitment process, identifying unanticipated logistical problems, uncovering local politics that may determine the success of the intervention, and assessing costs. Even when the intervention is not novel, feasibility pilot studies can provide useful information about how the treatment plan and other aspects of the experimental protocol will play out in the actual research setting (Grady & Hulley, 2007).
Process of Manualizing the Lifestyle Redesign® Intervention for Pressure Ulcer Prevention
The remainder of this article addresses the process of manualization as it was applied in the University of Southern California (USC)/Rancho Los Amigos National Rehabilitation Center (RLANRC) collaborative Pressure Ulcer Prevention Project (PUPP) for people with SCI. In this research program, which spanned 7 years and was funded by the National Institute on Disability and Rehabilitation Research and the National Center for Medical Rehabilitation Research, an intervention called Lifestyle Redesign Pressure Ulcer Prevention (LR–PUP) was derived from a qualitative database and subsequently manualized. This intervention is now being investigated through a large-scale RCT for its efficacy and cost-effectiveness in reducing the incidence of medically serious pressure ulcers in adults with SCI.
The steps taken to manualize LR–PUP and test its efficacy in an RCT followed a broad-based translational research blueprint used at USC for developing, delivering, testing the effectiveness of, and disseminating innovative occupational therapy interventions (Clark & Lawlor, 2009; Clark et al., 1997; Jackson, Carlson, Mandel, Zemke, & Clark, 1998). Figure 1 depicts how the blueprint was used for the manualization and effectiveness testing of the LR–PUP. First, a 3-yr ethnographic (qualitative) study of 20 adults with SCI and a history of recurring pressure ulcers was undertaken. Through in-depth interviews and participant observations, detailed information was gathered on the everyday life circumstances that contribute to the formation of pressure ulcers in adults with SCI (Clark et al., 2006; Jackson et al., 2010). In addition to generating academic publications (Clark et al., 2006; Fogelberg, Atkins, Blanche, Carlson, & Clark, 2009; Jackson et al., 2010; Seip, Carlson, Jackson, & Clark, 2010), this qualitative study yielded several intervention-related products, including a Stage 1 intervention manual (USC/RLANRC PUPP, 2005), a manual for rehabilitation professionals (USC/RLANRC PUPP, 2006b), and an online consumer manual (USC/RLANRC PUPP, 2006a).
Conceptual map of the PUPP Lifestyle Redesign Research Program.
As depicted in the center of Figure 1, these products were used to guide the design and implementation of a feasibility study that pilot tested the utility and soundness of the intervention. The feasibility study, in turn, led to an expanded literature search, the development of the Stage 2 LR–PUP intervention manual, a corresponding therapist intervention training program, and the construction of a fidelity scale; the manual, training program, and fidelity scale are now being used in the ongoing PUPP RCT. Figure 2 depicts the chronological relationship between the steps of the translational science process and the stages of manual development. The Stage 2 manual is currently being used in the ongoing PUPP RCT. After completion of the RCT, a Stage 3 manual will be produced for use in clinical settings.
Development of manuals in the PUPP Lifestyle Redesign Research Program.
Development of the Stage 1 Manual
The Stage 1 manual (USC/RLANRC PUPP, 2005) was constructed on the basis of two core sources. First, it broadly followed the general treatment approach that had been successfully applied in the USC Well Elderly Study (Clark et al., 1997) to promote positive health outcomes in older adults. The basic principles of that Lifestyle Redesign program included eight elements that were embedded in the LR–PUP intervention: (1) significance of the client–practitioner relationship; (2) client centeredness; (3) emphasis on social support; (4) application of health-related knowledge; (5) use of resources; (6) focus on daily life activities in multiple settings; (7) attention to existing, anticipated, or unanticipated life circumstances that influence risk; and (8) individualization (Clark, 1993; Clark et al., 1997, 2001; Jackson et al., 1998; Mandel, Jackson, & Clark, 1999).
Second, the Stage 1 manual incorporated provisional topics for emphasis that had been identified during the PUPP ethnographic study. Each topic described a factor (e.g., smoking, attendant care, self-advocacy) that was found to affect the participants’ pressure ulcer risk either directly or indirectly. Accordingly, the Stage 1 manual was subdivided into the following 14 units: (1) pressure ulcer knowledge, (2) self-advocacy, (3) attendant care, (4) changing body, (5) environment, (6) adaptive equipment, (7) habits and routines, (8) chronic pain, (9) participation and activity, (10) depression and other mental health issues, (11) social support, (12) transportation, (13) spirituality, and (14) wrap-up session. Each unit provided a description of the topic, noted suggested treatment activities, provided tips for therapists, and listed additional resources for both interveners and participants. An occupational therapy doctoral student (Mary Kay Wolfe) compiled the Stage 1 LR–PUP manual, and it was subsequently tested for feasibility in a pilot study.
The PUPP feasibility study, conducted in preparation for the PUPP RCT, had four aims: (1) to provide preliminary information about issues surrounding RCT methodology with the targeted population (e.g., strategies for recruitment), (2) to determine the viability of the basic intervention design, (3) to refine the initial Stage 1 intervention manual, and (4) to pinpoint the steps required to maintain the fidelity of the intervention. For the feasibility study, 6 participants (5 men and 1 woman) were recruited from a surgical unit specializing in the treatment of pressure ulcers.
The findings of the feasibility study suggested that the intervention was viable and potentially helpful in preventing pressure ulcers. In addition, the findings indicated that the intervention could be administered to clients of different racial and ethnic backgrounds and that principles of Lifestyle Redesign were readily translatable into the individualized treatment session format. In its existing form, however, the manual was unwieldy and cumbersome to use. The modules in the Stage 1 manual did not provide enough flexibility for interveners, and the manual did not provide a detailed structure for each therapeutic encounter. Because the manual failed to provide sufficient flexibility and structure, the interveners felt compelled to choose between providing a mechanized intervention and relying on their own clinical background to establish a therapeutic relationship without following the units described in the manual. On the basis of these results, the content was reorganized into six major units for the Stage 2 manual.
Development of the Stage 2 Manual
The Stage 2 manual was completed before implementing the PUPP RCT, and it is currently being used to guide intervention delivery in the trial. This revised version incorporates more refined descriptions and details of both the content and the process dimensions of the intervention.
Content.
The Stage 2 manual improved on the Stage 1 manual by incorporating modifications that were based on the most current literature on risk factors and proximal causes of pressure ulcer development (Clark et al., 2006; Rodriguez & Garber, 1994) and additional analyses of the data that had been generated during the PUPP ethnographic study. The latter analyses resulted in the development of a series of models depicting the process through which various risk factors interacted in complex ways in the context of clients’ everyday lives to result in pressure ulcers (Clark et al., 2006) and the identification of seven overarching principles that accounted for pressure ulcer development in people with SCI (Jackson et al., 2010). The models and principles were incorporated into the manual’s modularized units, and they also spurred the generation of new worksheets and treatment activities to be performed during the intervention sessions. For example, after the presentation of the theoretical model emphasizing consideration of the balance between buffers and liabilities (Clark et al., 2006), detailed clinical reasoning worksheets were included to facilitate therapeutic problem solving related to this concern.
Process.
In contrast to the Stage 1 manual, the Stage 2 manual combined two intervention approaches and was developed by a team of researchers and clinicians (Erna Blanche, Donald Fogelberg, Faryl Reingold, Florence Clark, and Mike Carlson). The first approach, which was included in the Stage 1 manual, entailed comprehensive principles of Lifestyle Redesign applied to pressure ulcer prevention for adults with SCI (Clark et al., 1997, 2006; Jackson et al., 2010). In addition, the techniques of motivational interviewing, a client–practitioner collaborative approach responsive to the participant’s stage of readiness to change (Rollnick, Mason, & Butler, 1999; Rollnick, Miller, & Butler, 2008), were enfolded into the intervention guidelines in the Stage 2 manual. The inclusion of a team of researchers and clinicians in the development of the Stage 2 manual helped guarantee that the manual answered to clinicians’ need for flexibility as well as individualization.
To address these concerns, the manual was refined so that the order of the modules could be altered on a case-by-case basis, each module included fixed and variable (optional, to be addressed only if relevant to a particular client) topics, and the use of the manual could be shaped by the content of the participant’s unfolding life story or narrative experience as it pertained to pressure ulcer risk. For example, for one participant the equipment module might consist of identifying funding sources such as private grants and foundations to purchase appropriate equipment, whereas for another participant, it might involve exploring the participant’s reasons for not using his or her currently owned equipment.
In the final rendition of the Stage 2 manual, modules were expanded and reorganized into six thematic units to be delivered during the first 6 mo of the 1-yr intervention. In addition, a tapering phase during which intervener support is gradually reduced was specified for the intervention’s final 6 mo. The six main topics addressed in the manual are (1) understanding pressure ulcer risk, (2) taking charge (advocacy), (3) assessing the physical environment, (4) nurturing social networks and meaningful relationships, (5) promoting happiness and personal well-being, and (6) planning the future. Each unit is divided into four in-person or phone contacts, and each contact includes both fixed and variable themes. With the exception of the first and the last modules, the topical content is flexible in that it can be used in any order according to each participant’s needs and unfolding life story in relation to threats to skin integrity. The manual specifies the components of each contact, including an introduction, an outline of goals, suggested activities, a list of resources for the intervener, articulation of areas for further exploration, references, and forms that can be used to expedite problem solving in response to emergent threats and concerns. Table 1 outlines the differences between the Stage 1 and Stage 2 manuals.
Comparison of Stage 1 and 2 Manuals for the Pressure Ulcer Prevention Project
Note. RCT = randomized controlled trial.
These components were outlined in Carroll and Nuro (2002).
The Stage 2 manual also contains elements to counteract three specific disadvantages of manualization described in the literature: (1) use of a single approach that does not reflect the eclectic and improvisational nature of clinical practice, (2) diminished intervention fluidity, and (3) a reduction in the effectiveness and art of therapy for experienced practitioners (Beutler, 1999; McMurran & Duggan, 2005; Westen, 2002). For example, it redresses the concern with balancing structure, flexibility, and individualization by incorporating more than one intervention approach (i.e., both Lifestyle Redesign and motivational interviewing), encouraging clinicians to rely on their own clinical reasoning when flexibility is required, directing interveners to use overarching theoretical principles and models laid out in the manual, and allowing clinicians to tailor the content of modules in response to the emerging threats to skin integrity.
Illustrating the flexibility of the treatment approach, the LR–PUP manual can be used to address the risk-relevant events that unfold in the client’s life. For example, 1 participant had recently decided to move temporarily to another state. His traveling arrangements required him to lie down in the back of a van for several hours, a situation that heightened his risk for the formation of pressure ulcers. Although the LR–PUP manual does not explicitly include specific content on how to avoid ulcers during the process of moving, it does contain information on how to prevent ulcer development during short car rides. In this case, the intervener used the opportunity to cover the relevant manual content and modified it in response to this client’s unique life situation. Beyond this application, the intervener also had to rework manual content so that it could be effectively delivered by phone while the participant was away.
As the PUPP RCT progresses, additional revisions will be recommended that will be incorporated into a Stage 3 manual before its dissemination to the community for widespread use. We anticipate that the manual will be sufficiently refined to enhance treatment integrity, facilitate staff training, promote quality assurance, and increase the potential for replication of the intervention (McMurran & Duggan, 2005; Wilson, 1998).
Conclusion
The current emphasis on evidence-based practice, in conjunction with the requirements of RCTs, creates the need for manualized interventions as part of the process of establishing intervention fidelity. To date, however, few manualized occupational therapy interventions exist (Nelson & Mathiowetz, 2004). One reason for this paucity is the continuing challenge for occupational therapy researchers to reconcile the client-centered and individualized nature of practice with the need to manualize interventions.
Our experience in the PUPP research program has revealed that occupational therapy treatment manuals need not be rigid or constrain therapists’ clinical reasoning; the second version of the LR–PUPP manual allows for individualization and flexibility within the context of an overarching structure, explicit guiding principles, and theoretical models (Vaishampayan, Clark, Carlson, & Blanche, 2010). In its second stage, the LR–PUP manual has the following desirable components: It combines two primary theoretical models, Lifestyle Redesign and motivational interviewing; its structure provides a menu from which interveners can select topics according to the client’s needs; it mandates common training and weekly supervision sessions to share successful strategies; it allows for the intervention to be anchored in the ongoing narrative experience of clients; and its overarching principles and content provide sufficient structure to enable replication for an RCT.
Footnotes
Acknowledgments
We gratefully acknowledge the support of the National Institute of Disability and Rehabilitation Research, University of Southern California Department of Education (H133G000062). The project was also supported by the Eunice Kennedy Shriver National Institute of Child Health and Human Development (NICHHD; R01HD05267). The content is solely the responsibility of the authors and does not necessarily represent the official views of the Eunice Kennedy Shriver NICHHD or the National Institutes of Health. We also thank Mary Kay Wolf and Faryl Saliman Rheingold for their work on the intervention manual. We acknowledge the contributions of the participants who have been enrolled in the PUPP studies and the support and effort of numerous other people, especially those on the LR–PUPP intervention team.