Abstract
Many occupational therapy practitioners consider evidence-based practice (EBP) to be the means by which occupational therapy can prove the validity of its services and thus support the legitimacy of our profession. The unquestioned acceptance of EBP as the way to establish credibility concerns me; unchallenged acceptance of any idea concerns me. Do practitioners accept EBP as the paradigm for guiding occupational therapy practice and research solely because it is presented as what we must do? I believe that practitioners must examine the implications for our profession of accepting EBP without question. In this article, I review EBP, present criticisms and concerns voiced by other professions and, finally, examine the implications of adopting an EBP perspective that replaces theory-directed practice.
The term evidence-based medicine first appeared in the published literature in 1991 (Montori & Guyatt, 2008). Health professions rapidly accepted the term and the ideas proposed by evidence-based medicine, labeling it evidence-based practice (EBP). Although they used the term EBP, they adopted evidence-based medicine’s definition and perspectives. Evidence-based medicine is often defined as “the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients” (Sackett, Rosenberg, Gray, Haynes, & Richardson, 1996, p. 71). Other definitions include the importance of using sound clinical judgment and addressing client concerns, values, and life situations (Gambrill, 2003; Melnyk & Fineout-Overholt, 2011). However, in practice, “evidence based” almost always refers to the use of the best available evidence from research.
Evidence-based medicine and practice emerged from a society that accepts the philosophical and epistemological perspective of positivism. Positivists propose that science is the only way to understand our world, devaluing qualitative and subjective explanations of events. The move toward evidence-based medicine occurred in the post–World War II era when, according to Kernick (1998),
As the British Medical Journal was dropped onto their doormats, little did doctors realise on that autumnal Saturday morning in 1948 that they were witnessing a momentous event. The first medical randomised control trial had appeared on the world scene, and a new era of medicine had begun. The experiment was quite straightforward. Patients with tuberculosis, unaware they were taking part in a clinical study, were randomised to receive streptomycin or nothing. With a plunge of a syringe, 3000 years of medical treatment based on experience and perceived effectiveness were at an end. (p. 1824)
Society’s growing acceptance of science has had profound effects on what professionals and consumers value and deem important. Nevertheless, despite this important moment in 1948, no organized movement or approach to the collection of medical evidence appeared in its aftermath. During the 1950s, Archibald Cochrane, a medical doctor, advocated for research to examine the efficacy of medical intervention. He had spent 4 years as a prisoner of war in a German camp treating 20,000 prisoners with diarrhea and other ailments with only aspirin and other basic remedies. During this time, only four prisoners died, three from gunshot wounds (Jensen, 2004). Cochrane decided that medical treatment was relatively unimportant and became committed to examining its effectiveness (Hill, 2000). In 1972, he convincingly argued in “Effectiveness and Efficiency” that the evidence supporting common health care interventions lacked reliability. In 1993, he cofounded the Cochrane Collaboration to promote evidence-based medicine (Fitzpatrick, 2000).
Lloyd-Smith (1997) introduced EBP to the profession in the British Journal of Occupational Therapy. However, it was not until Holm’s (2000) Eleanor Clarke Slagle Lecture that the idea of evidence-based occupational therapy became popular. Within a few years, the notion that all occupational therapy practice and research needed to be evidence based became institutionalized through the American Journal of Occupational Therapy, accreditation standards, guidelines for practice, and even the Occupational Therapy Code of Ethics and Ethics Standards (2010) (American Occupational Therapy Association, 2010). Holm asserted that practitioners needed research to ensure occupational therapy’s continued existence. She also suggested the adoption of a hierarchy of evidence to evaluate the quality of research. In her model, as in evidence-based medicine, systematic reviews based on randomized controlled trials (RCTs) constitute the most valid research.
Many occupational therapy scholars were quick to support Holm’s (2000) stance, promoting the adoption of EBP (Dirette, Rozich, & Viau, 2009; Glegg & Holsti, 2010; Ottenbacher, Tickle-Degnen, & Hasselkus, 2002; Valdes, 2010). Notably, authors provided compelling and clear arguments about why practitioners must provide EBP, and none questioned EBP’s assumptions. However, outside of occupational therapy, Kristiansen and Mooney (2004) wrote, “The question is not whether EBM is useful but whether all aspects of EBM and the EBM movement should be embraced uncritically” (p. 19). They advocated for cautious acceptance of the evidence-based paradigm, which requires critical assessment of its value to the profession of occupational therapy.
Concerns for Occupational Therapy
Evidence-based medicine raises three concerns. First, should occupational therapy adopt a hierarchy of evidence from medicine? Second, what are the consequences of assuming that RCTs provide the best, if not the only, evidence that establishes credibility? Third, and most important, should one assume that internal validity is more important than external validity when judging a study’s value?
Hierarchies of Evidence
Evidence-based specialists have developed rankings for evaluating research to assess the relevance of their findings. These hierarchies provide criteria for assessing the quality of research evidence. They also provide a structure for completing a systematic review to facilitate judgments about the value of the research findings. Hierarchies of evidence from medicine assume that experimental research is superior to all other research methods. However, no proof that one research design is better than another exists, thus calling into question the validity of any hierarchy of evidence that accepts the superiority of the RCT (Goldenberg, 2009; Worrall, 2007).
Tomlin and Borgetto (2011) proposed an alternative hierarchy based on their perspective that occupational therapy needs its own model for evaluating research. They proposed a research pyramid that addresses some of the concerns discussed in this article. The base of the pyramid is description research and the three sides are experimental research, outcomes research, and qualitative research (Figure 1). Each makes a unique contribution to evidence-based occupational therapy. However, most practitioners are only familiar with the hierarchy of evidence, similar to the hierarchy used in evidence-based medicine, proposed by Holm (2000).
The research pyramid.
In the hierarchies of Holm (2000) and Tomlin and Borgetto (2011), RCTs are rated as superior. In an RCT, the researcher randomly assigns participants to groups, a process that reduces allocation bias by creating—in theory—equivalent groups. For most hierarchies, the RCT is the gold standard for evidence. In fact, for some researchers, RCTs are the only method for establishing evidence (Sackett, Straus, Richardson, Rosenberg, & Haynes, 2000).
Stating that RCTs are superior is not proof of their superiority (Kristiansen & Mooney, 2004; Worrall, 2007). Indeed, the acceptance of RCTs as the gold standard raises three concerns. First, RCTs are not always feasible. For example, many therapeutic interventions cannot be randomized; too few patients may be available or ethical reasons may argue against the use of an RCT (Armstrong, 2007; Ashcroft, 2004). Second, RCTs are methodologically associated with a particular theory of statistical inference (Ashcroft, 2004); not everyone accepts this statistical inference. That is, “statistical information is always and only about rational subjective degrees of belief, rather than measures of objective probability; and indeed that the notion of objective probability as tied to the RCT is meaningless” (Ashcroft, 2004, p. 134). Third, and most relevant to occupational therapy, findings from RCTs do not provide reliable and strong evidence for clinical practice or real-life decisions. Instead, RCTs provide reliable and strong evidence for scientific questions (Pedersen, 2004).
Systematic reviews.
Systematic reviews are restricted reviews of published research that examine evidence to answer a specific question. The person conducting a systematic review determines population, participant characteristics, intervention, comparison measures, and outcome of interest (Higgins & Green, 2009; Russell et al., 2009). He or she then scrutinizes each study to ensure that it meets the accepted criteria for inclusion and selects appropriate studies. The reviewer analyzes each study and synthesizes the findings.
When conducting a systematic review, RCTs are rated at the highest level. Systematic reviews are important and provide valuable information, yet they have limitations. First, any systematic review is likely to miss relevant studies either because they were not published or because they were not published in English (Kristiansen & Mooney, 2004). Second, separate systematic reviews on the same question can have different conclusions (Shrier et al., 2008). Two people may use different criteria for selecting studies; thus, the conclusions drawn from their systematic reviews may be based on different sets of research studies. Even when two separate researchers use the same criteria, resulting in the same set of research studies, they may come to different conclusions on the basis of their individual analyses, thereby raising questions about the validity of systematic reviews (Hopayian, 2001). Third, some scholars (Mosey, 1996) believe that basic research tests only specific hypotheses from the guidelines for intervention and does not test the effects of the whole intervention. Rejection of a given hypothesis is only that; it is not a rejection of the entire guidelines for intervention.
Systematic reviews provide limited guidance about the effectiveness of an intervention. Some only establish that the evidence available to answer the question is insufficient (Petticrew, 2003). None provide the complete information necessary to treat an individual client (Dickinson, 1998). When a systematic review does not support an intervention, it may be because of the absence of evidence rather than the lack of effect. Nevertheless, systematic reviews can help clinicians make decisions. They provide valuable information about research, summarized in a systematic and organized manner.
Importance of external validity.
When examining studies to evaluate their evidence, practitioners ask whether the observed change is the result of intervention. The random assignment of research participants ensures internal validity but does not address external validity.
Internal validity establishes causal relationships by estimating the accuracy of inferences. However, although internal validity allows practitioners to conclude that the intervention contributed to the change, they cannot rule out the contributions of other factors. Moreover, hierarchies of evidence from medicine do not address external validity (the generalization of findings to other people or other situations), which is an important concern for practitioners because it relates to how the findings reflect the real world. Practitioners ask, “Would this intervention work with people from different cultures or in different situations?” We also recognize that an intervention might logically work under one condition but not another or in one situation and not another. External validity includes attention to context. According to the Occupational Therapy Practice Framework: Domain and Process (2nd ed.; AOTA, 2008), context is part of the profession’s domain of concern: the support of a person’s health and ability to participate in life activities.
EBP, however, focuses on objective truth; it does not concern itself with questions about time, space, and context (Hansen, 2004). Reflecting on nursing, Hansen (2004) wrote that
patients and nurses are living, thinking, feeling and acting individuals, relating to one another in different ways and situations, where communication in a broad sense always takes place in time and space. The trusted notion that truth corresponds to objective reality does not take into account the fact that acting is contextual, that interpretation always is at stake, and that it is difficult (impossible) to produce generalizability from one group of nurses or patients to another (external validity). (p. 38)
The therapeutic relationship, clearly a key element of practice, is not reflected or valued in EBP because of its concern for objective truth.
Evidence-Based Practice and Occupational Therapy
Occupational therapy has struggled in establishing its own identity, distinct from medicine. However, our adoption of medicine’s criteria for EBP has again strengthened its connections to medicine. When we try to find evidence to support evidence-based occupational therapy, we often cannot find relevant studies. And, when we do, the results are usually inconclusive or irrelevant (Straus & McAlister, 2000).
Consider the following scenario: A therapist receives a referral for a 60-yr-old woman with poststroke left hemiplegia. After evaluation, the therapist decides to use neurodevelopmental treatment (NDT) because the client wants to be able to use both hands to complete self-care and recreational activities. Committed to practicing evidence-based occupational therapy, the therapist examines studies related to NDT from Medline and CINAHL, searching on the key term neurodevelopmental treatment from 2007 through January 19, 2011. After reviewing abstracts, the therapist identifies six relevant articles. The most relevant is a systematic review by Kollen et al. (2009) examining 16 studies involving 813 stroke patients who received intervention based on NDT. Kollen et al. concluded, “There was no evidence of superiority of Bobath on sensorimotor control of upper and lower limb, dexterity, mobility, activities of daily living, health-related quality of life, and cost-effectiveness. Only limited evidence was found for balance control in favor of Bobath” (p. e89).
Should the therapist conclude that the evidence does not support the use of NDT? If so, she must find another intervention approach. What is the possibility that the evidence supporting that alternative approach would be any stronger? Or, the therapist could decide that although the evidence does not yet support the superiority of NDT in relation to other approaches, it is at least as effective as others. Is this rational? If she decides to use NDT despite the lack of evidence, could she still argue that she is providing EBP because she has already concluded that there is nothing proven to work better? If she does use NDT, is she providing EBP?
In reality, little scientific evidence exists to support the unqualified effectiveness of many interventions. This fact does not mean, however, that no evidence supports their effectiveness. Empirical evidence can come from other sources and from other forms of scientific inquiry (Earle-Foley, 2011). I propose that we look at EBP as only one source of information about effectiveness. Although evidence-based findings provide some important information about the effectiveness of some interventions, they do not address the abundance of significant concerns occupational therapy practitioners have.
Accepting EBP as occupational therapy’s primary perspective only reestablishes and strengthens our connection to medicine and the medical model. Medicine’s domain of concern includes medications or medical interventions that can be studied easily using RCTs. Occupational therapy’s domain of concern includes human occupations, work, self-care, and leisure in context. This domain is difficult, if not impossible, to examine using RCTs. I propose that we look for alternative models, such as Tomlin and Borgetto’s (2011) research pyramid for evidence-based occupational therapy. If we choose to judge our evidence for practice on the basis of a hierarchy established by medicine, we will never achieve a credible level of societal acceptance.
What does all this mean for occupational therapy? The current acceptance of EBP in occupational therapy is grounded in a valid ethical obligation that practitioners do no harm but rather do the best for their clients (Ashcroft, 2004). It has its limits, however, and critics are demanding that EBP supporters provide evidence that it improves patient outcomes (Straus & McAlister, 2000). I support this directive.
In addition, we have to be careful not to label current practices as EBP without proper evidence to protect our professional turf (Gambrill, 2003). The definition of evidence-based occupational therapy must be consistent with occupational therapy’s goals, values, and domain of concern. We must develop a model for evaluating research that reflects our profession. Nursing and social work are already doing this—those fields are carefully considering what evidence supports their intervention efficacy. In occupational therapy, we might, for example, put qualitative inquiry at a higher level because its findings get closer to understanding the lived experience of occupational therapy clients. We might also consider using the term evidence-informed practice to describe our evidence-based approach (Glasziou, 2005). Evidence-informed practice would synthesize all the evidence available, including the theoretical rationale for the intervention, often forgotten in EBP.
Finally, occupational therapy must not allow EBP to move practitioners away from theoretically based practice. Although we need to establish the scientific basis of our practices, we must also develop sound theoretical guidelines for intervention and establish their efficacy. Then, we can examine their efficacy with complex research designs. Occupational therapy researchers must evaluate and report outcomes with individual clients on the basis of clearly identified treatment objectives. When we do not get the expected outcomes, we should look for and develop alternative, theoretically based guidelines. We need to develop theoretically based guidelines for intervention, and we need to be able to study the contextual aspects of intervention, which are important for the growth of our profession and our practice knowledge. The profession must resist the attractiveness of EBP’s reductionism, which explains the effects of intervention on the basis of a few key variables (Mooney, 2004).
Directions for Action
In conclusion, I propose that practitioners are obliged to
Understand the strengths and limitations of EBP to use it to guide practice effectively. Practitioners need to acknowledge EBP as a tool we can use in our clinical decision making; it is not the principal way to identify useful and meaningful interventions.
Use the findings of quantitative research as one form of evidence to inform practice and acknowledge the importance of qualitative research, clinical knowledge, and the uniqueness of each client.
Continue to develop a broader and more inclusive system for the evaluation and use of evidence, such as a hierarchy of evidence that reflects the profession, building on the work of Tomlin and Borgetto (2011) or an alternative model.
Learn and understand the language of EBP, such as the number needed to treat (NNT, Armstrong, 2007; Cook & Sackett, 1995), a statistic that clearly informs us about the real effect of an intervention. The NNT identifies the number of clients one needs to treat for one beneficial outcome (Walter, 2001). Using this measure of statistical and clinical significance, we can share our clinical uncertainty honestly with clients to support our philosophical belief in client-centered care. Researchers also need to conduct research using samples with the appropriate effect size to determine the real effects of intervention.
Prepare for the inevitable arguments that payers will use to limit payment for services on the basis of the lack of EBP-supported evidence as they attempt to cut costs or justify rationing decisions.
Footnotes
Acknowledgments
This article is based on a Plenary Address titled “Evidence-Based Occupational Therapy: Cult, Fad, or Relevant?” delivered at the New York State Occupational Therapy Association State Conference, October 22, 2011.