Abstract
Those who ignore history are doomed to repeat it
George Santayana
American philosopher and poet, 1863–1952
This is the first paper in a series of five describing the use of evidence to support decisions made in clinical practice. The series covers large elements of Statement 2: The general practice consultation, Statement 3.3: Ethics and values based medicine and Statement 3.5: Evidence-based practice of the GP Curriculum.
Curriculum Box
All GPs should be able to
Ask the ‘right questions’ Find the appropriate literature from the widest available sources Apply rigour in appraising the literature Place the answers in the appropriate context
GPs should have the ability to
Demonstrate that they base their treatment and referral decisions on best available evidence Apply rigour to scientific research to decide whether evidence is applicable to the primary care setting and appropriate to the individual Demonstrate sufficient knowledge of the breadth of scientific evidence in order to provide the best information for the individual and his or her illness Demonstrate understanding that evidence needs to be gathered from the most appropriate, rather than the most readily available source. GPs should be able to determine whether evidence presented to them is sufficient and rigorous enough to be analysed in the context of a patient.
In the 21st century, health care clinicians, managers and patients expect to see the findings of research incorporated into clinical practice, taking into account the needs and wishes of individual patients. In this series, we will examine why that happens — and often does not happen — and what clinicians and managers can do to improve the use of evidence in consultations. Papers 1 and 2 are based on comprehensive literature searches undertaken as part of a programme that started in 2002 by the National Prescribing Centre. These two papers outline the back ground to the science of evidence-based medicine (EBM) and consider the extent to which it informs practice. Paper 3 describes under-recognized but evidence-based pragmatic approaches to enable high-quality research findings to be identified, considered and where appropriate incorporated more often and with less difficulty into routine clinical practice. It also contains details of mater ials for further study for generalists and especially GP registrars. Paper 4 is based on a published systematic review. It describes the characteristics and actions associated with more successful adoption of change and the implications for health care organizations such as hospital trusts, primary care trusts and practice-based commissioning groups and their equivalents in other health systems. The final paper describes a clinician's progress on a journey to meet the real-world challenges of using evidence in medical practice, using a narrative approach.
In 1992 there was one Medline citation for EBM; in 2004 there were more than 13000 (Strauss, 2004). The development of EBM was a response to several factors which shook a confidently held view that modern scientific medicine, as practised at the time, was rational, empirical and founded on a solid research base (Muir Gray, 2001). By at the latest the mid-1980s, it had become clear that there were variations in service delivery and clinical practice which could not be explained by variations in the underlying health of populations. There were gaps in implementation of evidence — interventions that were likely to be beneficial were not being incorporated into clinical practice and also interventions which were of dubious value or even known to be harmful persisted. Need — and demand — for services was increasing at a rate which outstripped the available resources. The emerging discourses of accountability, patient centredness and patient empowerment all required that decision-making processes at both individual and population level needed to be more open and more objective. Finally, there was a growing awareness of the limitations of many of the sources of advice and guidance in which practising clinicians placed their trust.
In less than two decades, the EBM concept and the approaches used within that concept have evolved significantly. The initial emphasis was developing and disseminating techniques for the critical review of published research studies, encouraging clinicians to search for the best available research evidence and employ their new critical review skills to evaluate it and use evidence which was most valid to guide practice. This was in contrast to relying solely on personal clinical experience and pathophysiological rationale supplemented by expert opinion and tradition.
Further developments have included the creation of the Cochrane Collaboration, an international community of practice dedicated to generating and applying best evidence, and the emergence of numerous secondary publications summarizing studies of high methodological quality (Clinical Evidence, ACP Journal Club, etc.). Current emphasis is on integrating clinical expertise (particularly in diagnosis and in performing technical procedures) with research evidence and with patients’ circumstances and preferences to arrive at optimal (evidence-based and patient-centred) decisions. The work of the National Institute for Health and Clinical Excellence (NICE) in England and the Scottish Intercollegiate Guidelines Network has been based on these concepts. The focus of the EBM community has moved from attempting to persuade educators that EBM should be a ‘mainstream’ topic on undergraduate and postgraduate curricula, to how to teach it; and from trying to persuade clinicians that they should practise according to the principles of EBM, to addressing how a busy clinician might find and apply best evidence at the bedside and in the clinic (Strauss and Jones, 2004).
Internationally, billions of pounds are spent on primary research (i.e. original empirical studies) and on secondary research (preparing systematic reviews, meta-analyses and other summaries). The NHS has invested substantial effort and money in delivering clinical guidelines and guidance to the NHS by establishing NICE and has focused clinical activity using National Service Frameworks and other performance management mechanisms. In 1998 one of the most highly respected medical journals in the world could state that ‘the concept of evidence-based medicine has emerged as one of the fundamental elements in Westernstyle clinical medicine’ (Hjelm and Tong, 1998). But over a decade later, research evidence still does not flow into practice like ‘water through a pipe’. At best, implementation of new research is sluggish and patchy. The problems that stimulated the emergence of the EBM movement are still with us.
The authors of this series have conducted several comprehensive literature searches and a systematic review to identify and analyse traditional and contemporary models of ‘getting research into practice’, combining these with an evolving programme of writing and teaching in the USA and UK. We have supplemented the literature searches with ‘snowballing’ (such as pursuing references cited in papers identified in the search) and from our personal collections of papers. In this series of five papers, we also incorporate evidence describing the psychology of decision making, consider ways in which clinicians and patients could access timely, valid and relevant information and describe the approaches and skills required to incorporate evidence from research into patient-centred consultations.
Lessons from history
The medical profession has a long and ignominious history of failing to implement significant findings from research. Probably the longest and most remarkable delay was the failure to adopt the use of lemon juice to prevent scurvy. First demonstrated by James Lancaster in 1601, practice remained unchanged until James Lind repeated the experiment in 1747. The British navy still did not fully implement the evidence until 1795 and not until 1865 in the case of the merchant navy (Mosteller, 1981). More recent examples include the failure to give corticosteroids to pregnant women with premature rupture of membranes to reduce the incidence of hyaline membrane disease in their babies (Donaldson, 1992) and the 13-year delay between the demonstration of effectiveness (by meta-analysis of trials) of thrombolysis in myocardial infarction and its general acceptance into clinical practice (Antman et al., 1992).
In the last few years in the UK, some progress on consistent implementation of evidence in key disease areas has been made. For example, in England and Wales, there were 68 230 fewer coronary heart disease (CHD) deaths in 2000 compared with 1981. However, although more than 90% of patients are now given appropriate drugs after a myocardial infarction, most (58%) of the decrease in CHD mortality was attributable to change in risk factors (especially smoking) rather than medical or surgical treatments (Kelly and Capewell, 2004; Department of Health, 2005). A review of the impact of the NICE programme found progress on implementing its advice on prescribing, but there remained inconsistencies that were most prominent in implementation of guidance on procedures (Sheldon et al., 2004). Despite much excellent work, many gaps between research and clinical practice remain.
Lessons from today — implementation is complex
Three examples of poor implementation of evidence into contemporary therapeutics are given in Box 1. These demonstrate some of the key obstacles to practising EBM when it requires changes to the organization and delivery of services (Greenhalgh et al., 2004).
Firstly, we cannot rely on information alone to produce consistent changes in clinical practice — the pace of change in prescribing practice in response to warnings about the cardiovascular risks of NSAIDs indicates that there are significant barriers. A single exposure to new information, especially if only in written format, is unlikely to produce learning and certainly not behaviour change (Cottrell, 1999). Despite this being well recognized and increasingly acted upon in undergraduate and postgraduate education, new information still tends to land on UK clinicians’ desks or computers in text form and in single bursts — whether as newly published research, NICE guidance, Medicines and Healthcare products Regulatory Agency warnings or Department of Health policy. However, limited information may sometimes paradoxically have an impact far greater than warranted. One example is the effect on rates of immunization with combined measles, mumps and rubella (MMR) vaccine of the publication of a small case series hypothesizing a link between developmental delay, autism and intestinal abnormalities and vaccination with MMR (Elliman and Bedford, 2007).
Secondly, as in the specific example of COX II inhibitors, even when authoritative guidance on new health technologies became available, many clinicians chose not to follow it, preferring to listen to other Siren voices which perhaps appealed to their hopes or allayed their fears.
The way that clinicians, and human beings in general, acquire and use information and knowledge in consultations is discussed further in the second paper in this series. It seems that GPs rarely review the methods and content of trials. Instead they judge the trustworthiness of the source of trial evidence and interpret it within the context of the economic and social factors which impinge on their practice (Kairhurst and Huby, 1998). An ethnographic study in primary care found that clinicians work with ‘Collectively reinforced, internalised tacit guidelines, which were informed by brief reading, but mainly by their interactions with each other and with opinion leaders, patients, and pharmaceutical representatives and by other sources of largely tacit knowledge that built on their early training and their own and their colleagues’ experience. The clinicians, in general, would refine their mindlines by acquiring tacit knowledge from trusted sources, mainly their colleagues, in ways that were mediated by the organisational features of the practice, such as the nature and frequency of meetings, the practice ethos, and its financial and structural features, including the computer system’ (Gabbay and le May, 2004). This is all very well if the ‘mindlines’ reflect best evidence, but if not, they will serve to reinforce outdated or suboptimal practices.
Similar contextual drivers exist in secondary care. Approaches used by a clinician to choose a treatment are usually based on an internal mind map of the evidence and its implications for practice, derived from a variety of information sources involving brief reading and talking to other people (Prosser and Walley, 2006). It is not surprising then that passive, single-shot information provided by remote NHS organizations (which may not necessarily be trusted or may be suspected of having ulterior motives) produces variable implementation of research-based evidence. Most times, the internal mind map is very strong and the external new information is relatively weak.
Thirdly, as with antibiotic prescribing, clinical practice is influenced by factors other than the evidence base or even the health care professional's mind map of what he or she thinks is the evidence base. Health care has to be set in the context of the individual and the locality. Political, economic, social and cultural considerations sometimes outweigh science. Patients’ empowerment is, perhaps belatedly, becoming an important feature in clinical decision making. No treatment can ethically be provided without informed consent and yet traditional professions are still getting to grips with changed status whereby they are not the sole decision makers but rather advisers, and not sole repositories of knowledge but interpreters of information which is held jointly or severally.
Sometimes individuals may obtain their own mind map of new information and respond in what might naively be considered unexpected ways, such as expressing concern on the basis of scientifically flimsy associations between autism and MMR vaccination. Naively, that is, until it is recognized that health care decisions — like many decisions taken in life — are often made on the basis of emotions and not facts (Paling, 2003). Corporate profiteering, immature information systems and the growing gap between rich and poor also complicate the translation of strong evidence into practice (Reilly, 2004).
The challenges in practising EBM
Significant new health technologies — drug therapies, surgical interventions, new diagnostic tests and so on — ought to be subject to a robust technical appraisal that is available at the same time as the technology. But at the time a new health technology becomes available, there is often limited evidence of clinical or cost effectiveness, never mind safety.
Even when high-quality evidence is available and is distilled using high-quality methods into summaries designed for brief reading by health care professionals and lay people (such as NICE guidelines and guidance), there is the constant problem of particularizing evidence from population averages in studies to individual patients, updating and timeliness. Randomized controlled trials, while inherently designed to minimize bias between the groups, have important limitations — they may recruit highly selected patients, offer single interventions, use comparators that are not the current standard, etc. And of course, even when we have high-quality evidence, our patient may be intolerant or have a drug interaction or simply not ‘get on with’ the intervention deemed best practice.
The implementation gap — three contemporary examples
When they first became available, COX II-selective NSAIDs were heavily marketed by manufacturers, enthusiastically supported by a number of specialists and received extensive coverage in the lay media, medical magazines and journals. Dyspepsia as a side effect of NSAIDs is a common experience. COX II-selective NSAIDs were heralded as promising a freedom from the much rarer, but potentially life threatening, upper gastro-intestinal (GI) ulceration and bleeding that NSAIDs can also cause. In the summer of 2001, NICE guidance stated that the use of COX II-selective inhibitors was ‘not recommended for routine use in patients with rheumatoid arthritis (RA) or osteoarthritis (OA)’ (NICE, 2001). In 2002, a National Prescribing Centre MeReC Briefing said The GI safety of rofecoxib and celecoxib has been assessed in large clinical outcome trials which, on first analysis, show benefits over nonselective NSAIDs in the incidence of serious upper GI complications. However, longer-term GI data from the celecoxib study (CLASS) and cardiovascular adverse event data from the rofecoxib study (VIGOR) have questioned the risk/benefit profile of these new drugs and, until they are better understood, it seems sensible not to use them routinely in large numbers of people.’ Prescribing data for England show that, despite this advice, prescribing of celecoxib and rofecoxib doubled from around half a million items per quarter in 2001 to over a million items per quarter by the autumn of 2004 (NHSBSA(PPD), personal communication) when concerns about increased cardiovascular risk with rofecoxib led to its withdrawal from the market by the manufacturer. If we assume 4 million prescriptions a year, each for 28 days duration, then there may have been a third of a million people taking a COX II in 2003–04. The increased risk of myocardial infarction with COX II inhibitors has been estimated at three in 1000 patients per year (MHRA, 2006) suggesting that approximately 1000 premature or avoidable myocardial infarctions per year could have been associated with COX II inhibitor prescribing in England at 2003–04 levels.
In October 2006 the Commission on Human Medicines (CHM) extended the cardiovascular safety warnings to additional NSAIDs. They stated that long-term, high-dose, traditional NSAIDs may be associated with a small increased risk of thrombotic events. Further, diclofenac (particularly at the higher dose of 150 mg daily) may have a small thrombotic risk (including risk of myocardial infarction) similar to that of etoricoxib and possibly other coxibs. In addition, there may be a small thrombotic risk for ibuprofen at high doses (e.g. 2400 mg/day) but at low doses, epidemiological data do not suggest an increased risk of myocardial infarction. Naproxen is associated with a lower thrombotic risk than COX IIs — epidemiological data do not suggest an increased risk of myocardial infarction. However, some risk cannot be excluded (CHM, 2006; Jick et al., 2007). Prescribing volumes of traditional NSAIDs did not change in the UK as a result of this advice (NHSBSA(PPD), personal communication). The National Prescribing Centre discussed the safety issues of NSAIDs again in MeReC Extra 30 (National Prescribing Centre, 2007) which was published in November 2007 and supported dissemination of the findings using its trainer and associate networks. Only then has there been change, but this is limited, is not geographically uniform and there remains a far greater volume of prescribing of diclofenac than ibuprofen and naproxen (see Fig. 1). Based on the CHM warning, the estimate in MeReC Extra 30, with some caveats, is that contemporary levels of diclofenac prescribing may account for up to 2000 premature or avoidable myocardial infarctions in England each year.
Changes in NSAID prescribing in England: 2006–07 and 2007–08. NHSBSA (PPD), personal communication. EM, East Midlands SHA area; EE, East of England, L, London; NE, North East; NW, North West; SC, South Central; SEC, South East Coast; SW, South West; WM, West Midlands; YH, Yorkshire and Humber. This example is reviewed in more detail in another recent Lancet article (Jick et al., 2007).
In 1998 two reports, one from the House of Lords (1997–98) and one from the Chief Medical Officer's Standing Medical Advisory Committee (SMAC, 1998), raised significantly the national profile of antibiotic resistance as a substantial threat to the public health. However, prescribing of antibiotics in England had already fallen from a peak of 49 million items a year in 1994–95 to around 41 million items in 1998–99 (NHSBSA(PPD), personal communication). Further falls reduced prescribing to 38 million items in 2001–02 since when small increases have occurred. It might be assumed that negotiations in many thousands of consultations nationwide were concluding much more often in a decision not to prescribe. In fact, three studies using different methodologies have shown that the number of consultations for acute respiratory tract infections has fallen at least as fast, if not faster, than the volume of prescribing (Unsworth and Walley, 2001; Fleming et al., 2003; Ashworth et al., 2005). A fourth study in USA confirms this finding (McCaig et al., 2002). In other words, an alternative explanation is that doctors have continued to prescribe at almost the same rate per consultation, but patients are presenting less often with acute respiratory infections. Prescribing of antibiotics in the UK remains higher than many European countries and is increasing (NHSBSA(PPD), personal communication) and bacterial resistance rates follow prescribing volume.
For established guidelines, when is it appropriate to update and incorporate the latest trial data? As soon as an additional, single, large, well-conducted randomized controlled trial is published? Should a change in practice await a further systematic review or further trials? If the trial results are confirmed, early adoption of this new evidence would be advantageous at individual and population levels. However, if an updated systematic review or further studies failed to confirm the benefit, then many people would have received inappropriate treatment, some would undoubtedly suffer side effects from the new approach and limited health care resources may not have been deployed to maximum benefit.
What is the future for EBM?
Is the next phase of EBM the one where we learn to accept that EBM forms only part of the formula for improving patient outcomes? Without question, it forms an important part but not the whole of a ‘best practice portfolio’ since at policymaking level, the ‘evidence’ in EBM is frequently contested with every statistically significant finding having its own set of ‘ah-buts’. The essential tenet of EBM — that we should use the best available evidence to guide patient-centred care — is seen by most practitioners these days as self-evident. But the EBM movement does not seem to have delivered all that some people hoped it would. Is part of the reason that those hopes and expectations did not fully acknowledge the messy reality of clinical practice, human dimensions of decision making and organizational change and policymaking? Is EBM seen by some — even many — clinicians as an externally imposed task which adds to the difficulties of real-world consultations, rather than helping to resolve them? Do some of us persuade ourselves that we are practising EBM, when in fact we are not? What of substance can we put in place that does acknowledge these realities?
The information explosion is here. Are we masters of information or victims of it? The weight and credibility of evidence seems to change with the contexts in which it is generated, presented and applied. Do practitioners recognize when they lack the best evidence for making a decision? Are they able to find it, and if they can, do they have the skills not only to understand it but to translate it into terms both they and their patient can understand and apply to the decision they have to make?
It has been clear for many years now that health care systems and individual clinicians have not been fully using some important clinical evidence when making clinical decisions. But evidence is now also available on how clinicians can better access and use that clinical evidence in consultations, the biases in how decisions are made by humans are better described, as is how organizations can best support that decision making and so achieve more appropriate implementation of research evidence. Are we going to learn from our history, or repeat it, this time by failing to implement the evidence on how to implement research? How wide will be the gap remaining between publication of this evidence on implementation and it being acted upon? As long as it took for thrombolysis to be adopted as routine practice, for maternal steroids to be used in preterm labour or for lemon juice to be used to prevent scurvy?
Do the next generation of GPs want to continue demonstrating the truth of Santayana's saying or can they escape that fate?
The next paper in this series will examine in more detail how clinicians acquire and use knowledge in consultations.
Key points
In less than two decades, the EBM concept and the approaches used within that concept have evolved significantly. Current emphasis is on integrating clinical expertise (particularly in diagnosis and in performing technical procedures) with research evidence and with patients’ circumstances and preferences to arrive at optimal (evidence-based and patient-centred) decisions. But at best, implementation of new research is sluggish and patchy: information alone rarely produces consistent changes in clinical practice, the way that clinicians and human beings, in general, acquire and use information and knowledge in consultations is usually not considered, and clinical practice is influenced by factors other than the evidence base. The EBM movement does not seem to have delivered all that some people hoped it would. Is part of the reason that those hopes and expectations did not fully acknowledge the messy reality of clinical practice, human dimensions of decision making and organizational change and policymaking? Is EBM seen by some — even many — clinicians as an externally imposed task which adds to the difficulties of real-world consultations, rather than helping to resolve them?