Abstract
Clinical practice guidelines (CPGs) are valuable tools for health care providers and support evidence-based medicine (EBM). Many organizations, including medical associations and government-affiliated agencies, develop and publish guidelines using varying methods. Unfortunately, many guidelines are not appropriately developed and certain recommendations are not based on the best available evidence. Recent efforts by EBM advocates are contributing to the improved quality of CPGs and more tools are becoming available to promote high-quality guideline development and use. This article describes the guideline development process and associated concerns and the advances in the field of CPGs. Tools to access and evaluate guidelines are also provided. Health care professionals equipped with an understanding of the process of guideline creation and tools for evaluation can appropriately utilize guidelines to improve patient care.
Introduction
The term “evidence-based medicine” (EBM) appeared in the published medical literature in the early 1990s and has increasingly become an influential principle in the practice of medicine. 1 Advocates of EBM, such as the Evidence-Based Medicine Working Group, emphasized the need to shift from unsystematic methods of care to care based on best evidence from clinical research. 2 EBM was early defined as “the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients.” 3 While the new EBM paradigm promoted the application of research findings to clinical decisions, there was concern that individual clinical expertise and patient preferences were overly deemphasized. Thus, more recently the definition of EBM has evolved to “the integration of best research evidence with clinical expertise and patient values” 4 recognizing that research evidence does not supersede the other components.
In the era of EBM, identifying, evaluating, and synthesizing clinical research are necessary skills and practices. However, the vast amount of literature published makes these activities seem overwhelming. Approximately 2000 to 4000 references are added daily to MEDLINE, the National Library of Medicine’s bibliographic database; almost 700 000 references were added to the database in 2010. 5 With the large quantity and varying quality of published research available, credible publications that have preevaluated and presynthesized relevant medical literature are of great help to the clinician. Clinical practice guidelines (CPGs) can be valuable EBM tools for the health care professional as they provide a review of the available literature and formulate recommendations based on the best available evidence. The purpose of this article is to provide a general overview of CPGs and provide tools to assist the health care professional access, evaluate, and use guidelines.
Clinical Practice Guidelines
In 1990, the Institute of Medicine (IOM) defined CPGs as “systematically developed statements to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances.” 6 However, the term CPGs has been used to refer many different types of clinical guidance including institutional protocols, consensus statements, and expert opinions. Thus to provide clarification, a new definition was published by the IOM in 2011: “Clinical Practice Guidelines are statements that include recommendations intended to optimize patient care that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care options.” 7 The new definition specifies that CPGs are recommendations derived from systematic methods of literature retrieval and evaluation.
CPGs are not mandates or a “cookbook” approach to medicine. As mentioned earlier, evidence alone is not sufficient to practice quality care. Quality care is a combination of evidence, clinical expertise, and patient preferences. 4 CPGs are tools that translate evidence from clinical trials into recommendations that can be applied in practice as the clinician deems appropriate; as clinical trial results represent aggregate data, individual patient factors should be considered prior to applying CPG recommendations. Guideline developing groups (GDGs) aid the clinician by completing the work of searching, retrieving, and evaluating volumes of medical literature. As stated on the National, Heart, Lung, and Blood Institute (NHLBI) Web site, “medical research…will only be able to improve the public health if they are translated into practice.” 8 Many organizations assist in translating evidence into practice by supporting the development of CPGs.
Concerns With CPGs
Guidelines are developed by many different types of organizations including international societies, national and local government agencies, professional societies, health care institutions, and managed care organizations. As different organizations have their own processes and purposes for guideline development, the quality of guidelines has varied greatly. Skepticism of the trustworthiness of guidelines has led to studies which revealed flaws in methods and potential bias in recommendations. Areas of criticism in the development of guidelines have included conflict of interest (COI) management, methodological rigor in literature retrieval/evaluation, and the strength of evidence upon which recommendations are based. 7,9
Conflict of Interest
There has been much published on the issue of COI in guidelines. Many reports and studies have found evidence of COI within guideline-writing panels. 10 –13 Evaluation of the composition of GDGs as well as surveys of guideline writers have found that the majority of the participants reported a COI. 10,11 Types of COI included being consultants, advisory board members, or employees of the pharmaceutical industry. Another study found that one-third of the writers of the CPG actually owned stock in companies affected by the guideline recommendations. 12 While it has been argued that COI does not automatically lead to bias, attention to appropriate disclosure and management of COI is undoubtedly warranted. 13
Literature Retrieval and Evaluation
Much of the process of developing guidelines involves conducting a systematic review. A systematic review is a “summary of previously conducted studies where the research to be included in the review is systematically identified.” 14 As such, the quality of the systematic review directly affects the quality of the CPG. Guidelines have been found to be lacking in transparency and/or rigor in the methods used to conduct a systematic review. 7 A study evaluating the consistency between guidelines (on oral medications for type 2 diabetes mellitus) and a separately published systematic review found that not all guidelines made recommendations consistent with available evidence. 15 Overall, guideline quality was observed to be low and many lacked appropriate, systematic methods to identify evidence. 15
Level of Evidence
By definition, guidelines provide recommendations and impact care decisions. However, the recommendations are based on evidence of varying quality and strength. There are not enough well-designed, clinical trials to support the decision-making process for many patient-specific situations. Consequently, some guideline recommendations are derived from low levels of evidence (eg, small observational studies). A review of practice bulletins published by the American College of Obstetricians and Gynecologists (ACOG) found that only 29% of the recommendations were based on good and consistent scientific evidence. 16 Another study found that a large proportion of recommendations in the guidelines produced by the American College of Cardiology and the American Heart Association (ACC/AHA) were based on lower levels of evidence or expert opinion. 17 Investigation of the guidelines published by the Infectious Diseases Society of America revealed that more than half of the recommendations were from lower levels of evidence. 18 When referring to guidelines, it is critical to examine the level of evidence that recommendations are based on. 19
There are other concerns in the guideline development process in areas such as representation and input from different stakeholders (eg, patients), editorial independence, and timely update. With increasing concern over guideline deficiencies, multiple organizations (as described in the following paragraphs) are involved in improving the guideline-generating process and advocating standards for the production of trustworthy guidelines.
Standards for CPG Development
Institute of Medicine
Concerns about the quality of evidence evaluation and synthesis led the US Congress, through the “Medicare Improvements for Patients and Providers Act of 2008” (section 304), 20 to mandate a study by the IOM of best practices for guideline development and systematic reviews. As a result of the study, the IOM released 2 consensus reports in March 2011: “Clinical Practice Guidelines We Can Trust” 7 and “Finding What Works in Health Care: Standards for Systematic Reviews.” 21 The publication regarding CPGs recommends compliance with 8 standards for the development of trustworthy clinical guidelines. The 8 standards address 7 :
Establishing transparency
Management of COI
Guideline group composition
Clinical practice guideline—systematic review intersection
Establishing evidence foundations for and rating strength of recommendations
Articulation of recommendations
External review
Updating
The report is available on the IOM Web site (http://www.iom.edu/Reports/2011/Clinical-Practice-Guidelines-We-Can-Trust.aspx) and provides a description of each standard. The standards help remove bias and incorporate multiple perspectives by calling for clear, transparent management of COI and a balanced and multidisciplinary GDG comprised methodological experts, clinicians, and other stakeholders, including patients. The standards also provide guidance for strong methodological rigor in conducting systematic reviews and assigning evidence and recommendation ratings.
Guidelines International Network
Guidelines International Network (G-I-N) was founded in 2002 and is an international organization supporting guideline development and implementation. 22 In an effort to develop international standards and consensus, the G-I-N proposed 11 “key components of high-quality and trustworthy guidelines.” 23 The 11 key components proposed by G-I-N are similar to the IOM standards and are provided in Table 1.
Guidelines International Network 11 Key Components for a High-quality and Trustworthy Guideline.a
a Adapted from “Guidelines International Network: Toward International Standards for Clinical Practice Guidelines.” 23
Appraisal of Guidelines for Research and Evaluation Collaboration
Another organization promoting quality guidelines is the Appraisal of Guidelines for Research and Evaluation (AGREE) Collaboration. The AGREE Collaboration is a group of international guideline developers whose purpose is developing a tool, the AGREE instrument, to assist in the evaluation of the guidelines. 24 Use of the AGREE instrument allows evaluation of the methodological rigor and transparency with which a guideline was developed, leading to an overall judgment of the quality of the guideline. The original AGREE instrument 25 was published in 2003 and was a 23-item tool evaluating 6 quality domains. The assessment tool was modified in 2009 to include public health considerations and published as AGREE II. 26 It remains a 23-item tool organized in the same 6 domains 26 :
Scope and purpose
Stakeholder involvement
Rigor of development
Clarity of presentation
Applicability
Editorial independence
Grading of Recommendations Assessment, Development, and Evaluation (GRADE) Working Group
Evaluating the quality of evidence is an important component of systematic reviews and clinical guideline development. However, inconsistent methods of rating quality have the potential to contribute to misunderstanding the strength of evidence. The goal of the Grading of Recommendations Assessment, Development and Evaluation (GRADE) Working Group is to develop a standardized approach to rate the quality of evidence and determine the strength of recommendations. 27 The group began in 2000 and has since published a series of articles titled “Grade Guidelines” describing best practices for rating quality of evidence and determining strength of recommendations. 28 In the GRADE approach, the overall quality of evidence is categorized as “high,” “moderate,” “low,” or “very low.” Factors such as study design, consistency of results, directness of evidence, and risk of bias contribute to determining the level of evidence. Based on the level of evidence as well as other factors such as risks and benefits of therapy, a category of strong or weak is assigned to the recommendation. The strength of recommendation correlates to the level of certainty associated with the recommendation. Many reputable organizations advocating EBM have endorsed the GRADE approach such as World Health Organization, Agency for Healthcare Research and Quality (AHRQ), BMJ Clinical Evidence, and Centers for Disease Control. 27 Detailed description of the GRADE approach and access to GRADE publications are available on their Web site, www.gradeworkinggroup.org.
Accessing CPGs
Guideline Developing Groups
Guidelines can be accessed in multiple ways. One of the most direct ways to access guidelines is through the use of guideline developers’ Web sites. As information dissemination is seen as a crucial part of guideline development, many CPG developers provide full-text access to both methods and results. Familiarity with credible medical associations involved in making recommendations for specific disease states may be the most efficient method to find relevant treatment guidelines. Table 2 provides the names and Web sites of some organizations that develop guidelines and provide the methods utilized in their development process. (Note: within professional society Web sites, CPGs often can be found in sections dedicated for health care professionals.)
Guideline Developing Groups.
Databases and Search Engines
Another way to access guidelines is through databases. There are multiple free, publicly available databases of CPGs (Table 3). A prominent, reputable database developed in the United States is the National Guidelines Clearinghouse (NGC, guideline.gov). The NGC is an initiative of the AHRQ of the Department of Health and Human Services. 29 The NGC Web site, guideline.gov, provides structured, standardized summaries of guidelines that meet certain inclusion criteria. To be included in the NGC, guidelines must be (1) systematically developed and include information that aids health care professionals and patients make appropriate health care decisions, (2) developed by appropriate professional organizations such as medical specialty associations or government agencies, and so on, (3) supported by corroborating documentation giving evidence to an appropriate review of the medical literature, and (4) available full text in English and developed, reviewed, or revised within the past 5 years. Both, international guidelines as well and guidelines developed within the United States can be found in guideline.gov. For example, guidelines developed by Scottish Intercollegiate Guidelines Network and National Institute for Health and Clinical Excellence are available through NGC. 29
As of May 2012, NGC contained over 2400 guidelines. 29 The Web site has several helpful tools to help navigate the numerous guidelines. Guidelines can be searched for by disease state or by issuing organization. Guidelines can be viewed side-by-side using the tool “Compare Guidelines.” NGC has also conducted systematic comparisons, summarizing areas of agreement and difference between different guidelines, for approximately 30 topics. For example, for the topic of urinary tract infection, NGC has compared and contrasted guidelines developed by the ACOG, Infectious Diseases Society of America/European Society for Microbiology and Infectious Diseases, and the Society of Obstetricians and Gynaecologists of Canada. 29
The NGC is a database solely indexing CPGs. There are other Web sites that serve as search engines for CPGs as well as other medical literature. CPGs are often published in peer-reviewed publications and thus indexed in literature databases such as MEDLINE. MEDLINE is freely searchable through PubMed.gov with filter settings that include Practice Guidelines. Other free search engines available to identify guidelines are the TRIP Database and G-I-N. The TRIP Database (tripdatabase.com) is a search engine allowing simultaneous searches through various sources including literature databases, journal Web sites, organization Web sites, guideline developers, and other EBM Internet sites. The sources for the TRIP database are published on their Web sites. Search results are organized by the level of evidence/publication type (ie, systematic review, primary research, guidelines, etc) as well as by clinical area. 30 G-I-N maintains International Guideline Library with public search access to over 6400 guidelines and related documents (as of April 2012), developed or supported by G-I-N member organizations. 31 Utilizing search engines rather than GDG Web sites provides a more comprehensive view of available guidelines. However, the search engines do not necessarily evaluate or vouch for the quality of indexed guidelines.
Tertiary References
With the emphasis on guidelines, many tertiary references and databases are including guidelines in their monographs and summaries. Pharmacist’s Letter, a subscription service providing evidence-based drug information now includes relevant treatment guidelines in search results. 32 Micromedex 2.0 provides guideline recommendations through its DRUGDEX Drug Consults. 33 Lexicomp provides summaries and hyperlinks of CPGs in its drug monographs. 34 Facts and Comparisons eAnswers provides information on guidelines in the section, “Comparative Efficacy Topics.” 35
Using CPGs
CPGs are tools to support the practice of EBM. As such, appropriate use of CPGs follows the standard processes of EBM. The steps of EBM are consistent with the systematic approach to answering a drug information question and can be generalized as 14,36 :
Formulate a clear answerable clinical question
Search systematically for the best available evidence
Evaluate the literature for applicability and quality
Synthesize findings and apply to clinical situation
Evaluate performance and identify ways to improve
CPGs, like all other sources, need to be evaluated for quality. Evaluating the quality of CPGs involves accessing and reviewing the methods used in the development of the guideline. The description of methods is provided in different ways by different GDGs. Sometimes the methodology is published with the guideline as done in the Antithrombotic Guidelines (9th edition) published by the American College of Chest Physicians (ACCP). 37 Other times GDGs, such as the ACC/AHA, publish a general manual for guideline development on their Web site and not within each specific guideline. 38 Figure 1 shows a summary of the guideline document development process provided in the “Methodology Manual and Policies from the American College of Cardiology Foundation/American Heart Association Task Force on Practice Guidelines.” 38 If methodology for the guideline development process is not available, the user is unable to determine the quality of the guideline and should be wary of the recommendations contained within.
Document development process. Reprinted with permission from Methodology Manual and Policies Methodology Manual and Policies From the ACCF/AHA Task Force on Practice Guidelines. Available at http://assets.cardiosource.com/Methodology_Manual_for_ACC_AHA_Writing_Committees.pdf. ©2010.The American College of Cardiology Foundation and American Heart Association, Inc.
Various tools are available to help evaluate the methodology of guideline development. As mentioned earlier, the AGREE II instrument is a publicly available tool (www.agreetrust.org) utilized for guideline evaluation. Another worksheet available to use in guideline evaluation was created by Pharmacist’s Letter based on the original AGREE instrument and is shown in Figure 2 (reproduced with permission). Guidelines can also be compared to the previously mentioned standards published by the IOM and the G-I-N. As part of the consensus report, “Clinical Practice Guidelines We Can Trust,” the IOM included qualities of trustworthy guidelines. As they state, to be trustworthy “guidelines should be based on a systematic review of the existing evidence; be developed by a knowledgeable, multidisciplinary panel of experts and representatives from key affected groups; consider important patient subgroups and patient preferences, as appropriate; be based on an explicit and transparent process that minimizes distortions, biases, and conflicts of interest; provide a clear explanation of the logical relationships between alternative care options and health outcomes, and provide ratings of both the quality of evidence and the strength of recommendations; and be reconsidered and revised as appropriate when important new evidence warrants modifications of recommendations.” 7
Worksheet to evaluate clinical practice guidelines (reproduced with permission).
While the standards and tools are helpful, guidelines that do not meet all standards proposed by the organizations described above may still be valuable and useful. For example, the NHLBI develops guidelines using rigorous methods but does not include patient and consumer representatives in the guideline development panel. However, the NHLBI allows for a period of public comment at which time patients and patient groups may give feedback on the guideline. 39 It is important to look at the guideline holistically for overall quality and not dismiss them because 1 standard or criterion is not met.
Before utilizing recommendations in guidelines, the date of creation or last update should be reviewed. It is important to determine whether the guideline is a reflection of the current evidence. Various time frames for guideline update have been suggested as providing an acceptable guideline currency. An inclusion criterion of the NGC is that the guideline has been developed, reviewed, or revised within the past 5 years. 29 Findings of a study of AHRQ resulted in the recommendation that guidelines should be reviewed every 3 years. 40 Of course, there is no clear cutoff point at which guidelines are out-of-date or before which guidelines are still current. However, consideration of date of publication and recent research in the area is necessary prior to using CPGs.
After the guideline’s quality and timeliness are deemed acceptable, each recommendation should be reviewed for level of evidence and strength of recommendation prior to use. Not all recommendations in a guideline are given with the same level of certainty. In the ACCP Antithrombotic Guidelines (9th edition), recommendations are given 1 of the 6 grades 1A, 1B, 1C, 2A, 2B, and 2C. 37 The grade 1A is the strongest recommendation assigned and indicates that “benefits clearly outweigh risks and burdens or vice versa.” The recommendation is based on “consistent evidence from randomized controlled trials without important limitations or exceptionally strong evidence from observational studies.” The grade suggests that the “recommendation can apply to most patients in most circumstances” and is unlikely to change. In contrast, a grade of 2C is a very weak recommendation, based on “low- or very-low-quality of evidence” and indicates that “other alternatives may be equally reasonable” and that further research may significantly impact the recommendation. 37 In addition, each recommendation should be reviewed to determine applicability to the patient population that will be treated.
There may be many situations in which relevant guidelines do not exist. Complex patients or rare clinical conditions may not be addressed in guidelines. In such situations, other resources are available to assist the health care provide practice EBM. These resources include
Agency for Healthcare Research and Quality (www.ahrq.gov), an agency within the Department of Health and Human Services providing evidence-based reports, technology assessments, and so on.
Centre for Effective Practice and Guidelines Advisory Committee (GAC; www.effectivepractice.org), a Canadian-based organization providing EBM programs and tools (NOTE: GAC reviews, endorses, and summarizes evidence and guidelines).
Centre for Evidence-based Medicine (www.CEBM.net), a center in the University of Oxford providing EBM educational resources and tools.
Cochrane Collaboration (www.cochrane.org), an international organization providing high-quality systematic reviews and additional EBM tools.
Evidence Updates (http://plus.mcmaster.ca/evidenceupdates/), a service providing quality ratings and a searchable database for current medical literature; created through the collaboration of the BMJ Group and the McMaster University's Health Information Research Unit.
National Institutes of Health (NIH) Consensus Development Program (www.consensus.nih.gov), a program convening conferences to produce evidence-based consensus statements regarding controversial medical topics.
Conclusion
Appropriately developed CPGs are valuable tools to assist health care professionals provide evidence-based, quality medical care. Unfortunately, not all guidelines are appropriately developed and concern remains over the trustworthiness of recommendations found in many guidelines. Criticism of guidelines abounds and ranges from a concern over the lack of consistency with the published evidence 15 to a concern over too much emphasis on published literature. 41 EBM advocates from both the private and the public sectors are concentrating efforts to improve the guideline development process and establish standards characteristic of trustworthy guidelines. International organizations such as the World Health Organization have made efforts to harmonize guidelines. 42 Thus, while there are many issues surrounding guidelines, the awareness of problems and research in guideline methodology are leading toward palpable changes in the field of CPGs. Guidelines remain a “significant advance in the effort to promote consistent clinical care standards that efficiently utilize evidence-based medicine.” 43 Implementation of guidelines may help reduce the significant gaps between recommended practice and the variations of care known to be prevalent today. 44,45 Health care professionals equipped with an understanding of the process of guideline creation and tools for evaluation can appropriately utilize guidelines to improve patient care.
Databases/Web Sites Indexing Clinical Practice Guidelines.
Abbreviations: CPG, clinical practice guideline; EBM, evidence-based medicine.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.