The Changing Landscape of Maintenance of Certification: History,Value and Evidence Base,and Future Impact on Forensic Pathology

Abstract

Maintenance of certification (MOC) is a current hot topic in all disciplines of medicine, and no less so in pathology and forensic pathology, specifically. The extent of physician discontent with MOC has risen to a fevered pitch over the past few years, with reporting on specialty board activities and physicians' reactions reaching the popular media. This invited review authored by several of the members of the National Association of Medical Examiners Education Subcommittee on the Development of Self-Assessment Modules provides an overview of MOC as it relates to forensic pathology. We address the history of MOC and its value as stated by the certifying bodies that created, promote, and administer MOC, including the American Board of Medical Specialties and American Board of Pathology. We further provide an analysis of the existing medical literature proposed as an evidence base for MOC, which is somewhat limited in its scope, particularly nonrobust in pathology, and nonexistent in forensic pathology. We discuss recent changes that medical specialties have made to prescribed MOC programs, potential alternatives to MOC, and the impact that MOC in its current and potential future forms may have on the field of forensic pathology, including effects on the workforce, courts of law, and training pathways.

Keywords

Forensic pathology Board certification Maintenance of certification American Board of Pathology Evidence based medicine

Introduction

A crossroads has been reached in medicine relating to board-certification and “maintenance of certification” (MOC). While the American Board of Medical Specialties (ABMS) has made MOC mainstream by requiring adoption by all 24 member medical specialties and limiting board-certification duration to a maximum ten-year period, a groundswell of grassroots physician movements and organizations have arisen in opposition to MOC. Editorials in the professional literature and reporting in the popular press and medical news have blasted MOC with commentary characterizing it as “expensive, time-consuming exercises in busy work”(1), or alternatively as “onerous paperwork” and “a waste of time and resources”(2), positing that MOC takes physicians' time away from patient care. Several state medical societies and the American Medical Association (AMA) have adopted resolutions against MOC, and/or against the linkage of MOC with licensure requirements, with many calling for an evaluation of the efficacy of MOC and assessment of the cost of time and money on the profession. Physicians have even gone so far as to develop alternative certification bodies to avoid MOC and the high-stakes maintenance of certification exams associated with it (3). More than 19 000 physicians reportedly signed a web-based anti-MOC petition, and a second petition garnered nearly 6000 signatures, advocating taking a “pledge of noncompliance” with MOC requirements (4). All of these events indicate that there is more than a little unrest in medicine regarding the current MOC structure.

Whether or not one accepts as an intellectual concept that MOC has merit to the physician and/or patient, the question frequently arises as to whether an adequate evidence base exists in the medical literature demonstrating an objective benefit to patients or physicians attributable to the completion of MOC activities. The perceived lack of a sufficient evidence base has been cited by medical societies (such as the State Medical Society of Massachusetts) as grounds for a moratorium on MOC. Particularly in pathology, and specifically forensic pathology, this evidence base has yet to be established.

This review seeks to analyze the history and development of MOC, including recent changes in pathology and other specialty boards' MOC requirements; describe MOC's proposed or purported value to physicians and patients; evaluate the evidence base for the merit of MOC in medicine in general and pathology in specific; and consider the potential future impact on our specialty, forensic pathology. The authors have attempted to make this review as current as possible, with the caveat that the landscape of MOC is rapidly evolving and it is inevitable that additional changes are ahead, and may potentially occur prior to publication.

Discussion

History of Maintenance of Certification

The concept of continuing or lifelong medical education is not a new one in medicine. In fact, its origins can probably be traced to the time of Hippocrates (Image 1), the “Father of Medicine” who lived in the 5th century BCE, and the evolution of the Hippocratic Oath. While not specifically mentioning continuing education, the underlying theme of this oath is that the physician strives to be the best physician he or she can be, particularly in the many modern versions of the oath (5 -7). A select timeline of the evolution of medical education standards and board organizations with focus on specific MOC events is presented in Figure 1.

Image 1

Engraving of Hippocrates by Peter Paul Rubens (public domain, 1638).

Figure 1

A select timeline of development of medical boards and maintenance of certification.

The AMA established not only the first formal physician medical education standards in 1847, but also the first code of medical ethics (8). The code of medical ethics has since undergone multiple revisions, with the last one in 2001. In all versions it has been emphasized that a physician's first responsibility is to his or her patients, and that he or she should “maintain a commitment to medical education” by continuing to “study, apply, and advance scientific knowledge” (9). In 1904, the AMA instituted an accelerated campaign to raise educational requirements for all physicians (10).

Formalized oversight and regulation of medical practice standards has some of its roots in the 1916 founding of the American Board of Ophthalmology. The basic principle emphasized by this board was the welfare of patients over other matters (similar to the Hippocratic Oath and AMA code), and importantly, “effective education was recognized as key to quality” (11). Ophthalmology was one of the first areas of specialization to emerge in medicine, and the field was not met with acclaim early on, with both the majority of physicians frowning upon specialization, and intense competition from other purveyors of refraction-correcting eyeglasses such as opticians and jewelers. The proceedings and minutes from the 1915 and 1916 American Ophthalmology Society meetings established some key components for the formation of medical board systems, which included education as a core value for board-certification, and the idea that board-certification provides the means for the public to gauge the competence of specialists. Also of note are the principles of the voluntary nature of board-certification, and that certification should not privilege specialist access to patients (11).

After this, representatives from the pioneering specialty boards of Dermatology, Obstetrics and Gynecology, Otolaryngology, and Ophthalmology, along with major physician, hospital, medical education and examining groups convened in June 1933 to formulate the education and certification of medical specialists. Additionally, the group had goals to establish a national system of standards for recognizing specialists and provide information to the public (12).

Ultimately the ABMS was formed. ABMS in its current form assists 24 medical specialties [including the American Board of Pathology (ABP)] in maintaining the standards for ongoing evaluation. ABMS does not dictate that each specialty board follow specific procedures, recognizing that different specialties of medicine have different practices and educational requirements, but does dictate much of the structure of MOC. Boards have flexibility to modify standards as necessary. The ABMS provides an outline for maintaining certification standards, which includes six core competencies, the same competencies that postgraduate medical education residency programs are required to address: 1) patient care; 2) medical knowledge; 3) practice based learning and improvement; 4) interpersonal and communication skills; 5) professionalism; and 6) systems-based practice.

While the American Board of Family Practice has been issuing time-limited certificates since 1970, and many other specialties adopted time-limited certificates throughout the 1980's and 1990's, the current format of MOC as detailed by the ABMS was formally adopted in 2000 by the 24 specialty and subspecialty member boards. According to literature produced by the ABMS, the goals of the MOC program are:

assuring physician engagement in self-assessment, life-long learning, and continued performance improvement” because “periodic assessment is necessary to assure the public that physicians are maintaining their clinical competence and providing high-quality care throughout their practice career (13).

The ABMS and the member boards then used this call from the “public” as grounds to create criteria for proving that doctors provide “quality patient care” that can be easily measured and documented, but that have been sometimes criticized as not particularly relevant to practice. Since the adoption of MOC, the criteria to “demonstrate commitment to lifelong learning and competence” have been adjusted by the ABMS, and with each new iteration, the requirements set out by the individual boards, including the ABP, have also changed.

The ABP was the last specialty board to adopt time-limited certificates (i.e., require a recertification exam), and now requires diplomates board-certified in 2006 and after to complete a structured program of MOC with a recertification examination (also known as MOC examination) every ten years. Since that time, the board has periodically modified and refined the process across all pathology subspecialties. The requirements given to those taking the ABP certification exam(s) in 2014 are shown in Figure 2 (14). The four parts of the ABP MOC program (also described in more detail in Table 1) consist of: Part I, Licensure and Professional Standing; Part II, Lifelong Learning and Self-Assessment; Part III, Cognitive Expertise; and Part IV, Evaluation of Performance in Practice (15). A Patient Safety Course is required under Part II, and is a newer requirement added in 2013. Diplomates are required to formally report to ABP the details of completed MOC requirements/activities every two years of the over-arching ten-year MOC cycle. A fee (currently $100) is assessed at each two-year reporting period. The costs associated with MOC are further described in Table 1.

Figure 2

Summary of American Board of Pathology (ABP) maintenance of certification Requirements, (courtesy of ABP, reproduced with permission).

Table 1

Four Parts of American Board of Pathology Maintenance of Certification Program, with Main Requirements and Current Estimated Costs

Part #	Part Title	Main Requirements – Diplomates Must:	Associated Costs
I	Licensure and Professional Standing	Hold a valid, unrestricted medical license	No additional cost over medical license already required
II	Lifelong Learning and Self-Assessment	Meet educational requirements, completing continuing medical education (CME) and self-assessment module (SAM) credits (a total of 70 hours CME per two-year cycle, including 20 hours SAMs); and an approved Patient Safety Course once in the ten-year cycle	Variable; SAM credits may require additional fees over the cost of just CME in some cases; Patient Safety Course minimum cost $ 100
III	Cognitive Expertise	Pass a secure, closed-book examination	$500 exam fee; any costs of exam preparation; costs of travel and lodging (exam site in Tampa, FL)
IV	Evaluation of Performance in Practice	Report laboratory accreditation status, participate in laboratory and individual performance improvement and quality assurance programs, and submit peer attestations as to a host of professionalism, ethics, and interpersonal skills	Variable; A $100 fee is also assessed by ABP to accompany the reporting form of all MOC activities required every two years in the ten-year cycle

The ABP general certification and MOC information booklets were last updated in February 2015. A substantial change from the 2013 booklet regards the requirement to maintain one's primary certification if one also possesses subspecialty certification. According to the 2013 online MOC booklet from the ABP, it was acknowledged that the specialty of Pathology is quite diverse, and that perhaps the pathologist may choose to continue maintaining certification(s) only in their specific subspecialty area(s) of practice. Counter to this, the ABP also wrote that it:

strongly encourages diplomates to maintain all certificates, especially through at least one 10-year MOC cycle, since practice circumstances frequently change. It is much easier to maintain certification than to become recertified (16).

Subsequently, a webinar hosted by ABP Chief Executive Officer (CEO), Dr. Rebecca Johnson, was held on October 28 and November 4, 2014 in order to provide diplomates with information to better understand the importance and process of MOC. One of the things touched upon during the webinar was that the ABP was considering making mandatory the maintenance of primary certificates [anatomic pathology (AP)/clinical pathology (CP), AP only, or CP only] for all subspecialty diplomates. On November 24, 2014, diplomates received an email stating that indeed, maintenance of primary certifications was now mandatory (17). The MOC booklet was updated in February 2015 to reflect this change (18). The rationale provided for this change was multi-fold, but a key component was the ABP's assertion that maintenance of the primary certification supports the ABP mission to “promote the health of the public and advance the practice and science of pathology” (17). Elaboration on this rationale, considered dubious by many subspecialist pathologists including forensic pathologists, will be presented under “Stated Value of MOC” below.

Criticism of and Changes to Maintenance of Certification in other (Nonpathology) Medical Specialties

As previously mentioned, the MOC process has been under fire, especially over the last two years, much of it stemming from changes made to the American Board of Internal Medicine (ABIM) MOC requirements in 2014 that were interpreted as particularly onerous, costly, and time consuming (19,20). In response to backlash from thousands of ABIM diplomates, particularly cardiologists led by the American College of Cardiologists (ACC), in February 2015 the ABIM backtracked on several of the newly instituted requirements (19). Specifically, they vowed to update the examination to reflect more pertinent knowledge, suspended some aspects of the program for at least two years (Practice Assessment, Patient Voice, and Patient Safety requirements), suspended any fee increases through 2017, altered the way in which a diplomate's MOC status is reported on their website, and vowed to become more flexible in accepting various forms of Accreditation Council for Continuing Medical Education (ACCME)-approved continuing medical education activities (19). Nonetheless, there are continuing efforts, led by the ACC, to further alter the ABIM MOC process to best meet the needs of the specialty and associated subspecialties (20).

One such impending change could be dropping the “high stakes” ten year exam conducted by ABIM and most other specialty governing groups, in favor of more frequent, continual quizzes that can be accessed from home (21). While the results of the quizzes could still impact certification, the thought is that more timely and relevant material presented in smaller doses would more accurately reflect the practitioner's knowledge and performance. The American Board of Anesthesiology (ABA) recently instituted this type of change to its MOC program, eliminating its ten-year recertification/MOC exam and replacing it with online quizzes requiring that diplomates complete 30 questions quarterly (22). Additionally, the ABA dropped a simulation requirement that required diplomates to travel to specific testing sites and often cost more than $2000 (22).

Likely fueled by the MOC backlash that has even reached mainstream media, where it has been described as “a civil war in American medicine” (23,24), more and more physicians are questioning their MOC programs. Several of the overseeing boards, including the American Board of Pediatrics (ABPED) (25), American Board of Emergency Medicine (ABEM) (26), and American Board of Nuclear Medicine (ABNM)(27), are actively seeking input to refine their MOC processes, with a focus on testing, lessening the burden of documenting quality improvement, and decreasing the fees for diplomates. Notably, while the ABNM is maintaining a singular high stakes exam, they are working to provide key articles for review by their diplomates free of charge, with plans for 30% of the questions on the MOC exam to come from these articles. They also plan to offer free continuing medical education (CME) and self-assessment module (SAM) credit (27). ABPED held a stakeholder conference in May 2015 and has pledged to make changes to help diplomates identify high yield articles, address the format and content of their MOC exam, and make it easier to identify ways to attain credit for quality improvement activities (25). Similarly, the ABEM held a stakeholder conference in October 2014 to address concerns and invited leaders of various academic and professional organizations to attend. Getting “buy-in” from the pertinent societies reportedly led to constructive conversations in helping mold the MOC process moving forward (26).

The ABIM learned the hard way that proceeding blindly with excessive, time consuming, and costly MOC requirements was not acceptable to its diplomates. Moving forward it seems clear that the ABMS and its specialty board members will be forced by physician diplomates as a body to evaluate their MOC programs piece by piece to determine if every aspect of the program is useful, pertinent, and cost effective.

Stated Value of Maintenance of Certification Promulgated by American Board of Medical Specialties and American Board of Pathology

According to the ABMS, “MOC plays a vital role in making sure physicians critically evaluate their practices, acquire new skills, and adapt their practices to changing patient health needs” (28).

ABMS asserts that:

Patients benefit from a physician's focus on the core competencies for quality care by receiving,

•

Better communicative relationships built on listening and sharing

•

Improved sensitivity to their needs and concerns

•

Demonstrated clinical knowledge concerning their medical condition

•

Coordinated care from interprofessional teams and integrated systems of care (28).

As for relevance to physicians themselves, the ABMS states that ABMS MOC is a widely recognized marker of quality recognized by credentialing organizations and the federal government, among others, and that through MOC, “board certified physicians advance the standard of specialty medical care nationwide for improved outcomes and better patient experiences” (28). (Further discussion of whether there is evidence of improved patient outcomes will be presented below). ABMS further asserts that specific physician benefits from participating in MOC are:

•

Focused learning based on individual practice needs

•

Increased practice-based and clinical efficiencies

•

Enhanced communication with colleagues, patients, and caregivers

•

Better working relationships within the system of care (28).

Perhaps most intriguing of all, the ABMS website presents a webpage of physicians quoted in support of MOC activities, presumably to point out that there are physicians in support of MOC. Such quotations include:

Education is central to MOC for both doctors AND patients. Knowledgeable, skilled doctors can continue to provide quality care. Informed patients can make better health care choices (29).

These quotations are from physicians in dermatology, internal medicine, family medicine, medical genetics, neurosurgery, nuclear medicine, physical medicine and rehabilitation, plastic surgery, and radiology. Of note, no quotations from pathologists or forensic pathologists are listed.

The ABP has also developed “talking points” which describe the rationale for MOC, and specifically, for requiring maintenance of primary certification. These were published in a prior issue of Academic Forensic Pathology in a Question and Answer session with ABP CEO Dr. Rebecca Johnson (30), and are reproduced here:

•

The mission of the ABP is “to promote the health of the public and advance the practice and science of pathology” …. Maintenance of primary certification supports that mission.

•

Maintenance of primary certification in the MOC program preserves and protects the status and value of ABP certification in pathology.

•

Maintenance of primary certification preserves and strengthens the foundations of our specialty.

•

As medicine and the field of pathology are rapidly evolving and advancing, the distinctions between anatomic and clinical pathology are becoming increasingly blurred. Maintaining primary certification best positions pathologists to meet the requirements of practice in the current and anticipated future environment of medicine.

•

Accountable care organizations will need and expect pathologists to play important consultative roles. Maintenance of primary certification enhances the credibility of pathologists in these organizations.

•

The public expects that physicians will maintain a core of general knowledge in their field, regardless of their subspecialization.

•

We are physicians first, pathologists second, and in some cases, subspecialists third. As a profession, we should maintain our identity as pathologists by maintaining our primary certification.

•

The ABP recognizes that diplomates' pathology practices are varied. The ABP's MOC program allows for diplomates to tailor their MOC Part II and IV activities to reflect their practice. [truncated]

When asked about the evidence base for MOC in the Question and Answer session (30), Dr. Johnson referred to the “ABMS Evidence Library” (31), and stated:

The ABP would welcome any research into the value of MOC specific to pathologists. There is some indirect evidence that Part IV activities improve pathologist's medical practice from the CAP performance improvement activities published in Archives of Pathology and Lab Medicine. (30)

The ABMS website has a section titled “ABMS Evidence Library” (31). The entire content of this library is 245 articles (when accessed 4/24/2015), addressing the effects of performance improvement modules and other MOC activities in clinical specialties such as family medicine and internal medicine. A single article was identified in this library that addresses pathology, authored by ABP CEO Dr. Rebecca Johnson (15). There were no articles focusing on forensic pathology.

Evidence Base for Maintenance of Certification

The link between specialty board-certification and improved patient care has been studied extensively in clinical medicine, although studies looking specifically at pathology and its subspecialties are not present in the literature. It has been shown that board-certification is correlated with adherence to professional guidelines (32). However, many of the studies currently in existence do not examine the link between certification and actual patient outcomes but rather cognitive competence as measured by board scores or process measures such as clinical testing following test practice guidelines. The data suggests a positive association, although this is not universal.

Sharp et al., in a 2002 study, identified 237 papers in the medical literature that examined quality of patient care and board-certification (32). Only 56 of these papers were considered of sufficient quality to undergo more detailed scrutiny. Of these, only 13 papers with 33 separate findings met their criteria, which included confirmation of board-certification, solid study methodology, and examining patient outcomes. Of the 33 findings, only 16 demonstrated a significant positive association with board-certification, 14 showed no association, and three showed a worse association with board-certification. Several other studies have demonstrated improved outcomes in some measures but no difference in other measures (33 -39). While not overwhelming, the preponderance of studies indicates a positive association between board-certification and good patient care. There needs to be some measure of clinical competence that assures both to other professionals and the general public that physicians have met or exceeded a minimum standard to practice in their chosen field. Board-certification is widely recognized as fulfilling that need.

While there have been a few publications discussing maintenance of certification and its relationship to pathology (15, 40 -43), there have been no scientific studies examining the relationship between board-certification or MOC and quality of practice in pathology or its subspecialties, including forensic pathology.

Once a physician has demonstrated that they possess the mental discipline and cognitive ability by attaining initial board-certification, what is the science behind the need to periodically reassess and recertify? In other words, what is the theory behind maintenance of certification? Levinson and Holmboe summarized the arguments in support of MOC (44). There is evidence that physicians are not always good at assessing their own performance and their own abilities. Some studies suggest that clinical competence decreases with age or that older physicians may fail to “keep up” with medical advances in their field. There is also evidence that residents who complete training programs with deficiencies in clinical skills may not correct these deficiencies once in clinical practice. In this same paper, they reviewed the results of an ABIM survey of internists performed between 2007 and 2009 to determine the physicians' perception of the value of MOC and their experience in the process. Those results indicated that for SAMs, 70% of internists felt that these modules provided a positive learning experience. Other positive comments included identification of areas for future learning and increasing physician awareness on how to improve patient care. Eighty-four percent rated the overall experience of the secure closed book test as high. Interestingly, despite the high overall rating, only 61% thought the secure test was a relevant test of their clinical knowledge. Other reviews in the literature echo Levinson and Holmboe's arguments (13,45).

Evidence that a secure, closed-book examination itself improves patient care is weak. Arguments in support of this exam center on the public's expectation that physicians regularly undergo such an examination. There is more evidence that associates quality of care with Part IV of the MOC process, the practice/performance improvement modules and Part II, self-assessment modules and continuing education, further discussed below. It is important, however, that the actual value of these improvement programs and educational activities is not completely offset by the time burden, undue complexity, cost, or factors beyond the control of participants–for example, poor response to patient surveys.

Only recently have there been any studies comparing time-limited and time-unlimited certificate physicians. Much of the evidence supporting MOC is associative and not necessarily causal. Again, there is no literature addressing the value of MOC in improving the practice of pathology or its subspecialties. Therefore, a review of literature relating to MOC in other medical specialties is necessary in order to understand the potential value of recertification as well as understanding the pitfalls and difficulties in evaluating both the efficacy as well as the relationship, if any, to improving patient outcomes and the quality of medical care.

MOC may have other utility apart from improving the quality of individual practice. Hess et al. described how utilizing the ABIM Practice Improvement Module (PIM) on diabetes, and a panel of eight internists, could establish a mechanism to identify a population of physicians who failed to meet the standard of a borderline physician in the care of diabetic patients (46). Of the 957 physicians in the study, 38 internists (4%) failed to meet the standard of practice as established by the panel. Closer examination of characteristics of this physician group revealed that they tended towards lower ratings of clinical competence/professionalism while in residency, had lower internal medicine primary certification scores, had lower MOC scores, and worked primarily in solo practice. This study does validate that PIMs may be useful in identifying a subset of physicians that may need additional guidance in improving quality of patient care and identifying areas specifically in need of emphasis. However, there was ample evidence that these physicians were already lower performers. The fact that solo practitioners made up a large portion of this group could suggest that solo practitioners may be a group “at risk,” although another possibility might be that solo practitioners may be more likely to be lower performers. Additionally, this study did not address patient outcomes.

In a small study reported by Simpkins et al., the asthma specific PIM was reviewed to determine if participation resulted in improved patient outcomes (47). This small study, with an intervention group consisting of eight practices and 19 internists, and a control group of eight practices and 21 internists, used as its primary outcome inhaled corticosteroid fill rate, as chosen by the authors. Using that primary outcome, there was no significant difference between the intervention group and control group in the primary outcome, inhaled corticosteroid fill rate. However, the intervention group was more likely to discuss asthma triggers with their patients, and patients themselves reported lower asthma symptom severity. The intervention group was less likely to have a written action plan for their patients. Since participants in the PIM are able to choose their own outcome measure, it is quite possible that the intervention group may have targeted an alternate outcome such as lower patient self-reported asthma severity, which may have resulted in more significant outcome improvement. However, it should be noted that only five physicians in the intervention group completed the PIM as directed.

Family practice was the first specialty to issue time-limited certificates, with the first certificates issued in 1970. Although there has been some evidence that suggests that older physicians farther out from initial training are subject to decline in cognitive skills, results from a 2013 Academic Medicine study suggest that with regular recertification older physicians' cognitive skills as measured by MOC scores do not decline but in fact may improve over time (48). Even though by the approximate age of 59 to 61 years performance fell slightly, these physicians still performed better than initial certifiers. However, physicians with gaps in their recertification perform significantly worse. This study however does not prove that regular certification is the cause of this improvement in performance. These physicians who comply with recertification requirements may represent a population of overall better performers. In addition, this study suggests that experience cannot be discounted in quality medical care. And again, actual patient outcomes were not examined.

In another paper that explored the relationship between Self-Assessment Module/Performance in Practice Module (SAM/PPM) completion and the quality of clinical care provided by family practitioners in type II diabetes patients, Galliher et al. examined 76 family practice physicians and 15 practices in a retrospective two-year study between 2007 and 2009 (49). There were three groups: one group participated in a diabetes SAM/PPM during the study period, a second group participated in other SAM/PPM, while a control group participated in no SAM/PPM. While limited by a small size and physicians who self-selected participation in the modules, physicians who participated in any SAM/PPM performed statistically better than the group who did not participate. Interestingly, there were more improvements in specific elements of diabetes patient care in the group participating in “other” (nondiabetes) SAM/PPM than in those who participated in the diabetes SAM/PPM. While the authors admitted this study might demonstrate a lack of effect of the SAM/PPM process, it is also very possible given the overlap of contents in the other MOC modules related to cardiovascular disease and hypertension with the treatment of diabetes patients that there is still a positive association between MOC participation and quality of patient care. While this study looked at process measures in the care of diabetes patients, actual patient outcomes such as hemoglobin A1c results, mortality, and morbidity were not studied. While this is a small study, it adds to the evidence that MOC and particularly the SAM and practice performance modules may have utility in improving the way physicians practice in clinical medicine specialties.

Holmboe et al. examined the relationship between cognitive skills as measured by MOC score and the quality of care for Medicare patients as indicated by claims review for diabetes patients, mammography screening for women, and lipid screening for patients with cardiovascular disease (50). This was a large study involving 3602 physicians and 220 340 Medicare beneficiaries. Those physicians in the lowest 25th percentile group were more likely international medical graduates (e.g., not U.S. or Canadian), more than 20 years out of medical school, or were solo practitioners. This group also performed significantly worse in achieving the diabetes mellitus process measures and in performing recommended mammography screening. While those with lower MOC scores were associated with worse process measures, one cannot argue that this association is causal. Those physicians with higher scores could well be physicians who naturally perform higher and are better trained. That the group with the lower scores also had a greater number of physicians graduating medical school greater than 20 years previously does support statements that cognitive skills can decline over time. As in Hess's 2010 study solo practitioners also made up a larger portion of this group, suggesting this as a potential group at higher risk. However, actual patient outcomes related to better disease detection and control, morbidity, and mortality were not examined, and so it is unclear if lower scores directly impact patient health.

While actual patient outcomes have often been overlooked in studying the effect of board-certification and MOC on quality of care (QOC), Hess et al. examined the 2005 – 2009 MOC scores and practice performance data from the ABIM diabetes PIM for 676 physicians (51). They found a statistically significant although modest association between QOC and MOC exam scores; patient outcomes demonstrated the highest association. Because the association was only modest, albeit statistically significant, the authors conceded other factors are important in good clinical practice such as quality office personnel and good physician interpersonal skills. Again, this association does not prove causality between the MOC process and QOC as again physicians with higher MOC scores may simply be physicians who are naturally higher performers and therefore more likely to provide higher quality medical care. In order to prove causality, it would be necessary to directly compare outcomes between physicians with time-limited certificates participating in the MOC process and time-unlimited certificates (i.e., physicians who have board-certification) but who do not participate in the MOC process. Until recently, no such studies existed.

In 2014, The Journal of the American Medical Association published two such articles involving the practice of internal medicine. Both articles also looked at actual patient outcomes. Gray et al. compared outcomes for patients of time-limited and time-unlimited certificate holders using ambulatory care sensitive hospitalizations (ACSH) as the primary patient outcome measure, which are by definition preventable through healthcare access and medical care (52). This large study found no significant difference between the two groups in the increase in ACSH; these hospitalizations increased at the same rate for both groups over time. However, they did determine that the annual per beneficiary health care costs, while increasing in both groups over the time study period, grew slightly but statistically significantly more slowly in the MOC group. The mean cost saving per patient was $167, or 2.5% of the mean cost. This cost savings is likely represented by the fact that the MOC group also had fewer specialty and nonspecialty office visits, performed fewer laboratory tests, and had lower imaging costs. The cost difference between minor and major procedures was not significant between the two groups. The authors acknowledged one flaw in the study in that ACSH recently has been found to relate more to access to care rather than quality of care. However, these results suggest that MOC might reduce cost to patients by helping improve medical knowledge of the treating physicians, allowing them to more efficiently utilize resources such as laboratory and imaging services as well as having a reduced need for specialty referral.

In the second study comparing board-certified internists, Hayes et al. performed a retrospective one-year analysis at of 71 time-limited certified internists and 34 non-time-limited certified internists at four Veterans Administration medical centers, looking at ten primary care outcome performance measures (53). There was no significant difference in outcomes between the two groups. However, this was a small study in a very specific academic group practice setting, and outcomes did not include mortality, morbidity or quality of life. These results may well fail to translate to other types of internal medicine or other medical specialty practices.

In conclusion, there is a growing body of literature concerning the relationship of MOC and the quality of medical practice and care. However, many of these studies in the past have not looked at actual patient outcomes but rather MOC scores themselves and or process measures. In addition, even papers that show a positive association between the MOC process and improved quality of care are unable to prove that the MOC process in and of itself was the cause of the improvement. Physicians who perform well in board-certification testing and in the MOC process may simply be physicians who are higher performers by nature and therefore are more likely to provide higher quality medical care. Only two papers as of this writing have directly compared time-limited and time-unlimited certificate holders in the field of internal medicine; neither study demonstrated a difference in overall patient care quality.

What is sorely lacking in the medical literature is any study of board-certification and/or MOC in pathology or any of its subspecialties. It is unclear how data derived from clinical specialties of medicine would translate to pathology. It is even more unclear given the unique practice setting of forensic pathology how any of this data is relevant to our specialty. Forensic pathologists therefore must be proactive in not only partnering with the ABP in developing MOC activities that are relevant to our practice, but also in critically evaluating the MOC process by identifying measurable outcomes which might be impacted through maintenance of certification. How do we measure efficacy and outcomes in forensic pathology? Forensic pathologists must readily communicate with the ABP concerns regarding the relevancy of programs offered and be prepared to develop educational opportunities following the guidelines established by ABMS for MOC.

Alternative Routes for Maintaining Certification

Given the controversies around MOC, and some physicians' desire to avoid the burden of MOC altogether, alternative routes for maintaining some form of board-certification have developed. The National Board of Physicians and Surgeons (NBPAS) was organized in 2014 by Dr. Paul Teirstein, Medical Director of the Scripps Cardiovascular Institute, because of the frustrations he and his colleagues experienced with the complex and expensive ABIM MOC process, about which many physicians made complaints as has been discussed earlier in this paper. NBPAS offers an alternative route for maintaining board-certification that emphasizes CME rather than MOC for all current ABMS specialties, including Pathology. (One must have been originally certified by one of those ABMS specialty boards, however, to qualify for NBPAS certification). Comments from Dr. Teirstein about MOC, which have been echoed by physicians from various specialties, include:

The overwhelming majority of physicians believe the medical knowledge modules and recertifying exams are onerous, and a poor use of their time, as well as MOC focuses on established knowledge while CME often goes further, offering new and future directions that keep the physician on the “cutting edge” (54).

He additionally points out that CME offerings are competitive and offer numerous choices, and if a physician doesn't like one option, there are always others. However, the ABMS boards have until now had a monopoly on certification and MOC, and their fees therefore are not affected by competition. Competitive pressure from the NBPAS may force specialty boards to re-evaluate their programs, both in terms of cost and in terms of relevance to and satisfaction of the physician diplomates.

The NBPAS has the following requirements for board-certification: previous certification by an ABMS member board (such as the ABP); possession of a valid, unrestricted license to practice medicine in at least one U.S. state; completion of a minimum of 50 hours of CME within the past 24 months by a recognized provider of the ACCME. CME must be related to one or more of the specialties for which the candidate is applying. Certain subspecialties require active privileges to practice that specialty in at least one U.S. hospital licensed by a nationally recognized credentialing organization; clinical privileges in the certified specialty cannot have been permanently revoked (54).

The current NBPAS cost is $169 for two years of board-certification (54), lower in comparison with MOC-associated costs with ABMS boards, for example the ABP, as described in Table 1 ($500 MOC Part III Examination fee, a $100 fee for every two year reporting cycle, plus an additional cost of a minimum of $100 for the ABP-required Patient Safety Course) (55). The costs of CME/SAM credits should not be included in this cost comparison, as CME is required for NBPAS also (though fewer hours); notably, however, SAM credits may be more expensive than traditional CME, or require an additional fee to be designated as SAM rather than CME. Other MOC-associated costs related to taking a secure exam include the cost of travel, lodging, time away from practice, exam preparation books/courses, and child care or other such arrangements if needed (though the ABP is currently piloting a program that will allow diplomates to take the exam from the comfort of their own home or office computer while under video surveillance, eliminating travel/lodging costs).

According to the NBPAS website and Facebook page, a dozen hospitals around the country have voted to accept NBPAS as an alternative to ABMS certification (54,56). This is a very small step toward more widespread recognition of this new certification body.

About a year ago, the ABP changed their longstanding policy of only requiring recertification in a pathologist's subspecialty, to now require primary recertification every ten years in AP and/or CP (as previously described), even if a subspecialty pathologist's practice is limited to the subspecialty area and does not entail either general AP or CP. This change has resulted in a small number of forensic pathologists certified since 2006 vowing not to participate in MOC and to simply let their ABP certification expire. The ABP has indicated that failure to participate in MOC, including the MOC exam, will result in the individual being listed and reported to any inquiries as no longer board-certified.

There are indications that some Medical Examiner offices would only require their current and future forensic pathologists to maintain Forensic Pathology board-certification (and allow their primary certification(s) to expire), if that were an option any longer with ABP. However, the question of whether a new NBPAS certification would be recognized in lieu of ABP certification by Medical Examiner and Coroner offices for employment purposes, and by courts generally accustomed to accepting testimony from ABP-certified forensic pathologist expert witnesses, remains to be seen. NBPAS certification may be accepted at a dozen hospitals around the country, but its acceptance in the pathology community and specifically forensic pathology has not been tested.

The ABP has given no indication of reconsidering the decision of last year to require maintenance of primary certification, which seems to be a sticking point also for some subspecialty pathologists. However, a refusal of enough pathologists to continue participation in MOC, and/or a defection to the NBPAS or other as-yet-undeveloped certifying body, could potentially lead ABP to reevaluate their position. Additionally, the changes previously described in this paper regarding other specialties jettisoning the ten-year exam in favor of other, less high-stakes evaluation methods, may also force ABP to reconsider their position on the exam altogether, much less its scope and content. An early indication of this was given by ABP CEO Dr. Rebecca Johnson in her comments at an invited presentation at the 2015 Annual Meeting of the National Association of Medical Examiners (NAME), with mention of the “MOCA Minute” (online quizzes) recently developed by the American Board of Anesthesiology as a replacement for the ten-year exam.

It is extremely relevant here to consider the fact that many other medical specialty boards offering subspecialty certifications do not require maintenance of the primary certification for subspecialists. For example, per the ABIM website, “You do not need to maintain certification in Internal Medicine to maintain certification in Cardiovascular Disease” (57). In fact, most subspecialist practitioners of internal medicine do not need to maintain their primary certification, including: critical care medicine, endocrinology, gastroenterology, geriatric medicine, hematology, palliative medicine, infectious disease, medical oncology, nephrology, pulmonary disease, and rheumatology (57). Likewise, a review of the requirements for various surgical subspecialties shows that primary certification does not need to be maintained for pediatric surgery, surgical critical care, or vascular surgery (58).

The rationale for not requiring maintaining specialty certification for certain subspecialties is not discussed on the websites for the various medical specialties, but presumably a major reason would be the highly specialized nature of the practice of these subspecialties. Certainly, forensic pathology is also a highly specialized area, arguably one of the most specialized subspecialties in pathology, with a majority of forensic pathologists limiting their practices to their subspecialty. The precedent set in other specialties to not require maintenance of primary certifications of their most specialized subspecialists certainly begs the question of the ABP as to why pathology is somehow different. The individual subspecialist pathologist is capable of considering what his/her future practice plans may be, and selecting accordingly the correct certificates to maintain.

Potential Future Impact of Maintenance of Certification on Forensic Pathology

There has been much discussion among forensic pathologists, regarding what the long-term impact of MOC may be on forensic pathology; these discussions can be heard at the NAME meetings, in NAME committees, and around the water cooler in many Medical Examiner/Coroner offices. Concerns regarding the need to maintain the primary certificate(s) (AP and CP if so boarded), and how the difficulty of that maintenance (particularly, the MOC exam) may change over time, are well-founded, considering exam pass rates in other specialties on the MOC exam. Declining exam pass rates were one factor leading to the extensive physician push-back seen by ABIM; first-time takers of the MOC Exam in Internal Medicine had a pass rate of 78% in 2013 and 80% in 2014 (59). Ultimately, a larger percentage went on to pass the MOC exam, but some require a second or third attempt (with the concomitant cost of the exam, travel, and time away from practice).

Thus far, pass rates for pathologists, and forensic pathologists, taking the ABP's MOC Exam have been extremely high; all forensic pathologists have passed the MOC exam (with the majority of those being required to maintain the primary certification(s) also in that exam). A total of 42 forensic pathologists have passed the MOC exam to date. In the most recent (August 2015) administration, eight diplomates took the AP/FP exam, six took AP/CP/FP, and six took FP only; all 20 passed (60). (Those taking FP only have non-time-limited primary AP or AP/CP certificates obtained prior to 2006).

While news of the high pass rate is very encouraging, there is no guarantee that this will continue, or that the exam will not be changed arbitrarily in structure or difficulty, much as it already has changed (regarding the options for content/scope for subspecialty pathologists) within the first ten-year-cycle of MOC. Forensic pathologists, both young and old, have given consideration to what will happen if young forensic pathologists are unable to maintain certification; will they be out of a job if they “lose” certification through exam failure, or through deliberate failure to comply with MOC? Will the already under-supplied FP workforce shrink? With some select forensic pathologists boarded since 2006 indicating they will not participate in MOC (or the MOC exam specifically) as a form of protest, we may learn the ramifications of these actions sooner rather than later.

The concern regarding required MOC is limited to forensic pathologists achieving board-certification since 2006, for now. ABP CEO Dr. Rebecca Johnson has stated that the ABP has no intention to require MOC of diplomates prior to 2006, though there is currently a voluntary process in place for those diplomates (with relatively few “grandfathered” forensic pathologists participating). Other medical specialties such as ABIM have set a precedent of breaking the social contract with their diplomates and changing such requirements, however (the ABIM having announced a requirement for “grandfathers” with time-unlimited certificates to begin participating in MOC and accruing “points” in a MOC system, or face being listed on the Board website as not in compliance with MOC). Some perceived this as an undisguised attempt to move older practitioners out of practice, while it may simply have been the board overreaching. Even if ABP has no intentions to begin requiring MOC participation of grandfathered diplomates certified before 2006, federal and/or state regulations might accomplish the same feat. The potential for medical licensure to become irrevocably linked with MOC participation exists, and remains a looming threat to forensic pathologists (and all pathologists) achieving board-certification prior to 2006. State medical societies and the American Medical Association have published strongly worded statements opposing such legislation.

Forensic pathology is perhaps the only medical specialty where even physicians long in retirement may regularly be called upon to testify in courts of law on cases in which the forensic pathologist participated when in active practice. Since board-certification is seen by the public and the courts as a marker of competence, it adds a perceived legitimacy to witness testimony by the forensic pathologist, and is often one of the foundations for the qualification of the expert witness, in addition to educational background and medical licensure. How required MOC, and resulting lapses in certification due to retirement, might affect the courtroom is unclear.

Additionally, forensic pathologists are uniquely situated in the public eye. If a primary purpose of MOC is to evaluate the quality of medical practice, it could be argued that forensic pathology is perhaps one of the few fields which is evaluated daily by multiple agencies (law enforcement, district attorneys, judicial, media), the community, and other physicians. A serious error can prove quite lethal to the professional's career, and this is regulated not by the ABP or medical licensing boards, but by the public. Further, offices with NAME accreditation are required to participate in quality assurance activities already, adding another layer of supervision and review. All of this oversight already in place begs the question of whether MOC adds any value in forensic pathology.

In response to the requirement for maintenance of a primary certificate, some have said that alternative training pathways should be created that lead to only a focused, terminal forensic pathology certification. This suggestion has been heard previously as a potential solution for the inadequate workforce numbers in forensic pathology. Such training pathways would likely shorten the training time period, and provide greater concentration in forensic pathology, while increasing the number of forensic pathologists that could be trained and certified over a given time, assuming adequate enrollees were available. No such program has as yet been developed, much less presented to or approved by the ABP, to our knowledge. Such a pathway could however be dangerous in that forensic pathology may lose ground as a respected medical specialty and subspecialty of pathology, or in its sometimes tenuous perceived place as the practice of medicine, by any perceived watering down of the training required to become a forensic pathologist. Therefore, any effort to establish this alternate pathway needs to be undertaken with caution, and the training should be rigorous, producing high quality, highly specialized forensic pathologists.

Conclusion

Many physicians regard MOC as burdensome, unnecessarily complex, and not meeting any true need of physician or patient. This may be even truer of pathology, and forensic pathology, given that many MOC activities seem to be designed more for clinical medicine, or general pathology, and are not reflective of our practices. Many of us would simply like to see MOC activities that are fitted to our practices and provide valuable education and improvement opportunities, if these activities are going to be required of us. To that end, NAME currently provides some of the most forensic pathology-relevant SAM credit opportunities available, and hopes to offer a forensic-specific online CME/SAM course to meet the Patient Safety Course requirement of MOC.

Other physicians, including some pathologists and forensic pathologists, feel that MOC is unnecessary altogether and should be abolished, as less clear evidence is established as to the value to both the individual physician and the overall quality of medical practice. In pathology, and specifically forensic pathology, there is no adequate evidence base for the value of MOC to pathologists or patients/population served. Additionally, in clinical medicine, any extant evidence base for the value of MOC to patients appears to be associative and not necessarily causal; the majority of studies do not even address whether MOC participation affects patient outcomes. It remains to be seen whether abolishing MOC altogether is a feasible goal at this point in time, given its entrenchment in all of medicine. Recent events suggest that at a minimum, substantial changes to MOC programs can and must be made to ensure that they are relevant and not cumbersome to diplomates. While some may argue that board-certification is “voluntary” and one can take it or leave it (if one doesn't like MOC), that is small comfort in a world where board-certification is now a requirement for hospital privileges, most medical payment systems, and most Medical Examiner positions (particularly, in NAME-accredited Medical Examiner/Coroner offices), making board-certification necessary for gainful employment for most pathologists and the majority of practicing forensic pathologists. This trend will only continue as regulation of medicine in general, and forensic pathology specifically, increases.

It remains unclear how much burden or impact MOC may have on forensic pathology as a specialty, including workforce numbers, an ongoing concern. Certainly, implementation of any rule or regulation that forced older (pre-2006) diplomates to participate in MOC would be disastrous, resulting in early retirements and reduction of a small and beleaguered board-certified forensic pathology workforce that already cannot fully meet the needs of the United States for forensic pathology services.

Footnotes

The authors have indicated that they do not have financial relationships to disclose that are relevant to this manuscript

References

Lowes

AMA sings same tune as MOC critics. Medscape Med News [Internet]. 2014 Dec 9 [cited 2015 Sep 30]: [13 paragraphs]. Available from: http://www.medscape.com/viewarticle/836215.

Ofri

Stop wasting doctors' time: board certification has gone too far. New York Times [Internet]. 2014 Dec 15 [cited 2015 Sep 30]: A29. Available from: http://www.nytimes.com/2014/12/16/opinion/board-certification-has-gone-too-far.html?_r=0.

National Board of Physicians and Surgeons [Internet]. San Diego: National Board of Physicians and Surgeons; [cited 2015 Sep 25]. Available from: https://nbpas.org/.

Teirstein

P.S.

Boarded to death–why maintenance of certification is bad for doctors and patients. N Engl J Med. 2015 Jan 8; 372(2): 106–8. PMID: 25564895. http://dx.doi.org/10.1056/nejmp1407422.

Wikipedia: the free encyclopedia [Internet]. San Francisco: Wikimedia Foundation; c2015. Hippocratic oath; [cited 2015 Sep 10]. Available from: https://en.wikipedia.org/wiki/Hippocratic_Oath.

Wikipedia: the free encyclopedia [Internet]. San Francisco: Wikimedia Foundation; c2015. Declaration of Geneva; [cited 2015 Sep 10]. Available from: https://en.wikipedia.org/wiki/Declaration_of_Geneva.

NOVA [Internet]. Arlington (VA): Public Broadcasting Service; c1995-2015. The Hippocratic oath today; 2001 Mar 27 [cited 2015 Sep 10]. Available from: http://www.pbs.org/wgbh/nova/body/hippocratic-oath-today.html.

American Medical Association [Internet]. Chicago: American Medical Association; c1995-2015. 1847 Code of medical ethics; [cited 2015 Sep 10]. Available from: http://www.ama-assn.org/resources/doc/ethics/x-pub/1847code.pdf.

American Medical Association [Internet]. Chicago: American Medical Association; c1995-2015. Principles of medical ethics; [cited 2015 Sep 10]. Available from: http://www.ama-assn.org/ama/pub/physician-resources/medical-ethics/code-medical-ethics/principles-medical-ethics.page.

10.

American Medical Association [Internet]. Chicago: American Medical Association; c1995-2015. AMA history timeline; [cited 2015 Sep 10]. Available from: http://www.ama-assn.org/ama/pub/about-ama/our-history/ama-history-timeline.page.

11.

O'Day

D.M.

, Ladden

M.R.

The influence of Derrick T. Vail Sr, MD, and Edward M. Jackson, MD, on the creation of the American Board of Ophthalmology and the Specialist Board System in the United States. Arch Ophthalmol. 2012 Feb; 130(2): 224–32. PMID: 22332217. http://dx.doi.org/10.1001/archophthalmol.2011.353.

12.

American Board of Medical Specialties [Internet]. Chicago: American Board of Medical Specialties; c2015. ABMS history of improving quality care; [cited 2015 Sep 10]. Available from: http://www.abms.org/about-abms/history/.

13.

Hawkins

R.E.

, Lipner

R.S.

, Ham

H.P.

American Board of Medical Specialties Maintenance of Certification: theory and evidence regarding the current framework. J Contin Educ Health Prof. 2013 Fall; 33 Suppl 1: S7–19. PMID: 24347156. http://dx.doi.org/10.1002/chp.21201.

14.

MOC 10-Year Cycle Requirements Summary [Internet]. Tampa: American Board of Pathology; 2014 Dec [cited 2015 Jan 15]. Available from: http://www.abpath.org/images/pdf_4.2.1.pdf.

15.

Johnson

R.L.

The American Board of Pathology's Maintenance of Certification program update. Arch Pathol Lab Med. 2014 Apr; 138(4): 512–7. PMID: 24678682. http://dx.doi.org/10.5858/arpa.2013-0362-ra.

16.

ABP maintenance of certification booklet of information 2013 [Internet]. Tampa: American Board of Pathology; 2013 Sep [cited 2015 Sep 10]. 13 p. Available from: https://wds.dataharborsolutions.com/ABPathOrg/documents/MOCBofI.pdf.

17.

American Board of Pathology (Tampa, FL). MOC program policy change announcement [Internet]. Message to: ABP members. 2014 Nov 24 [cited 2015 Sep 30].11/24/2014.

18.

ABP booklet of information [Internet]. Tampa: American Board of Pathology; 2015 Feb [cited 2015 Sep 10]. Available from: http://www.abpath.org/BookletofInformation.pdf.

19.

American Board of Internal Medicine [Internet]. Philadelphia: American Board of Internal Medicine; c2004-2015. ABIM announces immediate changes to MOC program. 2015 Feb 3 [cited 2015 Sep 22]. Available from: http://www.abim.org/news/abim-announces-immediate-changes-to-moc-program.aspx.

20.

Shor

Addressing the maintenance of certification crisis calls for working together. J Am Coll Cardiol. 2015 Jul 28; 66(4): 478–80. PMID: 26205600. http://dx.doi.org/10.1016/j.jacc.2015.06.009.

21.

Lowes

ABIM says MOC 10-year exam may be on its way out. Medscape Med News [Internet]. 2015 Sep 16 [cited 2015 Sep 22]. Available from: http://www.medscape.com/viewarticle/851103?src=wnI_edit_newsal&uac=31277AG&impID=826529&faf=1#vp_1.

22.

Lowes

Anesthesiology board drops 10-year-exam for MOC. Medscape Med News [Internet]. 2015 Sep 14 [cited 2015 Sep 22]. Available from: http://www.medscape.com/viewarticle/850954#vp_1.

23.

Eichenwald

The ugly civil war in American medicine. Newsweek [Internet]. 2015 Mar 10 [cited 2015 Apr 2]. Available from: http://www.newsweek.com/2015/03/27/ugly-civil-war-american-medicine-312662.html.

24.

Eichenwald

A certified medical controversy. Newsweek [Internet]. 2015 Apr 7 [cited 2015 Apr 9]. Available from: http://www.newsweek.com/certified-medical-controversy-320495.

25.

Nichols

D.G.

ABP outlines MOC changes & long-term direction. American Board of Pediatrics Blog. 2015 Feb 25 [cited 2015 Sep 22]. Available from: https://blog.abp.org/blog/abp-outlines-moc-changes-long-term-direction.

26.

Counselman

F.L.

, Carius

M.L.

, Kowalenko

The American Board of Emergency Medicine maintenance of certification summit. J Emerg Med. 2015 Nov; 49(5): 722–8. PMID: 26375809. http://dx.doi.org/10.1016/j.jemermed.2015.06.058.

27.

Segall

ABNM maintenance of certification: getting it right. J Nucl Med. 2015 Jun; 56(6): 19N. PMID: 26033626.

28.

American Board of Medical Specialties [Internet]. Chicago: American Board of Medical Specialties; c2015. Value of board certification to health care; [cited 2015 Apr 24]. Available from: http://www.abms.org/board-certification/value-of-board-certification-to-health-care/.

29.

American Board of Medical Specialties [Internet]. Chicago: American Board of Medical Specialties; c2015. Specialists reflect on the value of certification; [cited 2015 Apr 24]. Available from: http://www.abms.org/board-certification/support-for-board-certification-and-abms-moc/specialists-reflect-on-the-value-of-certification/.

30.

Maintenance of certification: a Q&A session with the American Board of Pathology. Acad Forensic Pathol. 2015 Jun; 5(2): xiii–xvii.

31.

American Board of Medical Specialties [Internet]. Chicago: American Board of Medical Specialties; c2015. ABMS evidence library; [cited 2015 Apr 24]. Available from: http://evidencelibrary.abms.org/.

32.

Sharp

L.K.

, Bashook

P.G.

, Lipsky

M.S.

Specialty board certification and clinical outcomes: the missing link. Acad Med. 2002 Jun; 77(6): 534–42. PMID: 12063199. http://dx.doi.org/10.1097/00001888-200206000-00011.

33.

Norcini

J.J.

, Kimball

H.R.

, Lipner

R.S.

Certification and specialization: do they matter in the outcome of acute myocardial infarction?

Acad Med. 2000 Dec; 75(12): 1193–8. PMID: 11112721. http://dx.doi.org/10.1097/00001888-200012000-00016.

34.

Norcini

J.J.

, Lipner

R.S.

, Kimball

H.R.

Certifying examination performance and patient outcomes following acute myocardial infarction. Med Educ. 2002 Sep; 36(9): 853–9. PMID: 12354248. http://dx.doi.org/10.1046/j.1365-2923.2002.01293.x.

35.

Silber

J.H.

, Kennedy

S.K.

, Even-Shoshan

Anesthesiologist board certification and patient outcomes. Anesthesiology. 2002 May; 96(5): 1044–52. PMID: 11981141. http://dx.doi.org/10.1097/00000542-200205000-00004.

36.

Prystowsky

J.B.

, Bordage

, Feinglass

J.M.

Patient outcomes for segmental colon resection according to surgeon's training, certification, and experience. Surgery. 2002 Oct; 132(4): 663–70; discussion 670-2. PMID: 12407351. http://dx.doi.org/10.1067/msy.2002.127550.

37.

Pham

H.H.

, Schrag

, Hargraves

J.L.

, Bach

P.B.

Delivery of preventive services to older adults by primary care physicians. JAMA. 2005 Jul 27; 294(4): 473–81. PMID: 16046654. http://dx.doi.org/10.1001/jama.294.4473.

38.

Chen

, Rathore

S.S.

, Wang

Physician board certification and the care and outcomes of elderly patients with acute myocardial infarction. J Gen Intern Med. 2006 Mar; 21(3): 238–44. PMID: 16637823. PMCID: PMC1828098. http://dx.doi.org/10.1111/j.1525-1497.2006.00326.x.

39.

Turchin

, Shubina

, Chodos

A.H.

Effect of board certification on antihypertensive treatment intensification in patients with diabetes mellitus. Circulation. 2008 Feb 5; 117(5): 623–8. PMID: 18212279. http://dx.doi.org/10.1161/circulationaha.107.733949.

40.

McKenna

R.W.

, Davey

D.D.

, Johnson

R.L.

Maintenance of certification for pathologists: what we know so far. Am J Clin Pathol. 2005 Dec; 124 Suppl: S5–7. PMID: 16468413. http://dx.doi.org/10.1309/uqjdwcjgk6032d2g.

41.

Hammond

M.E.

, Filling

C.M.

, Neumann

A.R.

Addressing the maintenance of certification challenge: the College of American Pathologists response. Arch Pathol Lab Med. 2005 May; 129(5): 666–75. PMID: 15859640.

42.

Davey

D.D.

, Bennett

B.D.

Maintenance of certification: how will it affect pathology practice?

Arch Pathol Lab Med. 2007 Sep; 131(9): 1334–6. PMID: 17824786.

43.

Bennett

B.D.

, Grimes

M.M.

Certification and maintenance of certification: updates from the American Board of Pathology. Hum Pathol. 2011 Jun; 42(6): 770–3. PMID: 21315409. http://dx.doi.org/10.1016/j.humpath.2010.11.007.

44.

Levinson

, Holmboe

E.S.

Maintenance of certification: 20 years later. Am J Med. 2011 Feb; 124(2): 180–5. PMID: 21295199. http://dx.doi.org/10.1016/j.amjmed.2010.09.019.

45.

Lipner

R.S.

, Hess

B.J.

, Phillips

R.L.

Jr. . Specialty board certification in the United States: issues and evidence. J Contin Educ Health Prof. 2013 Fall; 33 Suppl 1: S20–35. PMID: 24347150. http://dx.doi.org/10.1002/chp.21203.

46.

Hess

B.J.

, Weng

, Lynn

L.A.

Setting a fair performance standard for physicians' quality of patient care. J Gen Intern Med. 2011 May; 26(5): 467–73. PMID: 21104453. PMCID: PMC3077491. http://dx.doi.org/10.1007/s11606-010-1572-x.

47.

Simpkins

, Divine

, Wang

Improving asthma care through recertification: a cluster randomized trial. Arch Intern Med. 2007 Nov 12; 167(20): 2240–8. PMID: 17998498. http://dx.doi.org/10.1001/archinte.167.20.2240.

48.

O'Neill

T.R.

, Puffer

J.C.

Maintenance of certification and its association with the clinical knowledge of family physicians. Acad Med. 2013 Jun; 88(6): 780–7. PMID: 23619076. http://dx.doi.org/10.1097/acm.0b013e3182900de4.

49.

Galliher

J.M.

, Manning

B.K.

, Petterson

S.M.

Do professional development programs for Maintenance of Certification (MOC) affect quality of patient care?

J Am Board Fam Med. 2014 Jan-Feb; 27(1): 19–25. PMID: 24390882. http://dx.doi.org/10.3122/jabfm.2014.01.130109.

50.

Holmboe

E.S.

, Wang

, Meehan

T.P.

Association between maintenance of certification examination scores and quality of care for medicare beneficiaries. Arch Intern Med. 2008 Jul 14; 168(13): 1396–403. PMID: 18625919. http://dx.doi.org/10.1001/archinte.168.13.1396.

51.

Hess

B.J.

, Weng

, Holmboe

E.S.

, Lipner

R.S.

The association between physicians' cognitive skills and quality of diabetes care. Acad Med. 2012 Feb; 87(2): 157–63. PMID: 22189883. http://dx.doi.org/10.1097/acm.0b013e31823f3a57.

52.

Gray

B.M.

, Vandergrift

J.L.

, Johnston

M.M.

Association between imposition of a Maintenance of Certification requirement and ambulatory care-sensitive hospitalizations and health care costs. JAMA. 2014 Dec 10; 312(22): 2348–57. PMID: 25490325. http://dx.doi.org/10.1001/jama.2014.12716.

53.

Hayes

, Jackson

J.L.

, McNutt

G.M.

Association between physician time-unlimited vs time-limited internal medicine board certification and ambulatory patient care quality. JAMA. 2014 Dec 10; 312(22): 2358–63. PMID: 25490326. http://dx.doi.org/10.1001/jama.2014.13992.

54.

Teirstein

Debate on maintenance of certification [Internet]. San Diego: National Board of Physicians and Surgeons; 2015 Feb 26 [cited 2015 Sep 20]. Available from: https://nbpas.org/.

55.

American Board of Pathology [Internet]. Tampa: American Board of Pathology; [cited 2015 Sep 20]. Available from: http://abpath.org/.

56.

Facebook [Internet]. Menlo Park (CA): Facebook Inc; c2015. National Board of Physicians and Surgeons; [cited 2015 Sep 20]. Available from: https://www.facebook.com/nationalboardofphysiciansandsurgeons.

57.

American Board of Internal Medicine [Internet]. Philadelphia: American Board of Internal Medicine; c2004-2015. Cardiovascular disease; [cited 2015 Sep 10]. Available from: http://www.abim.org/specialty/cardiovascular-disease.aspx.

58.

American Board of Surgery [Internet]. Philadelphia: American Board of Surgery; [cited 2015 Sep 10]. Available from: http://www.absurgery.org.

59.

First-time taker pass rates - maintenance of certification [Internet]. Philadelphia: American Board of Internal Medicine; 2015 [cited 2015 Sep 26]. 1 p. Available from: http://www.abim.org/pdf/pass-rates/moc.pdf.

60.

Johnson, Rebecca (Chief Executive Officer, American Board of Pathology, Tampa, FL). Data on MOC exam for forensic pathologists [Internet]. Message to Laura D. Knight 2015 Sep 15 [cited 2015 Sep 30].