To answer the question, “does CME work?” by reviewing the effectiveness of continuing medical education (CME) and other related educational methods on objectively-determined physician performance and/or health care outcomes. These interventions include educational materials, formal, planned CME activities or programs, outreach visits such as academic detailing, opinion leaders, patient-mediated strategies, audit and feedback, reminders, or a combination of these strategies.
Methods:
MEDLINE, ERIC, NTIS, the Research and Development Resource Base in CME and other relevant data sources including review articles were searched for relevant terms, from 1975 to 1994. Of those articles retrieved, randomized controlled trials of educational strategies or interventions which objectively assessed physician performance and/or health care outcomes were selected for review. Data were extracted from each article about the specialty of the physician targeted, the clinical subject of the intervention, the setting and the nature of the educational method, and the presence or degree of needs assessment or barriers to change.
Results:
More than two-thirds of the studies (70%) displayed a change in physician performance, while almost half (48%) of interventions produced a change in health care outcomes. Community-based strategies such as academic detailing (and to a lesser extent, opinion leaders), practice-based methods such as reminders and patient-mediated strategies, and multiple interventions appeared to be most effective activities. Mixed results and weaker outcomes were demonstrated by audit and educational materials, while formal CME conferences without enabling or practice-reinforcing strategies, had relatively little impact.
Conclusion:
Strategies which enable and/or reinforce appear to “work” in changing physician performance or health care outcomes, a finding which has significant impact on the delivery of CME, and the need for further research into physician learning and change.
AmaraRMorrisonJISchmidG. Looking ahead at American health care. Institute for the Future, Healthcare Information Center, Washington, DC: McGraw-Hill, 1981.
2.
Roles for medical education in health care reform. Association of American Medical Colleges. Academic Medicine1994;69:512–515.
3.
PoriesWJSmoutJCMorrisALewkowVE. U.S. health care reforms: Will it change postgraduate surgical education? World Journal of Surgery1994;18:745–752.
4.
HaynesRBDavisDAMcKibbonATugwellP. A critical appraisal of the efficacy of continuing medical education. Journal of the American Medical Association1984;251(1):61–64.
5.
DavisDAThomsonMAOxmanADHaynesRB. Evidence for the effectiveness of CME. A review of 50 randomized controlled trials. Journal of the American Medical Association1992;268:1111–1117.
6.
LauJAntmanEMJiminez-SilvaJKupelnickBMostellerFChalmersTC. Cumulative meta-analysis of therapeutic trials for myocardial infarction. New England Journal of Medicine1992;327:248–254.
7.
JaySJAndersonJG. Continuing medical education and public policy in an era of health care reform. Journal of Continuing Education of Health Professions1993;13:195–209.
8.
OxmanADThomsonMAHaynesRBDavisDA. No magic bullets: A systematic review of 102 trials of interventions to help health care professionals deliver services more effectively and efficiently. Canadian Medical Association Journal1995; 153:1423–1431.
9.
BertramDABrooks-BertramPA. The evaluation of continuing medical education: A literature review. Health Education Monograph1977;5(4):330–362.
10.
BeaudryJS. The effectiveness of continuing medical education: A quantitative synthesis. Journal of Continuing Education in Health Professions1989;9:285–307.
11.
McLaughlinPJDonaldsonJF. Evaluation of continuing medical education programs: A selected literature, 1984–1988. Journal of Continuing Education in Health Professions1991;11(1):65–84.
DavisDAFoxRD, editors. The physician as learner: Linking research to practiceChicago: American Medical Association Press, 1994, xiii.
14.
AndersonFAJr.WheelerHBGoldbergRJHosmerDWForcierAPatwardhanNA. Changing clinical practice. Prospective study of the impact of continuing medical education and quality assurance programs on use of prophylaxis for venous thromboembolism. Archives of Internal Medicine1994;154:669–677.
15.
AvornJSoumeraiSB. Improving drug-therapy decisions through educational outreach: A randomized controlled trial of academically based “detailing.”New England Journal of Medicine1983;308(24):1457–1463.
16.
AvornJSoumeraiSBEverittDERoss-DegnanDBeersMHShermanDSalemSchatzSRFieldsD. A randomized trial of a program to reduce the use of psychoactive drugs in nursing homes. New England Journal of Medicine1992; 327:168–173.
17.
BarnettGO. A computer based monitoring system for follow-up of elevated blood pressure. Medical Care1983;21:400–409.
18.
BassMJMcWhinneyIRDonnerA. Do family physicians need medical assistants to detect and manage hypertension? Canadian Medical Association Journal1986; 134:1247–1255.
19.
BerwickDMColtinKL. Feedback reduces test use in a health maintenance organization. Journal of the American Medical Association1986;255:1450–1454.
20.
BilliJEHejnaGFWolfFMShapiroLRStrossJK. The effects of a cost-education program on hospital charges. Journal of General Internal Medicine1987;2:306–311.
21.
BrimberryR. Vaccination of high-risk patients for influenza. A comparison of telephone and mail reminder methods. Journal of Family Practice1988;26: 397–400.
22.
BrittonMLLurveyPL. Impact of medication profile review on prescribing in a general medicine clinic. American Journal of Public Health1991;48:265–270.
23.
BrownerWSBaronRBSolkowitzSAdlerLJGullionDS. Physician management of hypercholesterolemia. A randomized trial of continuing medical education. Western Journal of Medicine1994;161:572–578.
24.
BuntinxF. Does feedback improve the quality of cervical smears? A randomized controlled trial. British Journal of General Practice1993;43:194–198.
25.
CallahanCMHendrieHCDittusRSBraterDCHuiSLTierneyWM. Improving treatment of late life depression in primary care: A randomized clinical trial. Journal of the American Geriatrics Society1994;42:839–846.
26.
ChambersCVBalabanDJCarlsonBL. Microcomputer-generated reminders: Improving the compliance of primary care physicians with mammography screening guidelines. Journal of Family Practice1989;29:273–280.
27.
ChaseCR. Medical information management. Improving the transfer of research results to presurgical evaluation. Medical Care1983;21:410–424.
28.
ChassinMRMcCueSM. A randomized trial of medical quality assurance: Improving physicians' use of pelvimetry. Journal of the American Medical Association1986;256:1012–1016.
29.
CheneyCRamsdellJW. Effect of medical records' checklists on implementation of periodic health measures. American Journal of Medicine1987;83:129–136.
30.
CockburnJRuthDSilagyCDobbinMReidYScolloMNaccarellaL. Randomised trial of three approaches for marketing smoking cessation programmes to Australian general practitioners. British Medical Journal1992;304:691–694.
31.
CohenDIJonesPLittenbergBNeuhauserD. Does cost information availability reduce physician test usage? A randomized clinical trial with unexpected findings. Medical Care1982;20:286–292.
32.
CohenSJChristenAGKatzBPDrookCADavisBJSmithDMStookeyGK. Counseling medical and dental patients about cigarette smoking: The impact of nicotine gum and chart reminders. American Journal of Public Health1987;77:313–316.
33.
CohenSJWeinbergerMHuiSLTierneyWMMcDonaldCJ. The impact of reading on physicians' nonadherence to recommended standards of medical care. Social Science and Medicine1985;21:909–914.
34.
CowanJA. Effect of a fact sheet reminder on performance of the periodic health examination: A randomized controlled trial. American Journal of Preventive Medicine1992;8:104–109.
35.
CummingsSRCoatesTJRichardRJHansenBZahndEGVanderMartinRDuncanCGerbertBMartinASteinMJ. Training physicians in counselling about smoking cessation: A randomized trial of the “Quit for Life” program. Annals of Internal Medicine1989;110:640–647.
36.
CummingsSRRichardRJDuncanCLHansenBVanderMartinRGerbertBCoatesTJ. Training physicians about smoking cessation: A controlled trial in private practice. Journal of General Internal Medicine1989;4(6):482–489.
37.
DavidoffFGoodspeedRCliveJ. Changing test ordering behavior: A randomized controlled trial comparing probabilistic reasoning with cost-containment education. Medical Care1989;27:45–58.
38.
DickinsonJC. Improving hypertension control: Impact of computer feedback and physician education. Medical Care1981;19:843–854.
39.
DietrichAJ. Cancer: Improving early detection and prevention. A community practice randomised trial. British Medical Journal1992;304:687–691.
40.
EvansCEHaynesRBBirkettNJGilbertJRTaylorDWSackettDLJohnstoneMEHewsonSA. Does a mailed continuing education program improve physician performance? Results of a randomized trial in hypertensive care. Journal of the American Medical Association1986;255:501–504.
41.
EverettGDdeBloisCSChangPHoletsT. Effect of cost education, cost audits, and faculty chart review on the use of laboratory services. Archives of Internal Medicine1983;143:942–944.
42.
GehlbachSHWilkinsonWEHammondWEClappNEFinnALTaylorWJRodellMS. Improving drug prescribing in a primary care practice. Medical Care1984;22:193–201.
43.
GullionDSTschannJMAdamsonTECoatesTJ. Management of hypertension in private practice: A randomized controlled trial in continuing medical education. Journal of Continuing Education for Health Professions1988;8:239–255.
44.
HeadrickLA. Efforts to improve compliance with the National Cholesterol Education Program guidelines. Results of a randomized controlled trial. Archives of Internal Medicine1992;152:2490–2496.
45.
HealeJDavisDANormanGRWoodwardCNeufeldVRDoddP. A randomized controlled trial assessing the impact of problem based versus didactic teaching methods in CME. RIME Proceedings1988;27:72–77.
46.
HersheyCOGoldbergHICohenDI. The effect of computerized feedback coupled with a newsletter upon outpatient prescribing changes. Medical Care1988;26:88–93.
47.
HersheyCOPorterDKBreslauDCohenDI. Influence of simple computerized feedback on prescription charges in an ambulatory clinic. Medical Care1986;24:472–481.
48.
JennettPALaxdalOEHaytonRCKlanssenDJSwansonRWWilsonJWSpoonerRWMainprizeGWWickettRE. The effects of continuing medical education on family doctor performance in office practice: A randomized control study. Medical Education1988;22:139–145.
49.
KottkeTEBrekkeMLSolbergLIHughesJR. A randomized trial to increase smoking intervention by physicians. Doctors helping smokers, round 1. Journal of the American Medical Association1989;261:2101–2106.
50.
LevinsonW. The effects of two continuing medical education programs on communication skills of practicing primary care physicians. Journal of General Internal Medicine1993;8:318–324.
51.
LinnBS. Continuing Medical Education: Impact on emergency room burn care. Journal of the American Medical Association1980;244:565–570.
52.
LitzelmanDKDittusRSMillerMETierneyWM. Requiring physicians to respond to computerized reminders improves their compliance with preventive care protocols. Journal of General Internal Medicine1993;8:311–317.
53.
LomasJEnkinMAndersonGMHannahWJVaydaESingerJ. Opinion leaders vs. audit and feedback to implement practice guidelines. Delivery after previous cesarean section. Journal of the American Medical Association1991;265:2202–2207.
54.
Magruder-HabibKZungWWKFeussnerJR. Improving physicians' recognition and treatment of depression in general medical care: Result from a randomized clinical trial. Medical Care1990;28:239–250.
55.
MaimanLABeckerHLiptakGSNazarianLFRoundsKA. Improving pediatricians' compliance-enhancing practices. American Journal of Diseases of Children1988;142:773–779.
56.
ManheimLMFeinglassJHughesRMartinGJConradKHughesEF. Training house officers to be cost conscious. Effects of an educational intervention on changes and length of stay. Medical Care1990;28:29–42.
57.
MartinHR. A trial of two strategies to modify the test-ordering behavior of medical residents. New England Journal of Medicine1980;303:1330–1336.
58.
MartonKITulVSoxHC. Modifying test-ordering behavior in the outpatient medical clinic. Archives of Internal Medicine1985;145:816–821.
59.
MayefskyJHFoyeHR. Use of a chart audit: Teaching well child care to paediatric house officers. Medical Education1993;27:170–174.
60.
MazzucaSAVinicorFEinterzRMTierneyWMNortonJAKalasinskiLA. Effects of the clinical environment on physicians' response to postgraduate medical education. American Education and Research Journal1990;27:473–488.
61.
McAlisterNHCovveyHDTongCLeeAWigleED. Randomised controlled trial of computer assisted management of hypertension in primary care. British Medical Journal (Clinical Research Education)1986;293:670–674.
62.
McConnellTSCushingAHBankhurstADHealyJLMcIlvennaPASkipperBJ. Physician behavior modification using claims data: Tetracycline for upper respiratory infection. Western Journal of Medicine1982;137:448–450.
63.
McDonaldCJ. Use of a computer to detect and respond to clinical events: Its effect on clinician behavior. Annals of Internal Medicine1976;84:162–167.
64.
McDonaldCJ. Protocol-based computer reminders, the quality of care and the non perfectibility of man. New England Journal of Medicine1976;295:1351–1355.
65.
McDonaldCJHuiSLSmithDMTierneyWMCohenSJWeinbergerMMcCabeGP. Reminders to physicians from an introspective computer medical record. Annals of Internal Medicine1984;100:130–138.
66.
McDonaldCJWilsonGAMcCabeGP. Physician response to computer reminders. Journal of the American Medical Association1980;244:1579–1581.
67.
McDowellINewellCRosserW. Computerized reminders to encourage cervical screening in family practice. Journal of Family Practice1989;28:420–424.
68.
McPheeSJBirdJAFordhamDRodnickJEOsbornEH. Promoting cancer prevention activities by primary care physicians. Journal of the American Medical Association1991;266:538–544.
69.
McPheeSJBirdJAJenkinsCNFordhamD. Promoting cancer screening. A randomized, controlled trial of three interventions. Archives of Internal Medicine1989;149:1866–1872.
70.
MeyerTJVan KootenDMarshSProchazkaAV. Reduction of polypharmacy by feedback to clinicians. Journal of General Internal Medicine1991;6:133–135.
71.
MorganM. Computerized concurrent review of prenatal care. Quality Review Bulletin1978;4:33–36.
72.
NortonPGDempseyLJ. Self-audit: Its effect on quality of care. Journal of Family Practice1985;21:289–291.
73.
PalmerRHLouisTAHsuLPetersonHFRothrockJKStrainRThompsonMSWrightEA. A randomized controlled trial of quality assurance in sixteen ambulatory care practices. Medical Care1985;23:751–770.
74.
PereraDRLoGerfoJPShulenbergerEYlvisakerJTKirzHL. Teaching sigmoidoscopy to primary care physicians: A controlled study of continuing medical education. Journal of Family Practice1983;16:785–788.
75.
PierceMLundySPalanisamyAWinningSKingJ. Prospective randomised controlled trial of methods of call and recall for cervical cytology screening. British Medical Journal1989;299:160–162.
76.
PinkertonRE. Resident physician performance in a continuing education format: Does newly acquired information improve patient care? Journal of the American Medical Association1980;244:2183–2185.
77.
PollackMMGetsonPR. Pediatric critical care cost containment: Combined actuarial and clinical program. Critical Care Medicine1991;19:12–20.
78.
ProzenMWD'AgostinoRBMitchellJBRosenfeldDMGuglielminoJTSchwartzMLTeebagyNValentineJMWoodWBJr., The usefulness of a predictive instrument to reduce inappropriate admissions to the coronary care unit. Annals of Internal Medicine1980;92:238–242.
79.
PozenMWD'AgostinoRBSelkerHPSytkowskiPAHoodWBJr., A predictive instrument to improve coronary-care-unit admission practices in acute ischemic heart disease. A prospective multicenter clinical trial. New England Journal of Medicine1984;310:1273–1278.
80.
PughJAFrazierLMDeLongEWallaceAGEllenbogenPLinforsE. Effect of daily charge feedback on inpatient charges and physician knowledge and behavior. Archives of Internal Medicine1989;149:426–429.
81.
PutnamRWCurryL. Impact of patient care appraisal on physician behaviour in the office setting. Canadian Medical Association Journal1985;132:1025–1029.
82.
PutnamRWCurryL. Physicians' participation in establishing criteria for hypertension management in the office: Will patient outcomes be improved? Canadian Medical Association Journal1989;140:806–809.
83.
RabinDLBoekelooBOMarxESBowmanMARussellNKWillisAG. Improving office-based physician's prevention practices for sexually transmitted diseases. Annals of Internal Medicine1994;121:513–519.
84.
RaischDWBootmanJLLarsonLNMcGhanWF. Improving antinuclear agent prescribing in a health maintenance organization. American Journal of Hospital Pharmacy1990;47:1766–1773.
85.
RestucciaJD. The effect of concurrent feedback in reducing inappropriate hospital utilization. Medical Care1982;20:46–62.
86.
RogersJL. Medical information systems: Assessing impact in areas of hypertension, obesity, and renal disease. Medical Care1982;20:63–74.
87.
RosserWWMcDowellINewellC. Use of reminders for preventive procedures in family medicine. Canadian Medical Association Journal1991;145:807–814.
88.
RubensteinLVCalkinsDRYoungRTClearyPDFinkAKosecoffJJetteAMDaviesARDelbancoTLBrookRH. Improving patient function: A randomized trial of functional disability screening. Annals of Internal Medicine1989;111:836–842.
89.
SanazaroPJ. Concurrent quality assurance in hospital care. Report of a study in private initiative in PSRO. New England Journal of Medicine1978;298:1171–1177.
90.
SibleyJCSackettDLNeufeldVGerrardBRudnickKVFraserW. A randomized trial of continuing medical education. New England Journal of Medicine1982;306:511–515.
91.
SoumeraiSB. A controlled trial of educational outreach to improve blood transfusion practice. Journal of the American Medical Association1993;270:961–966.
92.
SteeleMABessDTFranseVLGraberSE. Cost effectiveness of two interventions for reducing outpatient prescribing costs. DICP—Annals of Pharmacotherapy1989;23:497–500.
93.
StressJK. Evaluation of a continuing education program in rheumatoid arthritis. Arthritis and Rheumatism1980;23:846–849.
94.
StressJKBoleGG. Evaluation of an educational program for primary care practitioners, on the management of osteoarthritis. Arthritis and Rheumatism1985;28:108–111.
95.
StressJKHissRGWattsCMDavisWKMacDonaldR. Continuing education in pulmonary disease for primary-care physicians. American Review of Respiratory Diseases1983;127:739–746.
96.
SulmasyDPGellerGFadenRLevineDM. The quality of mercy: Caring for patients with ‘do not resuscitate’ orders. Journal of the American Medical Association1992;267:682–686.
97.
SzczepuraAWilmotJDaviesCFletcherJ. Effectiveness and cost of different strategies for information feedback in general practice. British Journal of General Practice1994;44:19–24.
98.
TapeTGCampbellJR. Computerized medical records and preventive health care: Success depends on many factors. American Journal of Medicine1993;94:619–625.
99.
TierneyWMHuiSLMcDonaldCJ. Delayed feedback of physician performance versus immediate reminders to perform preventive care. Medical Care1986;24: 659–666.
100.
TierneyWMMcDonaldCJHuiSLMartinDK. Computer predictions of abnormal test results: Effects on outpatient testing. Journal of the American Medical Association1988;259:1194–1198.
101.
TierneyWMMillerMEMcDonaldCJ. The effect on test ordering of informing physicians of the charges for outpatient diagnostic tests. New England Journal of Medicine1990;322:1499–1504.
102.
TurnerRCWaiversLEO'BrienK. The effect of patient-carried reminder cards on the performance of health maintenance measures. Archives of Internal Medicine1990;150:645–647.
103.
VinicorFCohenSJMazzucaSAMoormanNWheelerMKueblerTSwansonSOursPFinebergSEGordonEE. Diabeds: A randomized trial of the effects of physician and/or patient education on diabetes patient outcomes. Journal of Chronic Disease1987;40:345–356.
104.
WeingartenMABazelDShannonHS. Computerized protocol for preventive medicine: A controlled self-audit in family practice. Family Practice1989;6:120–124.
105.
WexlerJR. Impact of a system of computer-assisted diagnosis. American Journal of Diseases of Childhood1975;129:203–205.
106.
WhiteCWAlbaneseMABrownDDCaplanRM. The effectiveness of continuing medical education in changing the behaviour of physicians caring for patients with acute myocardial infarction. Annals of Internal Medicine1985;102:686–692.
107.
WhitePTPharoahCAAndersonHRFreelingP. Randomized controlled trial of small group education on the outcome of chronic asthma in general practice. Journal of the Royal College of General Practice1989;39:182–186.
108.
WigtonRSZimmerJLWigtonJHPatilKD. Chart reminders in the diagnosis of anemia. Journal of the American Medical Association1981;245:1745–1747.
109.
WilsonDMTaylorWGilbertJRBestJALindsayEAWillmsDGSingerJ. A randomized trial of a family physician intervention for smoking cessation. Journal of the American Medical Association1988;260:1570–1574.
110.
WinickoffRNColtinKLMorganMMBuxbaumRCBarnettGO. Improving physician performance through peer comparison feedback. Medical Care1984;22:527–534.
111.
WinickoffRNWilnerSNeisulerRBarnettGO. Limitation of provider interventions in hypertension quality assurance. American Journal of Public Health1985;75:43–46.
112.
WonesRG. Failure of low-cost audits with feedback to reduce laboratory test utilization. Medical Care1987;25:78–82.
113.
American Medical Association. Physician characteristics and distribution in the U.S.A., Chicago, January 1, 1995.
114.
GreenLKreuterMDeedsSPartridgeK. Health education planning: A diagnostic approach, Palo Alto, CA: Mayfield Press, 1980.
115.
RogersEM. Diffusion of innovation, New York: The Free Press, 1983.
116.
NowlenPM. A new approach to continuing education for business and the professions, New York: Macmillan, 1988.
117.
BanduraA. Social foundations of thought and action: A social cognitive theory, Englewood Cliffs, NJ: Prentice-Hall, 1986.
118.
ThomsonMAFreemantleNWolfFDavisDA, Oxman Associate Dean. Educational meetings, workshops and preceptorships to improve the practice of health professionals and health care outcomes. [Review for the] Cochrane Collaboration on Effective Professional Practice (CCEPP), 1997.
119.
FoxRDMazmanianPEPutnamRW. Changing and learning in the lives of physicians, New York: Praeger, 1989.
120.
ProchaskaJOVelicerWFRossiJSGoldsteinMGMarcusBHRakowskiWFioreCHarlowLLReddingCARosenbloomDRossiSR. Stages of change and decisional balance for 12 problem behaviors. Health Psychology1994;13:39–46.
121.
American Medical Association. The CME DirectoryChicago, IL: AMA Publication, 1994.
