Abstract
A pocket guide on management of substance use during pregnancy was developed by a group of Canadian care providers. One hundred and fifteen family medicine residents in 6 Canadian teaching sites were randomized to receive either the pocket guide or a paper summary on similar clinical topics, based on UpToDate, a comprehensive Web-based resource. At baseline, both groups completed a survey containing questions on beliefs, attitudes, experience, and training on pregnancy and substance use. Participants then answered 28 multiple choice questions about substance use in pregnancy, using either the pocket guide or UpToDate. Finally participants were asked to rate ease of use for the 2 resources. The results showed that
Introduction
Pregnant women who drink alcohol may be more receptive to screening and counseling than than nonpregnant drinkers (2, 3), and controlled trials have shown that women of child-bearing age will reduce their drinking in response to brief counseling interventions by physicians (1). However, research suggests that physicians do not routinely screen for alcohol use in pregnant women (4), and usually do not refer pregnant substance users to drug treatment programs (5). Medical school and residency curricula may in part be responsible for the low rate of identification and management. Medical students, residents, and physicians have inadequate knowledge about substance use in pregnancy (6). The amount of curricular time spent learning about substance use in medical school is grossly inadequate compared to the time devoted to problems of similar prevalence (7). Yet controlled trials have found that educational interventions are effective in improving physicians’ skills (8).
One strategy to help overcome the knowledge gap is to give residents immediate access to clinical information when assessing a patient with a substance use problem. Pocket guides provide concise and practical summaries of clinical knowledge and protocols for use in a discrete area of medicine. They are easy and inexpensive to produce, can be regularly updated when new evidence is available, and may be more convenient as a point-of-care tool than textbooks or computer programs. Pocket guides appear to be widely used by physicians despite having received little attention in knowledge transfer research. Two pre-post studies demonstrated that the use of pocket guides resulted in improvements in knowledge and self-reported behavior (9, 10).
The Pregnancy-Related Issues in the Management of Addictions (PRIMA) project, a national collaboration of perinatal providers with expertise in problematic substance use in pregnancy, developed a pocket guide to help clinicians who had little experience in this area. Now in its fourth edition, the PRIMA pocket guide has been distributed to over 20,000 physicians, midwives, nurses, and allied health professionals across Canada.
The objective of this study was to compare the knowledge and satisfaction of residents who use the pocket guide with those who use printed material from “UpToDate On-Line.” UpToDate was chosen as a comparator because it is one of the most popular and easy to use comprehensive medical information programs.
Methods
Intervention
Pocket Guide Development
The PRIMA team first developed a set of learning objectives for clinicians caring for pregnant substance users. The learning objectives were based in part on a series of evidence-based best practices (BPR) documents developed by PRIMA members. The group then reviewed a draft of the guide over several iterations until consensus was reached. The final draft was then reviewed by external experts from across Canada, including obstetricians, family physicians, midwives, nurses, and other academics. The printed guide, a 39-page 3 × 5-inch laminated booklet, begins with a brief review of emergency management. Specific drug classes are then reviewed, including alcohol, tobacco, opioids, benzodiazepines, cannabis, cocaine, and hallucinogens. Maternal, obstetrical, neonatal, and longer-term developmental effects are listed for each class of drug. Clinical topics include screening and early detection, management of maternal and neonatal withdrawal, breastfeeding, and pharmacotherapy. Special sections are included on infectious disease management, urine drug screening, counseling, and legal issues. The guide can be viewed online at www.addictionpregnancy.ca, and a printed copy can be requested by e-mailing
UpToDate
This Web program is widely used by physicians throughout North America. The investigators searched “UptoDate” for topics relevant to pregnancy and substance use, and collated an 88-page binder, based on topic headings similar to those of the pocket guide. The binder has a table of contents and color-coded tabs for ease of reference.
Instruments Used
Baseline Survey
A baseline survey asked about demographics, clinical experience, and educational exposure, as well as confidence, outcome expectations, and sense of responsibility in managing substance use in pregnancy. The questions were adapted from questions on similar medical surveys (11).
Knowledge Test
Twenty-eight multiple choice questions (MCQs) were developed, based on the PRIMA learning objectives. A first draft of the questions was created at a national meeting, and further refined by experts until consensus was achieved. Several external experts reviewed the MCQs to ensure that they addressed the PRIMA learning objectives and were appropriate for family medicine residents. For most questions, a short, case-based stem was presented, followed by 4 distracters and 1 correct answer. The questions were pretested on 10 family medicine residents and staff.
User Satisfaction Survey
After completing the 28 MCQs, residents were asked to rate ease of use for either UpToDate or the pocket guide, using a 6-point likert-type scale.
Ethics
Approval was obtained from the research ethics boards of participating universities. Informed consent was obtained from each subject. Participation was voluntary and individual results were only viewed by the research assistant. Residents received a pocket guide at the end of the session.
Recruitment
The residency program directors and chief residents in the family medicine departments at 3 medical schools in British Columbia, Alberta, and Ontario agreed to resident participation in this study. Residents were recruited at core days at individual hospitals. A PRIMA member described the study to the residents and obtained informed consent.
Sample Size
We calculated our sample size to detect a difference in knowledge scores of at least 20%. Assuming that residents’ mean knowledge score at baseline is 60%, a sample of 36 per group was needed, setting significance level at .05 and power (double-sided) at .80 (12).
Setting and Randomization
The trial was conducted at 6 family medicine teaching practices in Toronto, Edmonton, and Vancouver. Residents at each site completed the baseline survey. The subjects were then randomized to either the pocket guide or UpToDate group by picking an object at random out of an envelope. Students were given approximately 1 hour to complete the baseline survey and the MCQ test. They were then given the survey on ease of use.
Data Analysis
Demographic variables were tabulated. The proportion of correct responses on the MCQs was calculated for each group, and results were compared through independent t tests. Independent variables that could influence between-group differences were entered into a regression analysis (e.g., age, gender, program, and site).
Results
Subjects
One hundred and fifteen residents participated in the study, including 51 in Alberta, 39 in Ontario, and 25 in Vancouver. The overall participation rate was 90%. The mean age was 31 years old, with 29% being male. Fifty-three percent were first-year residents and 47% were second-year residents. There were no significant differences between the 2 groups in age (P = .574), sex (P = .704), or year of residency (P = .784).
Experience and Training
Approximately 70% of residents had provided care for a pregnant substance user at least once over the past year, and 53% had attended a lecture or seminar on substance use in pregnancy. The 2 groups did not differ significantly on any measure of experience or training.
Beliefs and Attitudes
Both groups were more confident managing alcohol and tobacco use during pregnancy (mean = 3.88, P = .784 and mean = 4.22, P = .384, respectively) than managing other drug use (mean = 2.79, P = .273). There were no significant differences between the 2 groups in their level of confidence.
Outcome Expectations and Role Responsibility
Outcome expectation is defined as the confidence that pregnant substance users will improve if the physician provides appropriate management. The UpToDate group had higher outcome expectations at baseline than the PRIMA group for drug use (3.70 vs. 3.20, P = .029) and smoking (4.46 vs. 3.90, P = .008). There were no significant differences for alcohol use. Both groups reported that, as family physicians, they had a responsibility to screen for substance use in pregnant women, but management was not their sole responsibility. There were no significant differences between the 2 groups in role responsibility.
Multiple Choice Questions
The PRIMA group scored higher than the UpToDate group overall and at each study site (61.27% vs. 42.86%, P < .001). In Vancouver, the PRIMA group's mean score was higher than the UpToDate score but the difference did not reach significance (P = .051). The PRIMA group scored significantly higher than the UpToDate group in all of the substance-specific questions (P < .001) except the benzodiazepine questions (P = .967).
Ease of Use
Overall, residents found the PRIMA laminate easier to use than UpToDate (mean = 2.73 vs. 4.36, P < .001). However, there were no statistically significant differences in ease of use ratings for residents in Vancouver. Overall, 98% rated the PRIMA laminate as user friendly (N = 55), in comparison to 44% for UpToDate (N = 50) (P < .001). Similarly, 96% of the PRIMA subjects reported that they would use the PRIMA laminate again (N = 53), compared to 78% for UpToDate (N = 49) (P = .005).
Discussion
One possible explanation for these results is that a comprehensive Web-based program may have only scattered and incomplete information for “orphan” disciplines such as substance use in pregnancy, making it impractical for point of care use. Comparatively, a quick reference pocket guide is based on information collected from many sources and synthesized by experts. However, the study had several limitations. The UpToDate subjects reported that the Web-based UpToDate resource was easier to use online than in print because of the “search” and “find” computer functions. However, we had essentially done the search and find process for the residents and printed out the results. Also, the PRIMA group designed the learning objectives, the pocket card, and the MCQs; this could have influenced the results. The group tried to minimize this by basing the MCQs on the PRIMA learning objectives, rather than on the contents of the pocket card. However, future comparison studies should consider using an independent group to develop the MCQs, if feasible. Finally, the study did not measure the utility of the pocket guide in improving residents’ actual clinical performance with pregnant substance users.
Conclusion
Family medicine residents using the PRIMA pocketbook had higher knowledge scores and found it easier to use than residents using UpToDate. The study suggests that concise clinical summaries are a more effective knowledge transfer strategy than comprehensive Web-based programs, at least for topics that span various medical specialties, such as substance use in pregnancy.
Footnotes
Acknowledgments
The study was funded by a grant from the Lawson foundation.