Secondary effects of an opioid-focused academic detailing program on non-opioid controlled substance prescribing in primary care

Abstract

Background

Educational outreach programs that focus on safe opioid prescribing and awareness of state prescription monitoring programs may modify clinicians’ prescribing behavior. The objective of this study was to evaluate the secondary effects of an opioid-focused academic detailing (AD) program on non-opioid controlled substance prescribing in primary care.

Methods

A quasi-experimental pre-post study of primary care clinicians exposed and unexposed to the AD program was conducted using data from the Illinois Prescription Monitoring Program from December 2017 to February 2019. Outcomes were mean monthly prescriptions for benzodiazepines (BZD), non-BZD sedative-hypnotics, and carisoprodol, per clinician. A difference-in-differences (DID) approach utilizing repeated-measures mixed-effects linear regression models was used to compare changes in outcomes six-months before and after the program.

Results

Mean monthly BZD prescriptions declined in both groups of clinicians (AD-exposed n = 151; controls n = 399) after implementation of the AD program. Although the mean monthly number of BZD prescriptions decreased in both groups after the AD program, BZD prescribing in the AD-exposed group declined at a slower rate following the AD program (DID = 0.73; 95% CI: 0.14, 1.31). The AD-exposed group had a 0.06 (95% CI: −0.11, −0.01) lower rate of mean monthly carisoprodol prescriptions compared to the control group following the AD program. There was no change in the rate of mean monthly non-BZD sedative-hypnotic prescriptions between the two groups.

Conclusions

The higher relative rate of BZD prescribing in the AD-exposed group compared to the control group following the AD program may be reflective of an unintended consequence of opioid-focused AD programs as clinicians learn to be cautious about opioid prescribing. Our findings may suggest the need for incorporation of targeted education on appropriate BZD prescribing into opioid-focused AD programs as a featured component. These findings warrant further consideration and investigation before large-scale implementation of opioid-focused educational outreach programs.

Keywords

Opioids academic detailing primary care benzodiazepines

Introduction

Evidence-based approaches to educational outreach, particularly academic detailing (AD) programs, are intended to improve medical decision-making.¹ AD uses specially trained personnel (i.e., detailers) to deliver current unbiased, evidence-based information via one-on-one, face-to-face visits with clinicians.² The Centers for Disease Control and Prevention (CDC) has endorsed AD as an evidence-based approach to combat the opioid crisis by supplementing clinicians knowledge on safe opioid prescribing.³ Prior studies have evaluated the impact of AD among clinicians in primary care settings to modify opioid prescribing behavior and utilization of state prescription monitoring programs (PMP) that collect data on the dispensing of federally controlled substances.⁴ –8 These studies found AD was associated with improved opioid guideline adherence,⁴^,⁵ reduced high-dose opioid prescribing,⁶ and increased PMP utilization.⁷^,⁸

In 2016, the CDC released the Guideline for Prescribing Opioids for Chronic Pain to provide evidence-based guidance to clinicians on safe opioid prescribing practices.⁹ AD programs that incorporate recommendations from the CDC Guideline may help clinicians learn to become more prudent about appropriate opioid prescribing. Key recommendations from the CDC Guideline include routinely reviewing information in state PMPs and avoiding concurrent prescribing of opioids and benzodiazepines (BZD). State PMPs can be used to assess high-risk patient behavior such as receipt of high opioid dosages or concurrent use of opioids and BZD which is associated with an increased risk for opioid overdose.¹⁰ As clinicians begin to use PMPs more consistently they may be more aware of a patient's history of prescription opioid and non-opioid controlled substance use.

Use of BZD and other select non-opioid controlled substances such as non-BZD sedative-hypnotics (i.e., eszopiclone, zaleplon, zolpidem, and zopiclone; the Z-drugs) and carisoprodol is associated with increased risks of abuse and overdose death.¹¹^,¹² BZD and non-BZD sedative-hypnotic utilization have increased in the past two decades due to more chronic use for anxiety and insomnia despite limited evidence to support their long-term use.¹³ Increased abuse and misuse of carisoprodol have been noted reasons for its federally regulated classification as a schedule IV controlled substance.¹⁴ Similar to carisoprodol, BZD and non-BZD sedative-hypnotics are also schedule IV controlled substances and are reported in state PMPs. Thus, AD programs focused on safe opioid prescribing that also include an educational component on PMP use may impact the prescribing of both opioids and non-opioid controlled substances. The potential secondary effects of opioid-focused AD programs on prescribing of non-opioid controlled substances remain relatively unexplored despite the continued implementation of educational outreach programs to improve opioid prescribing among clinicians amidst the opioid crisis. Therefore, the objective of this study was to evaluate the secondary effects of an opioid-focused AD program on non-opioid controlled substance prescribing in primary care.

Methods

Design

This was a quasi-experimental pre-post study conducted to evaluate the secondary effects of an opioid-focused AD program on prescribing of BZDs, non-BZD sedative-hypnotics, and carisoprodol in primary care. The AD program was comprised of a larger overall initiative to evaluate the impact of AD on opioid prescribing behavior in primary care which has been published elsewhere.¹⁵ This current study utilized prescribing data from the Illinois Prescription Monitoring Program to evaluate changes in mean monthly prescribing of BZD, non-BZD sedative-hypnotic, and carisoprodol. Primary care clinicians who received an opioid-focused AD program were compared to a control group of primary care clinicians who did not receive the AD program. The AD program was delivered between June 2018 and August 2018. Changes in prescribing were compared six months before the AD program (December 2017–May 2018) and six months after the conclusion of the AD program (September 2018–February 2019) between both clinician groups. The Office for the Protection of Human Subjects at the University of Illinois at Chicago (UIC) approved the study, and informed consent was obtained from all study participants who received the AD program. Clinicians were not compensated for study participation.

Academic detailing program

The AD program was implemented through a partnership with a large health system that provides primary care services throughout the Chicago metropolitan area and its surrounding suburbs from June 2018 to August 2018. The program was endorsed by the health system's chief medical officer and medical director of pain management. Clinicians who specialized in internal medicine or family medicine were encouraged to participate. Research staff from UIC were provided contact information by the health system to schedule voluntary visits with clinicians. The AD visits consisted of one-on-one, in-person interactions between clinicians and academic detailers conducted at the health system's immediate care/walk-in clinics. The duration of AD visits averaged approximately 15 min. During each visit, the detailers discussed information related to safe and appropriate opioid prescribing and tailored the interaction based on the needs of the clinician. The content delivered in each visit (i.e., first and second visit) was similar and included information on the CDC Guideline and tailored metrics on the clinician's past opioid prescribing. Six key messages from the CDC Guideline were selected as the focus of the detailing visits. These messages were: (1) Check the PMP for high opioid dosages and prescriptions from other clinicians, (2) Avoid concurrent BZD and opioid prescribing, (3) Use non-opioid treatments as first-line or routine therapy for chronic pain, (4) Start low and go slow when using opioids, (5) Incorporate strategies to mitigate risk factors for opioid-related harms, and (6) Offer treatment for opioid use disorder. As part of the visit, each clinician was provided with information on their opioid prescribing behavior as reflected in the PMP in the past six months that included: (1) average (i.e., mean) number of monthly PMP queries, (2) average number of monthly opioid prescriptions, (3) the number and percentage of total opioids prescribed categorized by daily morphine milligram equivalent (MME) thresholds of <50 MME/day, 50–89 MME/day, and ≥90 MME/day, (4) average days supply/opioid prescription, (5) average daily MME/opioid prescription, and (6) average monthly number of patients co-prescribed opioids and BZDs.

The AD program was delivered by academic detailers who comprised of 8 first-year and second-year doctor of pharmacy students and 2 licensed pharmacists from the UIC College of Pharmacy. Detailers were selected to participate based on their interest in delivering the AD program. Participation by the detailers was voluntary and no prior formal training in AD was required for detailers to participate. The detailers were specially trained by research staff who had completed formal training provided by the National Resource Center for Academic Detailing.

Sample

Clinicians were eligible for an AD visit if they were licensed healthcare clinicians with opioid prescriptive authority (i.e., Doctor of Medicine [MD] and Doctor of Osteopathy [DO], nurse practitioners [NP], and physician assistants [PA]) who practiced in primary care, specifically family medicine or internal medicine. Resident physicians and pediatric specialties were excluded. We considered all clinicians who received at least one of two planned AD visits as exposed to the program (AD-exposed). To account for secular changes in opioid prescribing behavior and similar prescribing habits between groups, a control group of clinicians who also practiced primary care and specialized in family medicine or internal medicine at other large health systems in the Chicagoland area was selected and identified from publicly available information on their health system websites. Clinicians in the control group were not recruited or contacted by research staff nor did they receive AD visits during the study period. Baseline characteristics of the clinicians receiving AD (i.e., sex, clinician specialty, clinician type, and years of practice) were collected during the AD visit. Baseline characteristics for each clinician in the control group were collected via publicly available information (e.g., health system websites).

Outcomes

The three outcomes measured in this study were the mean monthly number of BZD, non-BZD sedative-hypnotics (i.e., eszopiclone, zaleplon, zolpidem, and zopiclone), and carisoprodol prescriptions, per clinician. These outcomes were measured during the pre-AD program period (December 2017–May 2018) and compared with outcomes in the post-AD program period (September 2018–February 2019) between the AD-exposed and control groups.

Analysis

Characteristics of the AD-exposed and control groups were compared at baseline with the chi-square test for categorical variables. For continuous variables, Student's t-tests were used to compare differences in means. A difference-in-differences (D-I-D) approach was used to compare pre-post changes in outcomes between the AD-exposed and control group.¹⁶ Repeated measures mixed-effects linear regression models, which are a commonly used analytic approach in D-I-D analyses, were used to account for within-subject clustered and longitudinal data while concurrently adjusting for the amount of correlation within each subject. For each model, an unstructured covariance structure and a random effect for each clinician to account for the correlation of outcomes within each clinician. Model fit statistics, such as Akaike Information Criterion (AIC), were used to assess model fit with smaller statistic values indicating a better fit of the model to the data. Baseline characteristics including sex, clinician specialty, clinician type, and years of practice were adjusted for in the model to account for any differential influence or confounding on the prescribing outcomes. The model included main effect terms for binary indicators of AD exposure (yes or no) and time (pre-AD program or post-AD program) and an interaction between the main effects. The β coefficient for the model interaction term represents the D-I-D estimate for the AD program's effect on the outcomes. More simply, the D-I-D estimate represented the pre-post difference in the prescribing outcomes between the AD-exposed group and the control group. For each outcome, a subgroup analysis was conducted where the population was restricted to clinicians who prescribed at least one non-opioid controlled substance in the pre-AD program period as the AD program would not be expected to impact clinicians who did not prescribe in the pre-AD program period. A 95% confidence interval (CI) was presented for each β coefficient in the D-I-D analyses. A two-sided p-value < 0.05 was considered statistically significant. SAS version 9.4 (SAS Institute Inc.: NC, Cary) was used to perform statistical analyses.

Results

A total of 550 clinicians, 151 in the AD-exposed group and 399 in the control group, were included in the analysis (Table 1). Clinicians were primarily physicians (90.4%), specialized in internal medicine (58.5%), and practiced for a mean of 20 years. Mean years of practice were higher in the control group compared to the AD-exposed group (21 vs. 18, p < 0.01). Compared to the control group, a higher proportion of AD-exposed clinicians were NP or PA (17.2% vs. 6.8%, p < 0.01) and specialized in family medicine (76.2% vs. 28.3% p < 0.01).

Table 1

Baseline characteristics of clinicians.

Characteristics	Overall		AD-exposed		Control		p
Total clinicians, n	550		151		399
Sex, n (%)
Female	286	(52.0%)	88	(58.3%)	198	(49.6%)	0.07
Male	264	(48.0%)	63	(41.7%)	201	(50.4%)	0.07
Years of practice, mean (SD)	20	(11)	18	(11)	21	(11)	<0.01
Clinician type, n (%)
MD	423	(76.9%)	87	(57.6%)	336	(84.2%)	<0.01
DO	74	(13.5%)	38	(25.2%)	36	(9.0%)
NP	34	(6.2%)	18	(11.9%)	16	(4.0%)
PA	19	(3.5%)	8	(5.3%)	11	(2.8%)
Clinician specialty, n (%)
Family medicine	228	(41.5%)	115	(76.2%)	113	(28.3%)	<0.01
Internal medicine	322	(58.5%)	36	(23.8%)	286	(71.7%)	<0.01

Table 2 shows there were no pre-post differences in the mean monthly number of BZD prescriptions per clinician within the AD-exposed group (24.10 vs. 22.08, p = 0.08), however, the mean monthly number of BZD prescriptions per clinician was significantly lower in the post-AD program period within the control group (21.94 vs. 19.19, p < 0.01). The pre-post mean monthly number of non-BZD sedative-hypnotic prescriptions per clinician within the AD-exposed group (8.39 vs. 8.25, p = 0.75) and within the control group (8.20 vs. 8.02, p = 0.51) were not significantly different. The mean monthly number of carisoprodol prescriptions per clinician was significantly lower in the post-AD program period within the AD-exposed group (0.32 vs. 0.22, p < 0.01) and within the control group (0.27 vs. 0.23, p = 0.04).

Table 2

Summary of pre-post AD program mean monthly prescribing statistics and difference-in differences (D-I-D) results for AD-exposed vs. control.

	AD-exposed			Control
	Pre	Post	^Δ	Pre	Post	^Δ	D-I-D	(95% CI)	D-I-D restricted subgroups	(95% CI)
BZD prescriptions	24.10	22.08	−2.02	21.94	19.19	−2.75^a	0.73^a	(0.14, 1.31)	1.10^a	(0.44, 1.75)
Non-BZD sedative-hypnotic prescriptions	8.39	8.25	−0.14	8.20	8.02	−0.18	0.04	(−0.22, 0.31)	0.08	(−0.24, 0.41)
Carisoprodol prescriptions	0.32	0.22	−0.10^a	0.27	0.23	−0.04	−0.06^a	(−0.11, −0.01)	−0.21^a	(−0.34, −0.06)

p-Value < 0.05.

Δ:

represents the pre-post difference in the prescribing outcomes within the AD-exposed group and the control group. D-I-D: represents the pre-post difference in the prescribing outcomes between the AD-exposed group and the control group. D-I-D restricted subgroups: represents the pre-post difference in the prescribing outcomes between the AD-exposed group and the control group among clinicians with non-opioid controlled substance prescribing data in the pre-AD program period.

Although the mean monthly number of BZD prescriptions decreased in both groups after the AD program, in the main D-I-D analyses BZD prescribing in the AD-exposed group declined at a slower rate following the AD program by 0.73 (95% CI: 0.14, 1.31) (Table 2). There was no significant change (0.04 [95% CI: −0.22, 0.31]) in the pre-post mean monthly rate of non-BZD sedative-hypnotic prescriptions between the AD-exposed and control groups following the AD program. Although infrequently prescribed in both groups before and after the AD program, the rate of mean monthly carisoprodol prescriptions was marginally lower by 0.06 (95% CI: −0.11, −0.01) in the AD-exposed group compared to the control group following the AD program.

When restricting the D-I-D analyses to clinicians who prescribed at least one BZD prescription in the pre-AD program period, the mean monthly number of BZD prescriptions in the AD-exposed group declined at a slower rate following the AD program by 1.10 (95% CI: 0.44, 1.75) in the AD-exposed group (N = 143) compared to the control group (N = 334). Among clinicians who prescribed at least one non-BZD sedative-hypnotic prescription in the pre-AD program period, there was no significant change (0.08 [95% CI: −0.24, 0.41]) in the pre-post mean monthly rate of non-BZD sedative-hypnotic prescriptions between the AD-exposed (N = 135) and control groups (N = 307). Lastly, among clinicians who prescribed at least one carisoprodol prescription in the pre-AD program period, the rate of mean monthly carisoprodol prescriptions was marginally lower by 0.21 (95% CI: −0.34, −0.06) in the AD-exposed group (N = 50) compared to the control group (N = 124) following the AD program.

Discussion

This study explored the extent to which an opioid-focused AD program with a PMP educational component had secondary effects on non-opioid controlled substance prescribing in primary care. The mean monthly number of BZD prescriptions was lower in both the AD-exposed and control groups in the period after the AD program was administered, but the rate of decline in the AD-exposed group was slower by almost one BZD prescription per month per clinician. There were no meaningful changes in non-BZD sedative-hypnotic between the AD-exposed and control groups. A statistical difference, though marginal, was found between the two groups in carisoprodol prescribing. These results were consistent when restricting to only those clinicians who prescribed at least one non-opioid controlled substance in the pre-AD program period. The findings of the study suggest that opioid-focused AD programs may have secondary effects on the prescribing of non-opioid controlled substances outside of opioids.

The change in BZD prescribing after the AD program in the AD-exposed group compared to the control group was unexpected. The rate of BZD prescribing was greater by nearly one prescription per month in the AD-exposed group relative to the control group following the AD program. While this higher rate may seem inconsequential on the individual clinician level, based on the range of the 95% CI as few as 250 to more than 2,300 additional BZD prescriptions would be dispensed annually among this sample of 151 clinicians. Interestingly, a concurrent study among the same group of clinicians found that 1.48 (95% CI: −2.48, −0.47) fewer total opioid prescriptions were dispensed per month per clinician among AD-exposed clinicians who expressed a willingness to change their opioid prescribing behavior compared to AD-exposed clinicians who did not express a willingness to change their opioid prescribing behavior after the AD program.¹⁷ The contrasting relative changes in the rates of opioid and BZD prescribing after the AD program may be suggestive of a compensatory shift to BZD prescribing triggered by opioid-focused AD programs.

Growing evidence has continued to emerge on the lack of significant benefit opioids provide in managing chronic and musculoskeletal pain conditions relative to non-opioids like non-steroidal anti-inflammatory drugs and acetaminophen.¹⁸^,¹⁹ Due to the increased awareness of the harms associated with opioids, clinicians may divert their prescribing to drug classes with a similar lack of benefit and potential harms of their own, such as BZD.²⁰ This notion may be supported by recent data on disparate national prescribing trends which highlighted increased BZD use for musculoskeletal pain conditions and declining opioid use.²¹^,²² These trends in prescribing may be suggestive of a shift from problematic opioid prescribing toward problematic BZD prescribing. The higher relative rate in BZD prescribing among the AD-exposed group compared to the control group following the AD program may be suggestive of a potential shift toward BZD as the prescribing of opioids for pain management continues to wane among clinicians. However, a potential explanation for this shift toward BZD could be possibly explained by clinicians recognizing a need to treat anxiety that was previously being treated with opioids. Our findings suggest opioid-focused AD programs should be cognizant of this potential unintended secondary effect on the prescribing of non-opioid controlled substances like BZD while also further probing into the prescribing rationale of BZD if possible. Consideration of this unanticipated effect on BZD prescribing is warranted before large scale implementation of AD programs on safe opioid prescribing.

Our findings may also support the growing calls to action to address overuse and overprescribing of BZD.²³ Opioid-focused AD programs can be leveraged as an opportunity to intervene on both opioids and BZD prescribing. BZD-focused AD programs have been associated with reductions in BZD, although demonstrated among prescribers who care for the veteran population.²⁴ In terms of policy initiatives at the health system level, delivery of focused education on evidence-based guidelines for BZD prescribing, risks of BZD use, and alternative pain management options may be important components to integrate into opioid-focused AD programs to help address inappropriate prescribing of BZD. Additionally, incorporation of clear, actionable guidance into opioid-focused AD programs on how to use the data found in state PMPs may help to facilitate informed clinical decision-making. Other policy initiatives at a state level could include the development and integration of guideline-concordant recommendations for safe opioid and BZD prescribing directly into state PMP databases.⁹ These recommendations may help to facilitate clinicians’ decision-making when reviewing the history of a patient's controlled substance use to determine the medical necessity and appropriateness of opioid or non-opioid controlled substance prescriptions.

These results should be interpreted in the context of several limitations. AD-exposed clinicians received the program whether they prescribed controlled substances in the pre-AD program period or not. By including clinicians who did not prescribe controlled substances in the pre-AD program period, the impact of the AD program on prescribing of non-opioid controlled substances may have been attenuated. To account for this, we conducted subgroup analyses including only those clinicians who prescribed at least one non-opioid controlled substance in the pre-AD program period. The findings from our study may not be generalizable to other health systems in other geographical areas in the United States since the clinicians used for this study were from large health systems in the Chicago metropolitan area. Although we are unable to obtain patient characteristics of those who received prescriptions from clinicians in the AD-exposed and control groups, we do not expect any potential differences in patient characteristics to impact study findings since the provision of primary care services were the principal focus for each clinician to be included in the study. While the control group clinicians were selected based on primary care specialty and geographic location, selection bias may have been present due to the differences in baseline characteristics between the two primary care groups. Although baseline characteristics were dissimilar between the two groups, the impact of the baseline characteristics on prescribing outcomes was not expected to be differential between the pre and post-AD program periods due to their immutable and time-invariant nature. Unmeasured differences between the AD-exposed group and control group may have affected the impact of the opioid-focused AD program on non-opioid controlled substance prescribing. Additionally, the control group's health systems’ focus or messaging on benzodiazepines could also have produced trends in benzodiazepine prescribing that contributed to the difference between the AD-exposed group and the control group. Lastly, this study was limited to selected primary care specialties of family medicine and internal medicine, thus impacting generalizability to other clinician specialties.

In conclusion, a concerning secondary effect of an opioid-focused AD program may be a compensatory shift toward higher BZD prescribing as clinicians become more careful regarding opioid prescribing. Our findings may suggest the need for the incorporation of targeted education on appropriate BZD prescribing into opioid-focused AD programs as a featured component. Further investigation of these findings is warranted before the widespread implementation of opioid-focused educational outreach programs.

Footnotes

Acknowledgments

The authors thank and acknowledge the clinicians who participated in this study and the following individuals for their contributions: Mary Smart, Sarette Tilton, Aleksandrina Ruseva, Dayna Redini, Esther Lee, Nevena Varagic, Shannon Menard, Victoria Kulbokas, Ammarah Nadeem who participated as academic detailers from the University of Illinois at Chicago College of Pharmacy; Darin Jordan, MD, Ankur Dave, MD, Reinhold Llerena, MD from AMITA Health; Craig Berberet, Stanley Murzynski, Edward Dowllar, Andrew Hollo from the Illinois Prescription Monitoring Program; Dejan Jovanov from the Illinois Department of Public Health; Jamie Mells, PhD, Wes Sargent, EdD from the Centers for Disease Control and Prevention.

Authors’ contributions

C.D.S., A.S.P., and T.A.L. contributed to the research conception and design; C.D.S., A.S.P., S.P., and T.A.L. contributed to the acquisition of data; C.D.S. contributed to the analysis of data; C.D.S., S.Y.C., M.A.F., A.S.P., L.K.S., and T.A.L. contributed to the interpretation of results; C.D.S., S.Y.C., M.A.F., A.S.P., L.K.S., and T.A.L. contributed to the writing/revisions of the manuscript.

Funding

This research was funded in part by the Centers for Disease Control and Prevention [Grant #1U17CE002739-01]. The funding organization had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

References

Avorn

, Soumerai

SB.

Improving drug-therapy decisions through educational outreach. A randomized controlled trial of academically based “detailing”. N Engl J Med. 1983;308(24):1457–1463.

Soumerai

, Avorn

Principles of educational outreach (‘academic detailing') to improve clinical decision making. JAMA. 1990;263(4):549–556.

Carroll

, Green

, Noonan

RK.

Evidence-based strategies for preventing opioid overdose: what's working in the United States: an introduction for public heath, law enforcement, local organizations, and others striving to serve their community. https://stacks.cdc.gov/view/cdc/59393. Published 2018.

Lasser

, Shanahan

, Parker

, . A multicomponent intervention to improve primary care provider adherence to chronic opioid therapy guidelines and reduce opioid misuse: a cluster randomized controlled trial protocol. J Subst Abuse Treat. 2016;60:101–109.

Liebschutz

, Xuan

, Shanahan

, . Improving adherence to long-term opioid therapy guidelines to reduce opioid misuse in primary care: a cluster-randomized clinical trial. JAMA Intern Med. 2017;177(9):1265–1272.

Kattan

, Tuazon

, Paone

, . Public health detailing-a successful strategy to promote judicious opioid analgesic prescribing. Am J Public Health. 2016;106(8):1430–1438.

Larson

, Browne

, Nikitin

, . Physicians report adopting safer opioid prescribing behaviors after academic detailing intervention. Subst Abus. 2018;39(2):218–224.

Barth

, Ball

, Adams

, . Development and feasibility of an academic detailing intervention to improve prescription drug monitoring program use among physicians. J Contin Educ Health Prof. 2017;37(2):98–105.

Dowell

, Haegerich

, Chou

CDC guideline for prescribing opioids for chronic pain–United States, 2016. JAMA. 2016;315(15):1624–1645.

10.

Sun

, Dixit

, Humphreys

, Darnall

, Baker

, Mackey

Association between concurrent use of prescription opioids and benzodiazepines and overdose: retrospective analysis. BMJ. 2017;356:j760.

11.

Parsaik

, Mascarenhas

, Khosh-Chashm

, . Mortality associated with anxiolytic and hypnotic drugs-a systematic review and meta-analysis. Aust N Z J Psychiatry. 2016;50(6):520–533.

12.

Horsfall

, Sprague

. The pharmacology and toxicology of the 'holy trinity'. Basic Clin Pharmacol Toxicol. 2017;120(2):115–119.

13.

Kaufmann

, Spira

, Depp

, Mojtabai

Long-term use of benzodiazepines and nonbenzodiazepine hypnotics, 1999-2014. Psychiatr Serv. 2018;69(2):235–238.

14.

Reeves

, Burke

, Kose

. Carisoprodol: update on abuse potential and legal status. South Med J. 2012;105(11):619–623.

15.

Saffore

, Tilton

, Crawford

, . Identification of barriers to safe opioid prescribing in primary care: a qualitative analysis of field notes collected through academic detailing. Br J Gen Pract. 2020;70(697):e589–e597.

16.

Dimick

, Ryan

AM.

Methods for evaluating changes in health care policy: the difference-in-differences approach. JAMA. 2014;312(22):2401–2402.

17.

Saffore

, Pickard

, Crawford

, Fischer

, Sharp

, Lee

TA.

Practice change intentions after academic detailing align with subsequent opioid prescribing. J Am Pharm Assoc. 2020;60(6):1001–1008.

18.

Krebs

, Gravely

, Nugent

, . Effect of opioid vs nonopioid medications on pain-related function in patients with chronic back pain or hip or knee osteoarthritis pain: the SPACE randomized clinical trial. JAMA. 2018;319(9):872–882.

19.

Busse

, Wang

, Kamaleldin

, . Opioids for chronic noncancer pain: a systematic review and meta-analysis. JAMA. 2018;320(23):2448–2460.

20.

Friedman

, Irizarry

, Solorzano

, . Diazepam is no better than placebo when added to naproxen for acute low back pain. Ann Emerg Med. 2017;70(2):169–176.e161.

21.

Agarwal

, Landon

BE.

Patterns in outpatient benzodiazepine prescribing in the United States. JAMA Netw Open. 2019;2(1):e187399.

22.

Schieber

, Guy

Jr. , Seth

, . Trends and patterns of geographic variation in opioid prescribing practices by state, United States, 2006–2017. JAMA Netw Open. 2019;2(3):e190665

23.

Lembke

, Papac

, Humphreys

Our other prescription drug problem. N Engl J Med. 2018;378(8):693–695.

24.

Bounthavong

, Lau

, Popish

, . Impact of academic detailing on benzodiazepine use among veterans with posttraumatic stress disorder. Substance Abuse. 2020;41(1):101–109.