Residents’ Experience of Screening,Brief Intervention,and Referral to Treatment (SBIRT) as a Clinical Tool following Practical Application: A Mixed-Methods Study

Abstract

Background

Screening, brief intervention, and referral to treatment (SBIRT), an evidence-based validated system for providing early detection and brief treatment of substance use disorders, has been widely used in the training of medical residents across specialties at a number of sites. This article investigates the effectiveness of SBIRT training during short-term follow-up at Albany Medical Center, one of the initial Substance Abuse and Mental Health Services Administration (SAMHSA) grantees.

Methods

Training outcomes were measured by training satisfaction following opportunities to apply SBIRT skills in clinical work, the rate at which these techniques were applied in clinical work, and the degree to which residents felt that the SBIRT training provided skills that were applicable to their practice. We examined differences in learning experience by postgraduate year and by program, and conducted a qualitative analysis in a convergent parallel mixed-methods design to elucidate barriers encountered by residents upon using SBIRT techniques in clinical practice.

Results

Residents remained highly satisfied with the training at 4-month follow-up, with 80.1% reporting that they had used SBIRT skills in their clinical work. Use of SBIRT techniques was high at 6-month follow-up as well, with 85.9% of residents reporting that they regularly screened their patients for substance use, 74.4% reporting that they had applied brief intervention techniques, and 78.2% indicating that SBIRT training had made them overall more effective in helping patients with substance use issues. Differences in application rates and satisfaction were found by specialty. Qualitative analyses indicated that residents encountered patient readiness and specific contextual factors, such as time constraints, externally imposed values, and clinical norms, as barriers to implementation.

Conclusions

Despite encountering obstacles such as time constraints and patient readiness, residents utilized many of the skills they had learned during SBIRT training in clinical practice and reported finding these skills useful in their management of patients with substance use issues.

Keywords

Brief intervention residents SBIRT training substance use substance use disorder

Introduction

Screening, brief intervention, and referral to treatment (SBIRT) training models are designed to address a need for improving physician education on the identification and treatment of substance use issues. Several residency training programs have initiated SBIRT training through Substance Abuse and Mental Health Services Administration (SAMHSA) grants and have demonstrated good outcomes. Albany Medical College instituted SBIRT medical residency training after receiving grant funding to create, initiate, and make sustainable a program for training residents with the goal of improving their ability to screen and identify at-risk use and to refer to treatment or perform a brief motivational interview–based intervention when indicated.¹

To date, several studies have shown SBIRT training to be generally well received by residents who give the training high satisfaction and utility ratings.¹–3 Trained residents demonstrated increased knowledge of SBIRT principles and addiction facts following training³,⁴ and have shown improved attitudes towards substance use intervention.⁵,⁶ Consistent with previous research,⁷ however, residents in our recent study⁸ were slightly less confident in their skills than expected at the completion of training; only 74% felt that they would be effective in applying SBIRT principles in their clinical work immediately following training.

Despite generally positive findings in the period immediately following training, less research is available on longer-term outcomes. Given the somewhat lower effectiveness scores provided by residents, it is important to examine their ability and interest in applying SBIRT principles in clinical practice. Ryan et al.³ found that pediatric residents remained satisfied following training, with over half conducting at least 1 brief intervention (BI) in their clinical work at a 9-month follow-up. Tetrault et al.¹ found similar rates of application, with 338 BIs performed during clinical practice within 30 days of training 199 residents. Indeed, implementation rates seem to hover around 50%, with about 50% of trained residents applying SBIRT skills with 10%–50% of their patients at follow-up.⁴ –7 Despite improved knowledge and confidence scores in the time period immediately following training, Schram et al.⁹ found that resident knowledge and confidence in addressing substance use with their patients decreased significantly over time. They concluded that booster sessions would likely benefit resident self-confidence and skill in application long term.

Clearly, the goals of training are not only to increase use of SBIRT techniques in regular practice but also to disseminate knowledge regarding the relevance of substance use identification across specialties within the field of medicine. The purpose of the current study was to examine follow-up data at 4 and 6 months post training. Four goals were pursued: (1) to assess the degree to which satisfaction with training was maintained after residents had a chance to practice SBIRT independently in their clinical work; (2) to examine the degree to which residents felt that the SBIRT training provided skills that were applicable to their practice; (3) to identify any barriers that may make the clinical application of SBIRT difficult for residents; and (4) to learn from residents what modifications could be made to training to make it more applicable to their clinical work. To meet this aim, we used a convergent parallel mixed-methods design by examining quantitative data by postgraduate year and by specialty, and then conducting a qualitative analysis of narrative data collected from residents who were asked about the barriers they have encountered while utilizing SBIRT in practice.

Methods

Participants

Participants were 204 of 255 (80%) residents who completed SBIRT training between 2010 and 2013 as part of residency program requirements. Residents from Internal Medicine (n = 53), Pediatrics (n = 30; including Medicine/Pediatrics combined program, n = 4), Emergency Medicine (n = 34), Ob/Gyn (n = 26), Psychiatry (n = 40), and Family Medicine (n = 21) participated in the current study. The study was approved by the Albany Medical College Institutional Review Board. Consent was obtained from all participants after the study procedures had been fully explained. In a previous study,⁸ these residents were shown to be highly satisfied with training immediately following its completion (90.6% reported feeling satisfied with the overall training). They demonstrated a significant increase in knowledge scores (t = 4.82, P < .001, d = .51) and found all components of the training to be useful in their learning.

Quantitative materials and procedure

Government Performance and Results Act (GPRA) follow-up measure

The GPRA measure was created by the Center for Substance Abuse Treatment to assess satisfaction with several aspects of grant-funded training programs. The GPRA follow-up consists of 13 items scored on a 5-point scale (1 = Very satisfied/Strongly agree to 5 = Very dissatisfied/Strongly disagree), 3 yes/no questions, and 1 item (“How useful was the information you received?”) scored on a 4-point scale (1 = Very useful to 4 = Useless). The GPRA tool was administered to residents approximately 4 months (mean = 124 days) after the completion of the standardized patient protocol.

SBIRT in practice survey

This survey is an 8-item questionnaire, administered 6 months after training, which asks residents to rate several statements regarding ways training has affected their current practice. Items are rated on a 5-point scale ranging from 1 = Strongly disagree to 5 = Strongly agree. The survey also includes 2 open-ended items with a text box for completing one's responses. These are “Please list any barriers you have encountered in utilizing SBIRT” and “What information would be helpful for future trainings?” These responses were used for the qualitative analysis described below. Of the 204 residents who participated, 78 completed this survey. Of those, 25 were in postgraduate year 1 (PGY1) at the time of training, 33 in PGY2, 14 in PGY3, and 6 were in PGY4. Twenty-five were from Psychiatry, 12 from Emergency Medicine, 11 from Internal Medicine, 11 from Ob/Gyn, 9 from Family Medicine, 7 from Pediatrics, and 3 from Medicine/Pediatrics. Two-proportion z-tests indicated significant differences in response rates for 2 programs completing the practice survey compared with the GPRA; these were Psychiatry (20% GPRA vs. 32% practice survey; z = 2.22, P = .03) and Internal Medicine (26% GPRA vs. 14% practice survey; z = 2.13, P = .03). No other differences were found for response rates by program.

Qualitative methods

Qualitative study participants

Of the 78 residents who completed the 6-month follow-up questionnaire, 44 completed the open-ended question regarding barriers encountered in utilizing SBIRT. Twenty-nine participants completed the open-ended question regarding information that would be helpful for future trainings.

Qualitative analysis

Two coders, neither of whom were involved in the quantitative analysis, analyzed the qualitative data using thematic analysis.¹⁰ As “a method for identifying, analysing and reporting patterns (themes) within data,”¹⁰^(p79) thematic analysis is a method that can be used flexibly but has clearly delineated steps, increasing the rigor of the analysis. In line with Braun and Clarke's¹⁰ recommendation, the coders made the following decisions prior to the data analysis: (a) What counts as a “theme” would be determined mostly by prevalence; however, the coders agreed to remain open to themes that seemed important, regardless of prevalence within the data set; (b) the analysis would focus on providing a rich thematic description of the entire data set rather than a detailed account of one aspect of the data; (c) the analysis of themes would be inductive rather than theory-driven; (d) the analysis would remain mostly on the semantic rather than latent level of the data; however, the coders acknowledged that all interpretive efforts may involve some latent reading of the data; and (e) in line with the research questions, the analysis of the qualitative data stayed within an essentialist/realist paradigm rather than a more constructivist paradigm.

The analysis was composed of 8 steps, as delineated by Braun and Clarke¹⁰: (a) the coders familiarized themselves with the data by reading the entire data set and noting any initial thoughts; (b) the coders generated initial codes independently; (c) the coders met and discussed the codes, noted any discrepancies and reached consensus around all of the codes; (d) the coders collated data relevant to each code, making sure the codes were supported by the data; (e) the coders independently collated codes into potential themes; (f) the coders met to discuss the themes and reached consensus around the themes, finalizing the name and definition for each theme, making sure that each theme captured the collated codes; (g) the coders generated a thematic map of the analysis; and (h) the coders selected extracts from the data that served as examplars for each theme, ensuring that the themes accurately captured the data. Throughout the analytic process and in line with qualitative analytic procedures, ¹¹ the coders kept memos as a way to keep track of and develop ideas as well as to raise awareness of any potential biases that may influence the data. Notable reactions or thoughts were discussed by the coders.

Results

Quantitative analysis

Follow-up satisfaction ratings

Scores on the GPRA showed that 81.9% of residents were “Satisfied” or “Very satisfied” with the overall quality of the training experience after a 4-month follow-up period. The mean satisfaction score was 1.91, which falls between “Satisfied” and “Very satisfied,” and the mode was 2 (“Satisfied”) (Table 1). The “Quality of instruction” and “Quality of materials” were likewise rated as satisfactory, with means of 1.83 and 1.90, respectively, and modes of 2 (“Satisfied”) for each aspect of training.

Table 1

Resident GPRA satisfaction ratings of training experience at follow-up: means, modes, standard deviations, cumulative percentages, and ANOVA on differences among programs.

		Very satisfied/Strongly agree	Satisfied/Agree
Item	N	n %	n %	Cumulative %	Mean	Mode	SD	F	P
Overall quality	204	58 28.4	109 53.4	81.9	1.91^a	2.00	.71	6.76	.000
Quality of instruction	204	65 31.9	111 54.4	86.3	1.83^a	2.00	.69	4.87	.000
Quality of materials	204	58 28.4	114 55.9	84.3	1.90^a	2.00	.72	6.07	.000
How satisfied with training experience	204	59 28.9	111 54.4	83.3	1.92^a	2.00	.77	7.10	.000
Training was relevant to SA treatment	204	90 44.1	98 48.0	92.2	1.67^b	2.00	.72	3.90	.002
Material has been useful to me in dealing with substance abuse	204	67 32.8	94 46.1	78.9	1.95^b	2.00	.87	6.93	.000
Training enhanced my skills	204	77 37.7	92 45.1	82.8	1.82^b	2.00	.77	4.62	.001
Training was relevant to my career	203	76 37.4	100 49.3	86.7	1.84^b	2.00	.89	7.68	.000
Training has enabled me to serve my clients better	204	68 33.3	88 43.1	76.5	1.99^b	2.00	.93	7.68	.000
I would recommend to colleague	204	66 32.4	84 41.2	73.5	2.04^b	2.00	.98	7.59	.000
How useful was info received?	191	56 29.3	104 54.5	83.8	1.90^c	2.00	.73	5.34	.000

Note. “Cumulative %” represents the percentage of trainees reporting that they Agree (Are satisfied) or Strongly agree (Are very satisfied) with the question presented.

1 = Very satisfied to 5 = Very dissatisfied;

1 = Strongly agree to 5 = Strongly disagree;

1 = Very useful to 4 = Useless. Analysis of variance revealed significant differences in mean ratings on all items by program.

In order to identify whether differences on satisfaction ratings existed by program, we used an analysis of variance (ANOVA) to compare mean scores across groups. The ANOVA revealed significant differences across groups on each satisfaction rating on the GPRA (Table 1). Post hoc comparisons, using Tukey's honest significance test, showed that Psychiatry residents rated the overall training satisfaction significantly higher than all other programs except Pediatrics, with mean ratings of 1.43 and 1.80, respectively. Mean satisfaction scores by all other programs ranged from 2.00 (Ob/Gyn) to 2.24 (Emergency Medicine).

Utility

After completing training and having time to practice SBIRT independently, a majority of residents (78.9%) agreed that SBIRT materials had been useful to them in working with substance use, and 76.5% stated that SBIRT training had enabled them to “Better serve their clients” (Table 1). Even more residents indicated that SBIRT training “Enhanced their skills” (82.8%), and 86.7% felt that SBIRT training was “Relevant to their career.” In terms of clinical utility, 83.8% reported that the “Information provided in the training was useful to them in general” and “Relevant to the treatment of substance abuse” (92.2%).

An ANOVA revealed significant differences across groups on all training items, with F-scores ranging from 3.90 to 7.68. Of particular interest were differences on 3 items: “Relevance to career,” “Better serve my clients,” and “Overall usefulness of training.” On “Relevance to career,” Psychiatry residents rated this item significantly higher (1.28) than did those in Emergency Medicine (2.36), Ob/Gyn (2.08), and Internal Medicine (2.00). Residents in Emergency Medicine rated “Relevance to career” significantly lower than did Pediatrics (1.63) and Psychiatry. No other group differences were found on this item.

When asked if SBIRT training enabled residents to “Better serve their clients,” Psychiatry residents were again found to rate this item significantly higher (1.48) than did those in Emergency Medicine (2.59), Family Medicine (2.38), and Ob/Gyn (2.12). Those in Emergency Medicine rated this item significantly lower than did those in Psychiatry, Pediatrics (1.73), and Internal Medicine (1.92). No other group differences were found.

In terms of “Overall usefulness,” the results were similar to those above, with Psychiatry residents rating this item significantly higher (1.47) than did those in Emergency Medicine (2.27) and Internal Medicine (2.02). Emergency Medicine residents also rated this item significantly lower than did Pediatrics residents (1.76). No other group differences were found.

Application and dissemination

Residents were also asked on the GPRA whether or not they had applied the skills they learned during SBIRT training in their work over the follow-up period. Over 80% of residents indicated that they had, and the rates of application were found to vary by program (Table 2). Pearson's chi-square analysis revealed significant differences across groups, with Psychiatry and Pediatrics reporting the highest percentages of use (92.3% and 90.0%, respectively) and Ob/Gyn and Emergency Medicine reporting the least use (65.4% and 67.6%, respectively).

Table 2

Resident GPRA ratings of training application at follow-up by program (N = 202).

	Applied training in your work				Shared info with others				Shared materials with others
Program	n ^Yes	%^Yes	χ2	P	n ^Yes	%^Yes	χ2	P	n ^Yes	%^Yes	χ2	P
Psychiatry	36	92.3			26	66.7			14	35.9
Pediatrics-Medicine/Pediatrics	27	90.0			20	66.7			10	33.3
Internal Medicine	42	82.4			17	32.7			10	19.2
Emergency Medicine	23	67.6			14	41.2			5	14.7
Family Medicine	16	76.2			13	61.9			7	33.3
Ob/Gyn	17	65.4			11	42.3			3	11.5
Total	161	80.1	12.69	.026	101	50.0	16.76	.005	49	24.3	9.86	.079

Note. n^Yes represents the number of trainees answering “Yes” to the questions about application. %^Yes represents the percentage of trainees answering “Yes.” Pearson χ² tests were used to analyze differences among groups.

Of further interest within the SBIRT model are the ways in which training information is disseminated to other professionals following learning. At 4-month follow-up, 50% of residents reported that they had shared information they learned with others, and 24.3% said they shared materials from the training with others. Significant differences were not found across groups on the question of shared materials, but there were significant differences among programs on rates of shared information. Within Psychiatry and Pediatrics, 66.7% of each reported sharing information with others. Internal Medicine demonstrated the lowest rates at 32.7%.

SBIRT in practice

Residents were administered a survey inquiring about their use of SBIRT skills during the 6-month period following training completion. Responses were examined by postgraduate year and by specialty (Tables 3 and 4).

Table 3

Ratings on 6-month follow-up survey of applied use of SBIRT principles among residents following training by postgraduate Year (N = 78).

	Strongly agree	Agree
Question	n %	n %	Cumulative %	Total mean	PGY1 (n = 25)	PGY2 (n = 33)	PGY3 (n = 14)	PGY4 (n = 6)
1. After training, I am more likely to talk to my patients about their substance use.	17 21.8	44 56.4	78.2	3.94	3.84	3.94	3.93	4.33
2. SBIRT training has enhanced my ability to screen patients for substance abuse/misuse.	16 20.5	46 59.0	79.5	3.94	3.92	3.88	3.93	4.33
3. I regularly screen my patients for substance use and misuse.	21 26.9	46 59.0	85.9	4.06	4.04	4.18	3.79	4.17
4. SBIRT training has enhanced my ability to perform a BI for substance abuse/misuse with my patients.	14 17.9	54 69.2	87.2	4.04	3.96	4.06	4.00	4.33
5. I use motivational interviewing/brief intervention techniques to assist my patients.	13 16.7	45 57.7	74.4	3.85	3.60	3.97	3.86	4.17
6. Since SBIRT training, I feel more confident in my ability to know when to refer patients for follow-up care for substance abuse/misuse.	13 16.7	51 65.4	82.1	3.91	3.88	3.88	3.86	4.33
7. SBIRT training has helped me be overall more effective in assisting patients with substance abuse/misuse.	14 17.9	47 60.3	78.2	3.91	3.72	3.97	3.93	4.33
8. My patients have made positive behavioral changes following SBIRT interventions.	9 11.5	33 42.3	53.8	3.54	3.44	3.55	3.50	4.00

Note. “Cumulative %” represents the percentage of trainees reporting that they Agree or Strongly agree with the question presented.

1 = Strongly disagree to 5 = Strongly agree. Analysis of variance revealed no significant differences in mean ratings on any items by postgraduate year.

Table 4

Mean ratings on 6-month follow-up survey of applied use of SBIRT principles among residents following training by program (N = 78).

Question	Emergency Medine (n = 12)	Internal Medicine (n = 11)	Ob/Gyn (n = 11)	Pediatrics-Medine/Pediatrics (n = 10)	Psychiatry (n = 25)	Family Medicine (n = 9)	F	P
1. After training, I am more likely to talk to my patients about their substance use.	3.50	3.64	3.91	4.00	4.20	4.11	1.70	.15
2. SBIRT training has enhanced my ability to screen patients for substance abuse/misuse.	3.42	3.64	4.18	3.70	4.32	3.89	3.33	.01
3. I regularly screen my patients for substance use and misuse.	4.00	3.82	3.91	4.20	4.20	4.11	.53	.76
4. SBIRT training has enhanced my ability to perform a BI for substance abuse/misuse with my patients.	3.58	3.91	4.18	4.00	4.24	4.00	2.49	.04
5. I use motivational interviewing/brief intervention techniques to assist my patients.	3.08	3.45	4.00	4.00	4.16	4.11	5.21	.000
6. Since SBIRT training, I feel more confident in my ability to know when to refer patients for follow-up care for substance abuse/misuse.	3.50	3.73	4.09	3.70	4.24	3.78	2.06	.08
7. SBIRT training has helped me be overall more effective in assisting patients with substance abuse/misuse.	3.33	3.64	3.91	3.90	4.24	4.11	3.12	.01
8. My patients have made positive behavioral changes following SBIRT interventions.	3.00	3.36	3.73	3.20	3.88	3.67	2.58	.03

Note. 1 = Strongly disagree to 5 = Strongly agree.

Overall, residents indicated that training increased the likelihood that they would talk to their patients about substance use (item 1, mean = 3.94) and has helped them to be more effective in assisting their patients with substance use issues (item 7, mean = 3.91). More specifically, they indicated that the training has improved their skills in the area of screening patients for substance use (item 2, mean = 3.94), performing BIs (item 4, mean = 4.04), and knowing when to refer patients for further treatment (item 6, mean = 3.91). Furthermore, 85.9% reported that they now regularly screen their patients for substance use (item 3), and 74.4% indicated that they are using motivational interviewing with their patients (item 5). When asked if their patients have made positive behavioral changes following SBIRT interventions, 53.8% indicated that they had. Although PGY4 residents appeared to rate the items generally higher, no significant differences by training year were found (Table 3).

Because we were also interested in training issues across programs, we used ANOVAs to test for significant differences by program (Table 4). Significant differences, when present, primarily fell along the lines of the Emergency Medicine residents rating the level of agreement significantly lower than did Psychiatry residents. This was the case on questions 2, 4, 7, and 8. On question 5, the Emergency Medicine residents were found to rate their use of motivational interviewing lower than did residents from all other programs, except for Internal Medicine.

Qualitative analysis: Barriers to utilizing SBIRT

The qualitative analysis resulted in 3 overarching themes regarding barriers to utilizing SBIRT (Figure 1) and 3 overarching themes regarding feedback for future trainings (Figure 2). Some of the themes were further delineated into subthemes.

Figure 1

Barriers to utilizing SBIRT.

Figure 2

Feedback for future trainings.

Contextual factors

Residents identified various contextual factors as barriers to utilizing SBIRT. One notable contextual factor was time. One resident reported, “I think the only barrier to using SBIRT might be the time constraint we are under when seeing patients.” Another resident stated, “usually in a 15 minute encounter, most patients have multiple medical problems to address,” highlighting the challenge of doing SBIRT when there are competing tasks that need to be completed in a short clinical encounter. One resident noted, “I feel we only have time to plant the seed.”

Another contextual factor noted by residents was clinical norms. Some residents reported that the perception that SBIRT is not relevant or appropriate to their department, which appeared to be based on the clinical norms for each department. For example, one resident reported that SBIRT is “not extremely relevant the day of surgery.” Another resident stated, “I think it would be more useful in inpatient setting as well as clinical setting.” Many residents in the emergency department noted the difficulty utilizing SBIRT due to specific departmental norms or requirements. For example, one resident noted the “[in]ability to establish follow up plans from the ED” as a factor that made implementing SBIRT challenging.

The third contextual factor identified by residents as a barrier to utilizing SBIRT was externally imposed values. Some residents noted the challenge of implementing SBIRT when the intervention is not valued by attending physicians. For example, one resident stated, “The attendings don't know about it and sometimes it's an issue when presenting patients and recommending SBIRT.” Another resident noted, “As a resident I am more evaluated on my time efficiency and not any use of preventative medicine.” Others noted that there is “no reimbursement” for utilizing SBIRT, highlighting the difficulty of utilizing interventions that may be effective but are not valued by third-party payers.

Patient readiness

Residents identified patient readiness as a barrier to utilizing SBIRT. One resident stated, “Some patients are not ready to change.” Another listed “motivation (lack of) for patients to change” as the main challenge. One resident noted, “teenagers don't see downsides [to substance use].”

No barriers identified

Ten residents listed “none” or “nothing” as barriers to implementing SBIRT. These explicit statements denying barriers suggested that some perceive SBIRT as an intervention that can be utilized despite some of the barriers noted within this data set.

Qualitative analysis: Feedback for future trainings

The following themes were identified regarding residents’ feedback for future SBIRT trainings: (a) posttraining resources, (b) revise training, and (c) none/no changes.

Posttraining resources

Residents reported wanting accessible reference materials after the training. For example, one resident asked, “Is there a website where this information is available?” Another resident stated, “A reference pocket manual would be helpful.”

Residents also reported wanting greater access to referral. For example, one resident asked for “a pamplet of sort [sic] with contact numbers or information of where to refer patient.” Another resident requested information regarding “outpatient resources for patients with and without insurance.”

Revise training

Residents identified various aspects of training that needed to be revised. One subtheme was a request to increase training. For example, one resident requested training on “addressing patients in pre-contemplative stage.” Another resident requested more demonstration, stating that it “would be more helpful if we can actually see the interaction between the standard patient and instructor instead of going over guidelines.” Yet another resident identified wanting “updates and follow up classes.”

Another subtheme under the larger theme of revise training was to streamline. For example, one resident stated, “SBIRT requires so much commitment to post-surveys, training, etc. Is there a way to consolidate them so that we don't feel constantly bombarded with ‘one more thing’?” Another resident suggested “making it 100% online, no classes, no tests.”

Residents also identified a need to target attendings for training. One resident indicated a need to “stress importance of intervention to attendings each year.” Another resident stated, “train the attending as well.”

None/no changes

Many residents stated that there were no changes they would make to the current training, and some explicitly stated that their training was a positive experience. For example, one resident reported, “SBIRT training does enhance the ability to screen patients for substance abuse and ability to perform an intervention.”

Discussion

Effective screening for substance use and intervention with patients when necessary are important aspects of treatment in all health care settings. SBIRT training programs provide a way to effectively teach skills and disseminate information on techniques that can be quickly and easily applied by physicians who encounter patients with substance use issues. This and previous research documents the ongoing satisfaction that residents have with the training and its relevance for clinical practice. After having had ample opportunity to practice SBIRT techniques in actual clinical work, residents remain highly satisfied with the training, show high rates of dissemination among their colleagues, and report that SBIRT has good clinical utility.

In our study, much of the success of the training seemed tied to enhancing trainee skill and confidence in working with patients regarding issues of substance use. A significant number of trainees reported that training made them more likely to talk to their patients about substance use and allowed them to be more effective in helping their patients with substance use issues. Likewise, they felt that SBIRT training provided them useful tools for engaging patients around issues of substance use. Indeed, after months of practice, residents continued to find the training relevant to their work, with more than 80% of trainees applying the skills they had learned to their daily work. This compelling finding supports the utility of SBIRT training in clinical practice.

Consistent with the goals of training, a majority of residents reported that training enhanced their ability to screen patients, to perform BIs, and improved their ability to know when to refer. Following training, 85.9% stated that they regularly screened for substance use issues, and 74% reported the use of motivational interviewing with their patients. The implementation of regular screening for substance use is an outcome that has potential for impacting the quality of substance use detection in the general population. These findings provide further evidence that screening and BIs are viable tools and speak to the willingness and ability of physicians to use these tools in their practice.

We found that although the application of skills is high among trainees in general, only about half believed their patients made positive behavioral changes following SBIRT interventions. Responses to this question varied by program, with the lowest scores for Emergency Medicine residents and the highest for those in Psychiatry. To some degree, these findings make sense given the differing nature of the patient-physician relationship in each specialty. Psychiatry residents often see patients for multiple visits over time, whereas an emergency department physician may only see a patient once, making it difficult to collect follow-up information on patient outcomes. At the same time, these results have particular relevance given recent findings in JAMA (Journal of the American Medical Association) that question SBIRTs broader effectiveness in intervening with drug use, in particular.¹²,¹³ However, given that residents find these techniques useful in their work with patients is an important indicator of the clinical utility of the SBIRT model. This and previous research⁷,¹⁴ indicates that SBIRT training may increase use of screening at the population level when widely disseminated. Additional research is certainly needed to better understand the mechanisms involved in producing change following an intervention, as well as to identify which patients are likely to benefit most, when applying SBIRT principles in practice. Yet, the rate at which residents appear to apply these tools is, at minimum, a testament for the need for models of substance use intervention in the health care field as a whole.

When examining ratings by program, a few important differences were found. Psychiatry and Pediatrics residents in general seemed to find SBIRT skills more relevant to their career, whereas Emergency Medicine residents appeared to consider them less relevant, at least when compared with residents from other programs in our study. Likewise, on some aspects, Obstetrics/Gynecology and Internal Medicine residents also seemed to find the SBIRT training to have somewhat less relevance to their work when compared with other programs. Psychiatry and Pediatrics residents reported the most frequent use, and Emergency Medicine and Ob/Gyn residents appeared to use it the least. This was a curious finding, perhaps especially given that SBIRT research originated in the emergency department¹⁵ and has since found champions in emergency care settings throughout the nation.¹⁶ –19 Given this notable history, we propose that it was not likely due to an absence of need in our settings, but more likely related to situational factors.

As evidence of this, results from the qualitative analyses suggested that although there is widespread use, some residents perceived contextual barriers to implementing SBIRT post training, including time limitations, the perception that SBIRT is not part of a departmental clinical norm, and the perception that SBIRT is not valued by attending physicians or by third-party payers. These findings suggest that SBIRT training may need to better address several factors. First, residents need to be educated about how to utilize SBIRT within departmental constraints, such as limited time in clinical encounters and within departments that may not traditionally focus on substance use interventions. It is interesting that many ED residents especially noted the challenge of including SBIRT into their clinical encounter due to time pressure and clinical norms. Yet a high proportion of ED patients have substance-related problems that contribute to them returning to the ED for new injuries, and a screening and brief intervention could lead to significant reductions in health care costs.²⁰,²¹ Second, SBIRT training may need to target attending physicians as well as residents, given that residents feel the pressure to utilize interventions that are valued by attending physicians.

Interestingly, residents also identified patient readiness as a barrier to utilizing SBIRT, suggesting that they perceived resistant patients as being poor candidates for an SBIRT intervention. Studies have shown that interventions targeted toward behavior change can be effective even for people in the precontemplation stage.²² Moreover, SBIRT utilizes motivational interviewing, which encourages evoking the patient's own ambivalence to induce contemplation for change.²³ The feedback from residents regarding patient readiness serving as a barrier suggests that SBIRT training may need to better educate residents about the stages of change and how immediate behavior change may not necessarily be the end goal for all patients, particularly for those in the earliest stage of change.

Residents also gave feedback about wanting posttraining resources, including reference materials and referrals for patients. Some residents requested further training on various aspects of SBIRT, whereas others requested that the training be streamlined. This contrary feedback highlights the tension inherent in substance use training: difficulty finding time in residents’ schedules for trainings²⁴ on one hand, and the need for greater substance use–related training in residency on the other hand.²⁵,²⁶ Despite these tensions, it was encouraging to find that residents are not only gaining relevant knowledge during training, but also that they appear to be willing and able to apply these skills in practice.

Limitations to this study should be noted. Although we had an excellent response rate on the satisfaction survey at follow-up (80%), only about a third (31%) of residents trained completed the practice survey. Furthermore, it should be noted that the practice survey was not entirely representative of all trainees in that there were proportionally more Psychiatry residents and fewer Internal Medicine residents responding when compared with the GPRA. These differences should be taken into account when interpreting the results. Thus, continued research on application following training will be important to more fully assess the practical utility of applying SBIRT in routine practice. Certainly, more research is needed to understand the impact resident interventions have on patient outcomes, but as dissemination and interest in learning these skills continue, the likelihood of fostering positive patient outcomes is increased.

Footnotes

Acknowledgments

The authors wish to thank the many colleagues, collaborators, and residents who have helped to make this program both possible and successful.

Author contributions

A. J. C. was involved in research conception and design, training, data analysis, interpretation of results, writing, and revision. V. I. B. was involved in research conception and design, training design, training, and writing. M. L. was involved in research conception and design, data analysis, interpretation of results, writing, and training. J. P. was involved in research conception and design, data analysis, interpretation of results, writing, and training. B. M. S. was involved in research conception and design, training design, training, and writing. I. M. M. was involved in research design, training design, and data collection. N. B. was involved in research conception, training, and data collection. M. R. L. was involved in research conception and design, manuscript editing, and training. P. P. was involved in training design, training, and research conception and design. A. A. was involved in training design, training, and research conception and design. C. J. H. was involved in research conception and design and training. S. H. was involved in research conception and training. S. D. G. was involved in research design and conception and training design.

Funding

Preparation of this report and the program described herein were supported by SAMHSA grant 1U79TI02069. SAMHSA provided funding for the development of training and the evaluation of training effectiveness. SAMHSA was not involved in the design of the training program, study design, or in manuscript preparation. The authors declare that they have no conflicts of interest.

References

Tetrault

, Green

, Martino

, .Developing and implementing a multispecialty graduate medical education curriculum on Screening, Brief Intervention, and Referral to Treatment (SBIRT). Subst Abus. 2012;33:168–181.

Bray

, Kowalchuk

, Waters

, Laufman

, Shilling

. Baylor SBIRT Medical Residency Training Program: model description and initial evaluation. Subst Abus. 2012;33:231–240.

Ryan

, Martel

, Pantalon

, . Screening, Brief Intervention, and Referral to Treatment (SBIRT) for alcohol and other drug use among adolescents: evaluation of a pediatric residency curriculum. Subst Abus. 2012; 33:251–260.

Muench

, Jarvis

, Boverman

, Hardman

, Hayes

, Winkle

. Tilling the soil while sowing the seeds: combining resident education with medical home transformation. Subst Abus. 2012;33:282–285.

Seale

, Velasquez

, Johnson

, . Skills-based training in alcohol screening and brief intervention: results from the Georgia-Texas “Improving Brief Intervention” project. Subst Abuse. 2012;33:261–271.

Marshall

, McLaurin-Jones

, Kalu

, . Screening, brief intervention, and referral to treatment: public health training for primary care. Am J Public Health. 2012;102:e30–e36.

Bray

, Kowalchuk

, Waters

, Allen

, Laufman

, Shilling

. Baylor Pediatric SBIRT Medical Residency Training Program: model description and evaluation. Subst Abus. 2014; doi: 10.1080/08897077.2014.954026.

Clemence

, Balkoski VI , Schaefer

, Lee

, Bromley

, Maisonneuve

, Hamilton

, Lukowitsky MR , Poston

, Hall

, Pieterse

, Antonikowski

, Glick

. Multispecialty SBIRT Training in an Academic Medical Center: Resident Training Experience across Specialties. Substance Abuse. In press.

Schram

, Harris

, Van Hook

, . Implementing adolescent SBIRT education in a pediatric residency curriculum. Subst Abus. 2014; doi: 10.1080/08897077.2014.936576.

10.

Braun

, Clarke

. Using thematic analysis in psychology. Qual Res Psychol. 2006;3:77–101. doi: 10.1191/1478088706qp063oa.

11.

Charmaz

Conducting grounded Theory: A Practical Guide Through Qualitative Analysis. Thousand Oaks, CA: Sage; 2006.

12.

Saitz

, Palfai

TPA

, Cheng

, . Screening and brief intervention for drug use in primary care: the ASPIRE randomized clinical trial. JAMA. 2014;312:502–513. doi: 10.1001/jama.2014.7862.

13.

Roy-Byrne

, Bumgardner

, Krupski

, . Brief intervention for problem drug use in safety-net primary care settings: a randomized clinical trial. JAMA. 2014;312:492–501. doi: 10.1001/jama.2014.7860.

14.

Whittle

, Buckelew

, Satterfield

, Lum

, O'Sullivan

. Addressing adolescent substance use: teaching SBIRT and MI to residents. Subst Abus. 2014; doi: 10.1080/08897077.2014.965292.

15.

Chafetz

, Blane

, Abrams

, . Establishing treatment relations with alcoholics J Nerv Ment Dis. 1962;34:395–409.

16.

Academic ED SBIRT Research Collaborative. The impact of screening, brief intervention and referral for treatment (SBIRT) on emergency department patients’ alcohol use. Ann Emerg Med. 2007;50:699–710.

17.

Academic ED SBIRT Research Collaborative. The impact of screening, brief intervention and referral for treatment in emergency department patients’ alcohol use: a 3-, 6- and 12-month follow-up. Alcohol Alcohol. 2010;45:514–519.

18.

D'Onofrio

, Degutis

. Integrating project ASSERT: a screening, intervention, and referral to treatment program for unhealthy alcohol and drug use into an urban emergency department. Acad Emerg Med. 2010;17:903–911.

19.

Nielsen

, Baird

, Mello

. A systematic review of emergency care brief alcohol interventions for injury patients. J Subst Abuse Treat. 2008;35:184–201.

20.

Gentilello

, Ebel

, Wickizer

, Salkever

, Rivara

. Alcohol interventions for trauma patients treated in emergency departments and hospitals. Ann Surg. 2005;241:541–550. doi: 10.1097/01.sla.0000157133.80396.1c.

21.

D'Onofrio

, Degutis

. Preventive care in the emergency department: screening and brief intervention for alcohol problems in the emergency department: a systematic review. Acad Emerg Med. 2002;9:627–638.

22.

Armitage

, Arden

. How useful are the stages of change for targeting interventions? Randomized test of a brief intervention to reduce smoking. Health Psychol. 2008;27:789–798. doi: 10.1037/0278-6133.27.6.789.

23.

Miller

, Rollnick

. Motivational Interviewing: Helping People Change. 3rd ed. New York: Guilford Press; 2013.

24.

Pringle

, Kowalchuk

, Meyers

, Seale

. Equipping residents to address alcohol and drug abuse: the national SBIRT residency training project. J Grad Med Educ. 2012;4:58–63. doi: 10.4300/JGME-D-11-00019.1.

25.

Lasswell

, Liepman MR , McQuade

, Wolfson

, Levy

. Comparison of primary care residents’ confidence and clinical behavior in treating hypertension versus treating alcoholism. Acad Med. 1993;68:580–582. doi: 10.1097/00001888-199307000-00021.

26.

Wakeman

, Baggett

, Pham-Kanter

, Campbell

. Internal medicine residents’ training in substance use disorders: a survey of the quality of instruction and residents’ self-perceived preparedness to diagnose and treat addiction. Subst Abus. 2013;34:363–370.