Abstract
The objective of this study was to evaluate the integration of a screening, brief intervention, and referral to treatment (SBIRT) curriculum for alcohol and other drug use into a pediatric residency program. Pediatric and medicine/pediatric residents in an adolescent medicine rotation located in an urban teaching hospital participated in the study. Main outcome measures were pre- and post-training knowledge scores, performance of the Brief Negotiation Interview (BNI), training satisfaction, and adoption of the BNI into clinical practice. Thirty-four residents were trained. Significant pre- to post-training improvements were seen in knowledge scores (P < .001) and performance as measured by the BNI Adherence Scale (P < .001). Residents reported high satisfaction immediately post-training and at 30 days on a 1–5 Likert scale: mean 1.41 to 1.59 (1 = very satisfied) (P = 0.23). Over a 9-month period, 53% of residents documented performing at least 1 BNI, of which 2/3 reported ≥2 BNIs in a subsequent clinical setting. The results show that integrating a SBIRT curriculum into a pediatric residency program increases residents’ knowledge and skills.
Introduction
Alcohol and other drug use and abuse continues to be a significant public health problem for adolescents and young adults in the United States and globally. The 2009 National Survey on Drug Use and Health (1) found that 10.4 million persons aged 12 to 20 years reported drinking alcohol in the past month. Whereas rates of current, binge, and heavy alcohol use among underaged persons declined between 2002 and 2008, the 2009 estimates of 27.2%, 18.1%, and 5.4%, respectively, document a worrisome upward trend. The rate of past-month illicit or prescription drug use among 12- to 17-year-olds also increased to 10.0% up from 9.3% in 2008 (1). The nationwide Monitoring the Future survey (2) also confirms that high school students, adolescents, and young adults have continued to use substances at high rates and experience significant morbidity as a result. Marijuana is the most commonly used illicit drug among teens, with 13.2% of 12- to 17-year-olds reporting any past-year use (2). Daily marijuana use also increased among 8th, 10th, and 12th graders from 2009 to 2010, and among 12th graders, marijuana use is at its highest point since the early 1980's with 21.4% of high school seniors reporting past-year use. This increase coupled with less perceived risk from marijuana use suggests rising future trends (2). After marijuana, prescription (Vicodin: 8.0% and OxyContin: 5.1%) and over-the-counter medications (cough medicine: 6.6% and Adderall: 6.5%) are the most prevalent drugs of abuse (2).
Alcohol and other drug use are associated with multiple negative health, social, and economic consequences. The leading cause of death for 15- to 20-year-olds is a motor vehicle crash, and according to the National Highway Traffic Safety Administration, 31% of these are related to alcohol (3). Risk-taking behaviors such as drinking alcohol and novelty-seeking activities may result from earlier development of the limbic system in the adolescent brain, prior to maturation of prefrontal cortex areas responsible for impulse control, judgment, reasoning, and problem solving (4–7). Risky behaviors such as alcohol and drug use often do not occur in isolation and are associated with other behaviors such as tobacco use, driving under the influence, early and unsafe sexual activity, and interpersonal violence (8, 9).
In response to these data, the American Academy of Pediatrics has developed specific guidelines through Bright Futures for the screening and counseling about alcohol and illicit substance use (10). Despite these recommendations, pediatric providers have been reluctant to incorporate these strategies into their practice. Fewer than 50% of pediatricians report screening teens for substance use and abuse in their general pediatric practices, and fewer than 25% report asking teens about drinking and driving (11). Isaacson et al. surveyed pediatric residency directors and found that only 32% required any substance use disorders content in their programs, with a Grand Rounds presentation being the most common curricular format (12).
We describe the evaluation of a modified screening, brief intervention, and referral to treatment (SBIRT) program integrated into a pediatric residency program. We hypothesized that pediatric residents who have completed the program would have greater knowledge and skills to screen and provide intervention, namely the Brief Negotiation Interview (BNI) (13) for adolescents with alcohol and other drug use and that pediatric residents would be satisfied with their training and adopt SBIRT into clinical practice.
Methods
Setting and Participants
Yale–New Haven Hospital is a 960-bed urban, teaching hospital located in New Haven, Connecticut, which ranks among the poorest cities in the United States, with 24.4% of its 350,000 residents living below the poverty level. The Adolescent Clinic provides approximately 2100 primary care visits per year for patients aged 13 to 22 years. The racial/ethnic mix is 59% African American, 33% Latino, 5% Caucasian, and 3% Asian/other. Participants were all 2nd year pediatric and 3rd year medicine/pediatric residents who were recruited during their required 1-month elective in adolescent medicine. This project was approved by the Yale University Institutional Review Board.
Core SBIRT Curriculum
Prior to the implementation of the curriculum there were no elements in the pediatric training program that specifically addressed screening, strategies for counseling and intervention. First, the Pediatric Core leader participated in a “train-the-trainer” session that included didactics, skill-based training, and opportunities to interact with experts in the field. Once trained, the Pediatric Core leader trained all residents throughout the study period. A 2-hour Core Curriculum was provided at the outset of the 1-month Adolescent Elective. This curriculum incorporated a didactic portion, a video demonstrating the BNI procedure, skills-based role-playing with immediate feedback, and discussion of strategies for integrating SBIRT into clinical practice.
The initial 30-minute didactic session described the epidemiology of alcohol and other drug (AOD) use among adolescents, including marijuana, club drugs and prescription drugs, the National Institute on Alcohol Abuse and Alcoholism (NIAAA) definitions and Diagnostic and Statistical Manual for Mental Disorders Fourth Edition (DSM-IV) criteria for problem use and addiction, and screening tools for adolescents and young adults. Specific adolescent-focused topics included brain maturation (14) and other developmental risk factors, their role in risk-taking and substance use initiation, and postulated long-term effects of AOD use on adolescent brain development (4–6).
A 40-minute video, developed by D'Onofrio and Pantalon, followed, outlining the components of the BNI with actual patient scenarios. Finally, the skills-based session (45 minutes) allowed residents to actively participate in several role-plays using actual adolescent patient cases. The Core leader observed each interaction and offered immediate feedback. Emphasis was also placed on how the BNI could be adapted for particular situations where pediatric patients presented with the spectrum of use and abuse, such as experimental or early use.
Available resources were discussed, including access to the Yale SBIRT Web site, www.yale.edu/SBIRT, for independent learning; the Web site contains the video, as well as additional pediatric slide presentations, lectures provided to residents in the other primary care specialties participating in the SBIRT project, and 4 additional pediatric case studies. Laminated, pocket-sized cards were provided containing the screening tools and the BNI steps. The initial development, training, and implementation of this federally funded SBIRT program, and description of how this program was expanded to incorporate existing curricular elements into primary care residency programs, has been reported elsewhere (15).
Modification of the SBIRT Curriculum Specific to Pediatrics
The screening component of the SBIRT was modified by supplementing the NIAAA questions and the two screening tools that included the CAGE (4 questions) and the CRAFFT (6 questions), the latter which was developed for and tested in adolescent populations (16). The BNI that has been developed and refined over the past 18 years was modified to address the resistance often encountered with the adolescent population. The BNI incorporates principles of motivational interviewing in an attempt to assist patients to reduce or abstain from unhealthy substance use (13, 17). It takes an average of 6 to 10 minutes to complete. In providing the BNI, the provider works with the patient to:
Raise the subject of alcohol/drug use and establish rapport. Provide feedback: Review patient's screen, make a connection to visit/illness, injury or self-reported risk-taking behavior, if possible; review NIAAA guidelines for low-risk drinking or consequences of drug use. Enhance motivation: Have patient self-identify on readiness to change, ruler from 1 being unwilling to change any aspect of their substance use to 10 being very ready; and develop discrepancy between use and self-ascribed reasons to change. Negotiate and advise: Negotiate goal, give advice, summarize, and provide follow-up and referral if needed.
Because many adolescents may not recognize their use as a problem, or not know where to ask for help (18), establishing rapport in the confidential clinic environment as the initial step in the BNI is important to opening a dialogue. In particular, this component of the BNI was emphasized in training, so that residents understood that asking permission to discuss a patient's use in a nonthreatening, nonlabeling way was paramount in reducing any resistance. Thus, though the steps of the BNI are similar in this modified pediatric version, emphasis was placed on how to have a conversation with a resistant adolescent. For example, adolescents who chose a “1” on the readiness ruler, were asked: “Have you ever done anything that you wish you had not done while drinking or using drugs?” or “What would have to happen for you to want to change.” Reflective listening at this time was emphasized as important in opening a dialogue. This aspect of the BNI is especially well suited for adolescents given that it is efficient and respectful, and addresses their developmental need for autonomy and sense that their own “voice” in choosing strategies for behavior change is being heard (19).
Evaluation Measures
Baseline Resident Characteristics
Prior to training, residents completed an anonymous demographic survey that has been tested and reported previously (20, 21). Information obtained included (1) demographics, such as residency program, age, gender, and race/ethnicity; (2) alcohol or other drug (AOD) education in previous medical/ professional training; (3) clinical experience with patients with AOD use; (4) personal experience knowing someone with an AOD problem; and (5) perception of how effective they thought they could be in changing patient behavior utilizing a brief intervention technique in a clinical encounter (i.e., self-efficacy), using a Likert-type scale from 1 to 10, where 1 means not effective at all and 10 means totally effective.
Training Outcome Measures
Knowledge and BNI performance measures were administered during the first week of the adolescent medicine rotation, prior to participating in the core curriculum (pre-training) and at the end of the 4-week rotation (post- training).
Knowledge The Knowledge test consisted of a total of 30 items. Ten items were related to the definition of at-risk drinking for men and women, criteria for the assessment of alcohol dependence, specific elements of screening tools, including the CRAFFT and the CAGE, and the NIAAA quantity and frequency questions. These questions comprised a Didactic subscale, with possible score ranging from 0 to 10, with higher score indicating higher correct total. The next 20 questions were True/False items related to a case study describing a 17-year-old intoxicated driver involved in a single car crash. These questions tested residents’ knowledge of the specific steps of the BNI and comprised the Case-Based subscale, with possible scores ranging from 0 to 20, with higher scores indicating greater number of correct items. A total score for all 30 items, as well as individual scores for the 2 subscales were calculated for each resident, for analysis. The Didactic subscale was adapted from a scale previously developed (20); the Case-Based scale was developed specifically for this study.
BNI performance BNI performance was measured with the BNI Adherence Scale (Figure 1), which was developed in our previous BNI randomized trials (13, 17). This scale has a total of 21 items with 18 “Yes/No” (i.e., present vs. absent) items, each of which pertains to either a prescribed (e.g., assessed motivational readiness to change) or proscribed (e.g., referred to patient as an “alcoholic”) critical action of the BNI, two 7-point Likert-type items assessing how well residents utilized reflective listening (either to reflect motivation or to redirect resistance), and excluding item #7 that lists the member reported from the Readiness Ruler. The scale has demonstrated excellent internal consistency (Kuder-Richardson's alpha = .94). Three independent raters (interrater reliability, intraclass correlation coefficient = .83) rated residents’ audiotapes of BNI performance on pre- and post-training role-plays with a standardized patient (SP) (described below). The BNI Adherence Scale yields 5 scores: (1) A Total score (sum of items 1 to 19) positively scored so that higher score = higher adherence to BNI; (2) the “Discussion of Problems” subscale score (sum of items 1, 3, 9, 14, and 18); (3) the “Discussion of Change” subscale score (sum of items 8, 15, and 17); and (4) 2 individual scores for items 20 and 21.
BNI Adherence Scale.
The SP role-play required that each resident conduct a clinical interview using screening and intervention techniques and implement the BNI with the same research associate (same clinical case both pre-and post-training) playing the role of a patient who is a 17-year-old female, presenting to the Adolescent Clinic for sexually transmitted infection screening following a “blackout” drinking episode she experienced at a party 48 hours earlier.
Training satisfaction Training satisfaction was measured immediately after and 30 days post-training with 4 questions (e.g., “How satisfied were you with the overall quality of the training?”). These questions were embedded within the Government Performance and Results Act (GPRA) for Best Practices, a 24-item scale developed by the Center for Substance Abuse Treatment to track the demographics and satisfaction with training of learners funded through federal grants. The items were scored on a 5-point Likert-type scale (1 = very satisfied to 5 = very dissatisfied). Scores for each individual item as well as a total score were calculated.
Adoption of BNI in clinical practice Survey Monkey, an online tool used to survey large groups around a specific question, was used to assess resident self-report of their use of the BNI in pediatric clinical settings (both inpatient, outpatient, and emergency department [ED]). Residents were asked at 3-month intervals, for a total of 9 months post-training, to report retrospectively the number of BNIs performed in clinical practice.
Statistical Analyses
Means and standard deviations or percents were calculated to describe baseline resident characteristics, and pre- and post-training scores in knowledge and BNI performance. Means and standard deviations were calculated for the training satisfaction scores that were measured immediately and 30 days post-training. Percent distributions were calculated for BNIs reported over the 9-month post-training period. Because of the small number of resident subjects, and the concern about the lack of a Gaussian distribution in the data, the Wilcoxon paired signed-rank test (nonparametric version of the paired t test) was used to assess statistically significant differences in all scale scores pre- and post-training. A P value of .05 was selected for significance. All statistical analyses were done using the SAS statistical program, version 9.2 (SAS, Chicago, IL).
Results
Evaluation Measures
Baseline Resident Characteristics
Table 1 describes the residents’ baseline characteristics. Thirty-four pediatric residents were trained from February 2009 through March 2011. The majority was female with a mean age of 28.5 years, and a range of 25 to 35 years. Fifty percent reported having between 0 and 5 hours of alcohol and/or drug education during their medical and professional training. The majority reported little or no clinical experience with patients with alcohol (88%) and drug (91%) problems. The mean and median scores of the residents’ perceived self-efficacy in motivating patients to change their alcohol and/or drug patterns were 4.53 (SD 1.50) and 4.0, respectively.
Baseline Resident Characteristics (N = 34)
Knowledge
Table 2 presents the pre- and post-training knowledge scores. There were statistically significant improvements in all scale scores.
Knowledge Scores Pre- and Post-Training (N = 34)
Based on Wilcoxon signed-rank test.
Bni Performance
Table 3 demonstrates the significant improvements in BNI performance from pre- to post-training in the Total score, and the “Discussion of Problems” and “Discussion of Change” subscale scores. However, differences between pre- and post-training for items 20 and 21, the “reflective listening” items, did not reach statistical significance.
BNI Adherence Score Pre- and Post-Training (N = 34)
Based on Wilcoxon signed-rank test.
Training Satisfaction
Training Satisfaction: Immediately Post-Training and 30 Days Post-Training (N = 34)
P value .24, based on Wilcoxon signed-rank test.
Adoption of the BNI in Clinical Practice
Of 34 residents trained, 32 responded to at least one survey request during the 9-month period following trainings, with the group reporting a total of 34 BNIs throughout this period. Including the 2 nonrespondents as non-BNI adopters (assuming that they did 0 BNIs post-training), 53% of the residents reported completing at least 1 BNI, and, of these, two thirds completed ≥2 BNIs.
Discussion
This paper demonstrates that a skills-based curriculum modified for use in a pediatric residency setting can increase knowledge and skills in screening and providing brief interventions to adolescents with alcohol and drug use. Moreover, this was shown to be possible despite residents having little or no prior experience with SBIRT or working with such patients. The results also demonstrate high levels of satisfaction with the training and the data regarding adoption of BNI skills in clinical practice following the training are encouraging.
At baseline, this cohort of pediatric and medicine/pediatric residents, in their 2nd and 3rd years of residency, reported having virtually no prior medical school or residency training and little experience working with youths with alcohol or other drug abuse problems. Given the high rates of use of both alcohol and marijuana use and the known mortality and morbidity that adolescents and young adults experience as a consequence, it is important that screening and intervening with alcohol and other drug problems be included as part of the formal comprehensive training for pediatricians throughout their residency. Prior studies have demonstrated the feasibility of including the SBIRT into curricula in emergency medicine programs, and in medical schools, and we have shown similarly that training in brief interventions integrated into the pediatric residency setting can lead to increases in their knowledge and skills.
Although the residents reported some level of confidence in their ability to care for these adolescents, as noted by their responses to the self-efficacy item on the baseline Resident Characteristics scale (4.53 out of 10), their pre-training skills in screening and intervention, as demonstrated in the “case-based” portion of the Knowledge scale and pre-training performance scores with the SP encounter, were very low. These results highlight the importance of skills-based learning, with practice and immediate feedback, as compared with pure knowledge-imparting didactic or mere confidence-boosting sessions, in enhancing residents’ actual clinical competence. The residency competencies recently developed by the American College of Graduate Medical Education includes the areas of practice-based learning and improvement. Curricular elements that enhance patient care practices, such as SBIRT and the BNI, directly address this (ACGME Outcome Project, http://wwwacgme.org/outome), and in our development of the SBIRT curriculum, we were attentive to these suggested learning approaches.
Although the pediatric residents’ performance of core BNI skills improved significantly on the total BNI Adherence Scale and the 2 subscale scores, indicating that the training improved actual skills, we did not find significant differences between pre- and post-training scores on the 2 reflective listening items (20 and 21). This may suggest a need for more detailed training and feedback on this important, but finer, skill of the BNI. It may, however, indicate that in brief interventions there may be less overall motivation or resistance to reflect upon, which could limit the residents’ scores on these continuous items.
Despite reports of medical residents’ resistance to learning about SBIRT (12), the residents evaluated in the current study reported high levels of satisfaction with the training immediately post-training. Further, these levels did not decrease significantly when reassessed 30 days post-training, suggesting that the curriculum was acceptable and useful to the residents and remained so even after the rotation.
Although a number of studies have demonstrated the efficacy of the brief intervention in changing patient behaviors for both alcohol and other drug use (22, 23), there are few studies that assess the effectiveness of training for residents and their ability to integrate specific skills into the clinical setting. This project addressed resident's knowledge, skills, and satisfaction regarding screening and intervention with the ultimate goal being their use of this information to motivate behavioral change in clinic patients with unhealthy alcohol and drug use. Although the number of actual BNIs performed by each pediatric resident was small, it is encouraging that two thirds of the residents reported having performed 2 or more BNIs in clinical settings beyond that of the formal adolescent medicine elective.
This study has several limitations. First, our institution has a significant SBIRT and addiction training infrastructure in place and the availability of project faculty to train the core specialty faculty compared to what may be available at other training institutions may limit the generalizability of our findings. Our resident sample is also small, limiting our analyses, and we were unable to assess residents’ skills with actual patients, given availability of patients with AOD problems and time limitations of the month elective. Additionally, all tracking of BNI performance is currently completed by self-report. In response to this last concern, we have integrated the CRAFFT into the electronic medical record for outpatient primary care visits, and residents are prompted to complete the CRAFFT, with specific boxes to check for each question on the CRAFFT as well as whether a BNI was performed. In the future, we will be able to query patient charts to verify performance, which may reduce report bias. We also have few pediatric faculty trained in other settings outside the Adolescent Clinic who can offer modeling and support to the residents. However, many of our pediatric subspecialists have requested training for themselves and their fellows, citing cases where AOD use has had significant impact on medical outcomes and well-being of their patients with chronic diseases such as asthma, liver disorders, and diabetes. These are future opportunities for the training of our pediatric and pediatric emergency department faculty.
In conclusion, this study demonstrates that modifying and integrating a validated SBIRT curriculum into a pediatric residency program is effective in increasing residents’ knowledge and skills in screening and performing the BNI for alcohol and drug use problems in adolescent populations. Residents were highly satisfied with the training and many demonstrated adoption of the BNI skills into clinical practice.
Footnotes
Acknowledgments
This work was funded by SAMSHA 5U79TI020253-03.