Abstract
Background: Prevention of unintentional opioid overdose deaths is effective through overdose prevention trainings (OPTs), in which laypersons are taught overdose response through six actions. This longitudinal study examines trainee knowledge 12 months after OPT completion. Methods: Participants were enrolled following OPT at six sites. At the 12-month follow-up, participants were asked to name the drug overdoses that naloxone reverses and name overdose response actions. A 6-point scale was created and was comprised of the number of correct overdose response actions mentioned (check breathing; administer sternum rub; call 911; give rescue breathing; administer naloxone; put victim in recovery position). Mean knowledge was compared by participant sociodemographic characteristics, confidence, and site of OPT training (indoors versus outdoors). Results: Of 344 OPT participants, 273 were reached at 12 months. Nearly all (99%) participants identified that naloxone reverses heroin overdoes; 77% identified that naloxone reverses opioid analgesics overdoses; and 68% identified that naloxone reverses methadone overdoses. Overdose response actions most frequently mentioned were giving naloxone (86%) and calling 911 (76%). The remaining four actions were mentioned by less than 40% of participants. Overall mean knowledge score was 2.7 out of 6. Mean knowledge scores were higher for college graduates than those with less than college education (3.2 vs 2.6, P < 0.001), for those who felt very confident (mean score [ms] = 2.9), compared to somewhat confident (ms = 2.4) and a little or not at all confident (ms = 1.5) in their ability to reverse an overdose (P < .001), and for indoor-training recipients (3.0 vs 2.5, P = 0.02). There were no differences in mean knowledge scores for trainees by age, race, or gender. Conclusions: These findings suggest the need for several improvements in OPT curriculum, including emphasis on naloxone reversal of opioid analgesic and methadone overdoses, and all 6 rescue actions. Lower knowledge scores among outdoor-trained participants likely reflect session brevity, suggesting that outdoor trainings need to be enhanced.
Introduction
Between 2000 and 2013, there were nearly 10,000 unintentional drug poisoning deaths in New York City (NYC), an average of 700 unintentional overdose deaths per year. 1 In 2014, more than three quarters (79%) of overdose deaths involved an opioid. 2 Teaching bystanders to recognize and respond to opioid overdose is one public health strategy to reduce opioid-involved overdose mortality. New York State permits licensed health care facilities, drug treatment programs, and not-for-profit community-based organizations to register as Opioid Overdose Prevention Programs (OOPPs) and conduct overdose prevention trainings (OPTs) for people at risk of overdose, and their friends and family members. During OPT, individuals learn to recognize and respond to an opioid overdose, and how to administer naloxone—an opioid antagonist that reverses the effects of opioid overdose. OPT curriculum includes the recommended actions to reverse an overdose—including use of naloxone—and which type of drug overdoses naloxone can reverse.
OPT is predicated on social cognitive theory, which posits that improving knowledge, skills, and self-efficacy of individuals in high-risk, drug-using communities will prepare them to assist during an overdose incident and effectively administer naloxone in the future. 3 , 4 A 2003 study in Baltimore, however, showed that emergency medical services providers were concerned that drug users lacked the ability to properly assess victims, administer naloxone, perform rescue breathing, and monitor victims for recurrent symptoms. 5 At the time of the study, only 1 state—New Mexico—had legalized layperson administration of naloxone. Since 2003, several studies have assessed knowledge following receipt of overdose prevention training and have shown that trainings significantly improve knowledge of risks and characteristics of and responses to overdose, as well as confidence in naloxone administration. 6 - 7 Most recently, a study in San Francisco found that brief education lasting only 5 to 10 minutes was sufficient for improving knowledge of overdose identification and management, both immediately following OPT and following receipt of a naloxone refill. 8 To our knowledge, however, there have been no systematic, prospective studies assessing knowledge and self-efficacy beyond 6 months after OPT. Longer-term knowledge and self-efficacy may be important determinants of the effectiveness of training drug users to be overdose responders.
This paper focuses on knowledge and self-efficacy among OPT participants 12 months after completion of OPT and receipt of a naloxone kit. Measuring these 2 factors offers insight into the durability of knowledge and self-efficacy over time and can help inform how OPTs can be improved to give participants the knowledge and confidence they need to use their training during witnessed overdoses.
Methods
Opt Setting and Study Enrollment
Between June 2013 and January 2014, we enrolled overdose prevention trainees from OPTs expected to have high attendance. These OPTs were held by the 6 largest Opioid Overdose Prevention Programs (OOPPs) in NYC. All but one of these OOPPs—the New York City Department of Health and Mental Hygiene (NYC DOHMH)—was a syringe exchange program (SEP), but not all participants enrolled into the study at SEP sites were clients of SEPs. Overdose prevention trainings were conducted indoors and outdoors; programs held trainings inside their offices for clients accessing syringe exchange or drop-in services, as well as outside at New York State–authorized mobile syringe exchange sites, where services were provided from a van or from under a tent. Each program had 1 or more of their staff members conducting OPTs and other outreach at each enrollment site. Although there was some variation in OPTs between sites and trainers, all OPTs followed the same basic curriculum, highlighting the risks and signs of opioid overdose; the actions to take when witnessing an overdose; which drugs naloxone has an effect on; and how to put a dose of naloxone together. Although training length was not systematically recorded, survey administrators observed that indoor trainings were generally 10 to 25 minutes in length, whereas outdoor trainings were typically 5 to 10 minutes in length.
Participants were enrolled immediately after they completed OPT and received a naloxone kit. Eligibility requirements included age 18 years old or greater, ability to complete an interview in Spanish or English, New York City residence, and ability to provide a contact phone number for follow-up interviews. Enrollment included undergoing a verbal consent procedure and providing demographic and contact information to trained, DOHMH interviewers. To maximize follow-up potential and prevent loss to follow-up, extensive contact information was collected, including phone numbers of family and friends, phone numbers of case managers and of utilized programs, and locations of frequented “hang-out” spots. Following enrollment, the same DOHMH interviewers administered a close-ended, approximately 10-minute baseline survey to participants in English and Spanish. Participants were surveyed by DOHMH interviewers again at 3, 6, and 12 months to evaluate knowledge of OPT and self-efficacy in using naloxone; knowledge and self-efficacy were not evaluated at baseline. Follow-up survey lengths varied based on skip patterns, ranging from 10 to 30 minutes. Participants were offered gift cards to Rite Aid and Duane Reade as compensation for their time: $10 at baseline, $25 at 3 months; $35 at 6 months, and $50 at 12 months.
When participants were due for follow-ups, interviewers returned to enrollment sites to search for participants and complete in-person interviews. Participants were called, e-mailed, and mailed post cards in advance, asking them to return to the enrollment site to complete their follow-up. For those who did not return to their enrollment sites and complete in-person interviews, interviewers attempted to reach participants until they responded and completed follow-up interviews over the phone.
This study was approved by the New York City Department of Health and Mental Hygiene's institutional review board.
Survey instrument
Between March 2013 and June 2013, the quantitative baseline survey was drafted and revised iteratively, in consultation with experts from the field; we conducted several rounds of cognitive pretesting, speaking with approximately 20 targeted community members and peer educators from various SEPs total to determine whether our questions were accurately measuring the constructs we intended and to ensure we were using appropriate, relevant, and simple language. During survey development, the authors looked at existing surveys on knowledge and self-efficacy, including the Opioid Overdose Knowledge Scale (OOKS) and the Opioid Overdose Attitude Scale (OOAS) by Williams et al., 9 but decided not to use either due to survey length; knowledge and self-efficacy were secondary outcomes for this study, and we sought to assess knowledge and self-efficacy in fewer questions than included on either instrument. Our survey questions were designed to reflect independent and dependent variables we identified through our study research questions. Questions about ethnicity and education were borrowed from the Community Health Survey (CHS). 10 After the first few days of survey administration, meetings were held for survey administrators to discuss issues with flow and grammar; changes were made to the questionnaire accordingly. This process and the process of cognitive pretesting were repeated during the development of the 12-month follow-up survey. Questions assessing knowledge were not added to the survey until after baseline surveys and study enrollment were already completed, and therefore knowledge was not assessed at baseline. The survey was translated and back-translated into Spanish by professional translators and then tested with native Spanish speakers from the target audience.
Measures
All participant characteristics are based on self-report at baseline. Participants reported race (non-Hispanic white, non-Hispanic black, Hispanic, and Other); gender (male, female, transgender); and education level (less than high school; grade 12 or GED; 1 to 3 years of college; and 4 or more years of college). Criminal justice involvement was defined as having been arrested, on probation, or on parole in the last 12 months, not counting minor traffic violations. Naloxone use was defined as having ever used naloxone on another person. Participants were asked about last 30-day substance use, including alcohol, heroin, and cocaine or crack. Participants were also asked about prior 30-day use of methadone, prescription painkillers, and benzodiazepines, but the survey did not distinguish prescribed from illicit use. Participants reported frequency of prior-year injection drug use as always, sometimes, rarely, or never. We asked participants to report current use of syringe exchange program services and methadone maintenance treatment. We recorded whether participant OPT took place in an indoor site or outdoor site.
Twelve months after OPT, we measured knowledge of naloxone efficacy to reverse specific drug overdoses, knowledge of actions to take when witnessing an overdose, and self-efficacy:
Naloxone efficacy
Participants were asked to identify which drug overdoses naloxone would reverse: “If someone was to overdose on ____, do you think naloxone would work, or do you not know?” Each drug choice was read aloud, with brand names when relevant (e.g., Vicodin, OxyContin, etc.), and participants responded with yes, no, or don't know. Correct choices included prescription painkillers, methadone, and heroin. Incorrect choices were alcohol, benzodiazepines, and cocaine/crack.
Actions to take when witnessing an overdose
In an open-ended, field-coded knowledge question, participants were asked to “walk the interviewer” through the actions they would take to help someone having an overdose. Survey administrators listened to responses and marked all mentioned actions off of a predetermined list of possible responses (Figure 1). The 6 correct answers, as defined by OPT curricula provided by the Harm Reduction Coalition, 11 were check for breathing, administer the sternum rub, call 911, give rescue breathing, administer naloxone, and put the victim in the recovery position. Although overdose prevention curricula suggest that these 6 actions be done in chronological order, participants were not asked to relay the actions in any order; the order in which respondents mentioned overdose response actions was neither recorded nor used as a marker of knowledge. Mentions of either “mouth-to-mouth” or “CPR” (cardiopulmonary resuscitation) were counted as having correctly mentioned rescue breathing. Individuals who mentioned CPR were asked to confirm inclusion of rescue breathing. Mention of chest compressions only (without rescue breathing) was not coded as a correct answer. Seven predetermined, incorrect answers were recorded, including giving a milk, salt, or cocaine shot, putting ice on the victim's genitals, hitting or burning the victim, running away or leaving the victim, or giving chest compressions without rescue breaths.
Question from 12-month follow-up survey assessing knowledge of actions to take when witnessing an overdose one year after receipt of overdose prevention training.
After responses for both knowledge questions were recorded at 3 and 6 months, interviewers provided participants with the correct answers. Although we understood that reeducating participants in this way could have biased our results, we felt that it was our ethical obligation to inform participants when they had incorrect knowledge and to confirm when they had correct knowledge.
Self-efficacy
We asked participants: “When you have your [naloxone] kit on you, how confident do you feel about your ability to reverse an overdose?” Confidence was measured using a 4-point Likert scale, ranging from not at all confident to very confident. Self-efficacy was also measured by asking participants whether they had shown someone else how to use naloxone, assuming trainees who feel most confident are more likely to share their knowledge.12
Analysis
Participant and training characteristics were described using univariate frequencies. Participants who reported “Other” race were excluded from analysis due to small cell size. For participant gender, transgender was categorized as male or female based on the participant's presented gender. Education level was dichotomized into less than college and some college or college graduate. Last-year injection drug use was dichotomized into yes and no. Confidence was collapsed into 3 categories (very confident, somewhat confident, and a little or not at all confident).
A 6-point knowledge scale was created to calculate the number of correct overdose response actions mentioned, with each correct action worth 1 point.
t tests and analysis of variance (ANOVA) were used to assess the relationship between participant characteristics (demographic characteristics, service utilization, injection drug use, training site, lifetime naloxone use, number of follow-up surveys completed, and self-confidence) and knowledge scale results. All variables significant (P < .05) at the bivariate level were included in a multivariable linear regression model. Analyses were conducted using IBM SPSS Statistics version 19 (IBM, Armonk, NY).
Results
Sample description
Of the 679 individuals completing OPT at the study sites between June 2013 and January 2014, evaluation staff were able to approach 433 (64%) for participation in the study, and 355 (52%) were enrolled. All but 4 participants were enrolled from SEPs, but not all participants enrolled at SEPs were SEP clients—some individuals were recruited while walking by mobile syringe exchange locations. The remaining 4 participants were enrolled from an OPT held at NYC DOHMH. At the 12-month follow-up, 6 participants were deceased, 17 incarcerated, and 3 were in residential treatment for substance use disorder and unreachable. A total of 327 participants remained eligible, and 274 were reached for follow-up, for a response rate of 84%. One person declined to answer the majority of survey questions and was removed from analysis. Of the 273 remaining participants, 115 (42%) received their OPT indoors and 158 (58%) were trained outdoors (on street corners or in the back of a van). See Table 1 for a description of the 273 participants.
Mean knowledge of overdose response steps at 12 months, by participant and training characteristics.
Statistically significant values are presented in bold.
P values (P < .05) indicate statistically significant differences in mean knowledge.
Total less than 273 due to a small number of participants in “Other” racial category set to missing.
Total less than 273 due to missing data.
Use of licit medications were not distinguished as prescribed or nonprescribed.
Naloxone efficacy
When asked, “If someone overdosed on (alcohol, prescription painkillers, benzodiazepines, cocaine or crack, methadone or heroin) would naloxone work, or do you not know?” nearly all participants (99%; n = 269) knew that naloxone worked for heroin and 77% (n = 209) knew that it worked for prescription painkiller overdose (Table 2). Sixty-eight percent (n = 185) knew that naloxone worked for methadone overdose. Most participants (87%; n = 237) knew that naloxone did not reverse the effects of alcohol, and 74% (n = 203) knew that it did not affect cocaine or crack. Only 48% (n = 130) knew that naloxone would not reverse a benzodiazepine overdose. Over one quarter of participants (27%; n = 73) gave a correct response for all 6 drugs listed, and another quarter (27%; n = 73) responded correctly for 5 out of 6 of the drugs.
Knowledge of naloxone efficacy (N = 273).
Knowledge of overdose response actions
The 2 actions most commonly mentioned as overdose responses were giving naloxone (86%; n = 236) and calling 911 (76%; n = 206) (Table 3). More than one third of participants (37%; n = 102) mentioned that they would check for breathing or perform rescue breathing (37%; n = 100). Nearly one quarter (23%; n = 62) mentioned that they would perform a sternal rub. Only 14% (n = 38) mentioned they would put the victim in the recovery position.
Knowledge of the 6 correct actions* to perform when witnessing an overdose (N = 273).
Check for breathing, give sternal rub, call 911, give rescue breathing, administer naloxone, put on side.
Mean participant knowledge score was 2.7. Mean scores were greater for participants who attended college compared with those who had less than a college education (3.2 vs. 2.6; P < .001), SEP enrollees versus nonenrollees (2.9 vs. 2.5; P = .006), recent alcohol users versus nonusers (2.9 vs. 2.6; P = .03), recent benzodiazepine users and nonusers (2.9 vs. 2.6; P = .04), between those who had ever administered naloxone in their lifetime and those who had not (2.9 vs. 2.6; P = .01), and between those who felt very confident (mean score [ms] = 2.9), somewhat confident (ms = 2.4), or a little or not at all confident (ms = 1.5) in their ability to reverse an overdose (P < .001). Indoor-trained participants had a higher mean score than outdoor-trained participants (3.0 vs. 2.5; P = .002). There were no significant differences in knowledge at 12 months by gender, race, criminal justice involvement, overdoses witnessed, or opioid treatment program participation. Despite our reeducation of participants each time they completed a follow-up survey, there were no significant differences in knowledge at 12 months by number of follow-up surveys completed. In a linear regression model, education, confidence level, training location, and benzodiazepine use were independently associated with differences in knowledge scores at the multivariate level (Table 1).
The most commonly mentioned incorrect actions were to put ice on the victim's genitals (6%; n = 15) and to give chest compressions without rescue breaths (4%; n = 12). Three percent (n = 8) said they would hit or burn the victim to wake them up, and 2% (n = 5) said they would administer a “salt shot.” Less than 1% of participants said they would give a “milk shot” (0.7%; n = 2) or that they would leave the overdose scene (0.3%; n = 1). No participants said they would give the victim a cocaine shot.
Self-efficacy
The majority of participants (96%) reported feeling very confident (75%; n = 204) or somewhat confident (21%; n = 56) in their ability to reverse an overdose 12 months after receiving their overdose prevention training (Table 1). Over half of participants (58%; n = 157) reported showing their naloxone to someone else in the 6 months prior to their 12-month follow-up survey, suggesting that OPTs may have an impact on individuals beyond those who are trained.
Discussion
Twelve months after a cohort of individuals received overdose prevention training and naloxone from NYC Opioid Overdose Prevention Programs, we found that knowledge of naloxone's drug-specific effectiveness, knowledge of the 2 most important overdose response actions (giving naloxone and calling 911), and self-efficacy were high. Knowledge of all 6 correct overdose response actions was poor.
Almost all participants knew that naloxone would reverse an overdose related to heroin, and about three quarters prescription painkillers. Sixty-eight percent knew that naloxone would reverse a methadone overdose—highlighting an important gap in knowledge, as methadone was involved in 21% (168 deaths) of NYC overdose deaths in 2013. Over one half of participants incorrectly believed naloxone would reverse a benzodiazepine overdose, which may be a result of overdose prevention trainers urging trainees to use naloxone whenever a victim is showing signs of any type of depressant overdose. This practice may have led to a belief among OPT recipients that naloxone works on any drug with depressant effects, including benzodiazepines.
Higher levels of confidence in the ability to reverse an overdose and receipt of indoor training were associated with greater knowledge of the 6 correct overdose response actions. Although we cannot be certain about the direction of the relationship between confidence and knowledge, it is likely that having greater knowledge makes participants feel more confident, rather than vice versa. It is possible that greater knowledge among those trained indoors was attributable to experiencing fewer distractions such as street noise, interruptions, inclement weather, and active syringe exchange. We hypothesize, however, that the difference in knowledge was primarily due to training duration. Although we did not systematically collect training length, the interviewers observed that indoor trainings were generally 10 to 25 minutes in length, whereas outdoor trainings were typically 5 to 10 minutes in length; additional time allowed for more detailed and focused instruction.
Our findings may differ from those of a recent study that found that 5- to 10-minute overdose education was sufficient for improving knowledge of opioid overdose response among first-time recipients of naloxone and among individuals returning for a naloxone refill. 10 Among those returning for refills, the mean number of days before return was 392; 69% of participants had previously received at least 1 refill, with a mean of 5.5 refills and 3.6 reversals. The high levels of knowledge may be explained by inclusion of participants with strong program engagement, as evidenced by the frequency of refills and reversals among the group. Additionally, their study was more lenient in measuring knowledge. Knowledge was measured by asking training participants to list what actions they would take to assist in an overdose event at each assessment, then coding all responses as “beneficial” or not; the number of beneficial responses were compared from pre- to posttest. In contrast, our outcome measure limited knowledge to only 6 specific actions included in the OPT curriculum and may therefore have excluded some beneficial responses, underestimating participants’ knowledge. For example, we did not include checking for a pulse, shaking the victim, calling the name of the victim, checking the victim's pupils, checking what the overdose was on, or giving chest compressions as correct responses, although these actions may be helpful during an overdose event.
In order to continue to aid overdose victims and improve overdose mortality rates, OPTs should emphasize that naloxone will work on all opioid overdoses—including methadone overdoses—and should make greater efforts to emphasize all 6 of the overdose response actions. OPTs should continue to highlight the importance of using naloxone and calling 911, and should also continue to address myths about incorrect ways to reverse overdoses. Although our findings suggest that indoor training is associated with greater levels of knowledge, we recognize the importance of street-based outreach and acknowledge that indoor trainings are not always feasible. When possible, however, outdoor trainings should strive to be as comprehensive as indoor trainings.
Limitations
This study had several limitations. Knowledge of naloxone efficacy and of the correct overdose response actions were not measured at baseline, making it impossible to measure change in knowledge from baseline to 12 months. We cannot know whether participant knowledge improved for all participants, or whether indoor-trained participants had higher levels of knowledge at the outset. Additionally, for those participants enrolled in an SEP, it is possible that the experience of participating in such a program meant higher knowledge at baseline due to prior exposure to OPTs. If this were the case, however, it is likely that SEP participants would continue to have higher knowledge scores than non-SEP participants at 12 months, due to continued exposure to OPT. Although we did find that SEP participants had significantly higher mean knowledge scores at the bivariate level, this difference in knowledge disappeared after controlling for education, location of OPT, confidence, and benzodiazepine use.
A second limitation was the spectrum of teaching styles and expertise amongst trainers of overdose prevention—as well as the length, scope, and setting of trainings; the quality of information delivered was inconsistent across OPT sites, and outdoor trainings experienced greater distractions such as cold weather and interruptions from the street. Participant retention also introduced bias—those participants with working telephone numbers, working addresses, or reliable connections with programs were those who we were able to follow up with over the course of the study. The loss of participants without access to these services may have left us with a sample whose knowledge and self-efficacy were higher than the general OPT-accessing population.
Despite these limitations, this is the first study in the United States to look systematically at knowledge and self-efficacy among opioid overdose prevention trainees 12 months after training. The study shows that drug users trained in overdose prevention know that they should use naloxone and call 911 during an overdose 1 year after being trained. Participants also have high levels of confidence in their ability to respond to an overdose 1 year after training. Individuals who attend overdose prevention trainings at NYC OOPPs have the requisite knowledge and confidence to respond to overdose in their communities and are key responders in reducing overdose mortality.
Footnotes
Acknowledgments
The authors thank Zina Huxley-Reicher for her hard work developing and evaluating this study.
Author contributions
Dr. Siegler conceived of, designed, and implemented the overall study, and oversaw all aspects of participant recruitment and follow-up, data management and analysis, results interpretation, and manuscript drafting. She made critical revisions to all manuscript drafts. Dr. Kunins provided supervision and oversight of study design, including sample size and population specification, development of recruitment and follow-up strategy, and questionnaire development. With coauthors, she planned the analysis for this manuscript. With coauthors, she interpreted results and developed paper outline and presentation of results. She made critical revisions to all manuscript drafts.
Funding
Funding for study design and implementation, data collection and analysis, and manuscript preparation and approval was provided by the Department of Health and Mental Hygiene. No external funding sources were used.