Development of Novel Video-Based First Responder Opioid Hazard Refresher Training

Abstract

First responders encounter many hazards in the execution of their duties, and exposure to hazardous materials such as opioids is a primary safety concern. The ongoing opioid crisis in the United States continues to be a major public health issue, with overdose deaths from opioids reaching epidemic levels. Although responders frequently encounter opioids, available data on safety and risk are not always well-communicated, and we identified a need for refresher and just-in-time training products on this topic. In response, we created a training video series that is informative, concise, and visually appealing. The video series, available on YouTube, was tested with a small initial population, with findings suggesting key questions for a larger study focused on integration of the refresher training with existing programs to optimize retention and adoption of safety practices.

Keywords

opioids first responders training personal protective equipment

Introduction

First responders encounter many hazards in the daily execution of their duties, and exposure to hazardous materials in the form of opioids is a growing safety concern. The ongoing opioid crisis in the United States, referring to the increase over several decades of both prescription and nonprescription opioid misuse, is a major public health issue, with opioid overdose deaths reaching epidemic levels.¹ In 2019, the most recent year for which final data are available, a total of 49,860 drug overdose deaths involved opioids, with 36,359 deaths (73%) involving synthetic opioids.² Potent synthetic opioids (mainly fentanyl and its analogs) have become widely available throughout the United States on the illicit drug market, with the number of samples tested by state and local laboratories increasing from 640 in 2010 to over 14,000 in 2015.³ The Centers for Disease Control and Prevention (CDC) attribute the most recent wave of opioid overdose deaths to synthetic opioids, and especially fentanyl, in contrast to previous waves of deaths linked to heroin and prescription opioids.¹ While fentanyl is approved by the U.S. Food and Drug Administration (FDA) for use as a medical anesthetic and analgesic, it is also illicitly manufactured and sold for its desirable recreational effects. One reason behind the rapid spread of fentanyl is that it is one of the most profitable illicit drugs for traffickers, with profit margins of up to millions of dollars per pure kilogram.⁴ Because of the high profitability of fentanyl, its ease of production and trafficking, and its appeal to those with opioid use disorder, it is likely to continue to be widely available for the foreseeable future. Furthermore, increasingly potent fentanyl analogs have been reported in the illicit market, including carfentanil, an opioid 10,000 times more potent than morphine, which is used as a large animal anesthetic in veterinary medicine.^4,5

The ongoing global pandemic of novel coronavirus disease 2019 (COVID-19), caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), has intersected with the opioid crisis and worsened it. Provisional mortality data through November 2020 published by the National Center for Health Statistics (NCHS) indicate that reported drug overdose deaths attributed to opioids in 2020 are set to exceed the previous record set in 2019.⁶ The causes of this increase are multifactorial, with proposed contributing factors including job loss and unemployment, social isolation during the pandemic, and the closure of in-person counseling and treatment centers. Furthermore, the pandemic has limited access to routine medical services, including those for chronic pain management, which may push some toward acquiring opioids on the illicit market.⁷

In recent years, as fentanyl and related substances have become more widespread in the United States, reported first responder encounters with fentanyl have increased substantially, specifically for responders involved in law enforcement and pre-hospital patient care.⁸ For example, the most recent data from the CDC show that from 2014 to 2015, eight states reported more than 500 law enforcement fentanyl encounters (i.e., drugs submitted for analysis) in 2015, compared to two states in 2014 and zero states in 2013. The total number of fentanyl encounters doubled from 5343 in 2014 to 13,882 in 2015, and this trend continues.⁹ With each encounter, the lives of first responders are put at risk, as exposure is a serious threat that requires training to ensure that personnel are protected.^10,11 In addition, guidance for first responders during the COVID-19 pandemic may conflict or complicate guidelines for responding to opioid exposure incidents. For example, although the use of hand sanitizer is recommended as a countermeasure against SARS-CoV-2, it is not recommended when opioids may be present, as it can increase the absorption of opioids through the skin.⁸

Although first responders frequently encounter opioids, the available information on safety and risk is not always well-communicated, which may lead responders to over- or underestimate risks in certain situations. For example, a joint statement released by the American College of Medical Toxicology (ACMT) and the American Academy of Clinical Toxicology (AACT) describes how the risk of fentanyl toxicity through skin exposure has been exaggerated by some sources, which may lead first responders to use excessive personal protective equipment (PPE) that may interfere with task performance and therefore introduce new risks.¹² Therefore, there is a clear need in the community for educational materials, and especially refresher and just-in-time trainings, that train first responders in how to protect themselves while still performing their duties and providing timely care. Such training tools may even be used by people who routinely encounter opioid overdoses but are not traditionally considered first responders, such as librarians and retail workers.¹³

Previous web-based training tools have been published by private companies, nonprofits, and government agencies that attempt to train both first responders and lay people on how to respond to an opioid-related overdose emergency, mostly focused on the use of naloxone (a medication designed to rapidly reverse opioid overdose).^14,15 These tools were very well-received by the target training populations, with a majority of the trainees reportedly being satisfied with the course’s content, format, and mode of delivery—indicating that internet-based trainings are viable tools. However, more generally, there is a dearth of information related to in-service (i.e., post-recruitment) responder training. Notably, a proposed systematic review of recent (2000–2015) research into in-service police training failed due to a lack of peer-reviewed, published research in this area.¹⁶ In addition, other studies conducted with laypeople and healthcare professionals trained in cardiopulmonary resuscitation and other basic rescue skills have shown a pattern of decreasing skill after certification, suggesting that short, easily accessible refresher trainings may have an important role in maintaining skills for this population.^17,18

Here, we describe the development of a YouTube training series for first responders: Preventing Opioid Exposure Training for First Responders (POET). The video format and content were developed based on interviews with nine active first responders and best practice documents published by federal agencies and industry groups. The videos are engaging, visually appealing, and concise, making them well-suited to use as refresher and just-in-time trainings.

Materials and Methods

Participants

The study involved 27 first responders in two groups. The first focus group of nine participants was interviewed as part of formative research to collect requirements from the training community for the development of the virtual training product in accordance with best practices in user-centered design principles. The second study group included 18 first responders who took the online training course and were tested for knowledge gains and retention. The second study group was provided with an incentive to improve study retention and received $20 for participating in the initial testing and $10 for participating in the follow-up testing. We applied for and were granted an educational exemption for IRB Review from the Gryphon Scientific Institutional Review Board (IRB00011291).

User-Centered Design Focus Group: Formative Research Interviews With First Responders

Following best practices in user-centered design, we gathered individual perspectives through interviews with first responders across the United States, including law enforcement, fire service, and emergency medical service (EMS) personnel to inform the design of the initial training product. The goal of these discussions was to collect data on workplace training preferences (e.g., in-person training versus e-learning courses) and viewpoints from responders about their experiences with opioid exposure in the workplace and previous trainings on occupational opioid exposure and related topics. Responses to the interview questions were anonymized and all efforts were made to ensure that references to specific individuals or first responder department names or locations were deidentified. The insights from this focus group were used to design the training product that was tested by the training study group.

Virtual Training Product Development

We developed training videos for first responders that are published online on YouTube. Three main topics are covered in the video training series: (1) routes of opioid exposure and the risks associated with each; (2) PPE and situational awareness; and (3) recognizing and responding to occupational opioid exposure events. The curriculum content was developed based on research conducted with the first responder training community, and published guidance documents and best practices from federal agencies and industry groups, including the InterAgency Board (IAB) for Emergency Preparedness and Response, the Centers for Disease Control and Prevention (CDC), the National Institute for Occupational Safety and Health (NIOSH), and the National Institute of Environmental Health Sciences (NIEHS).^8,19–21 Best practices in adult learning were leveraged in designing the curriculum following Malcolm Knowles theory of andragogy, including highlighting why participants needed to learn the curriculum, emphasizing how it aligns/orients with participants’ workplace challenges, incorporating the real life experience of participants in module discussions, and allowing for self-direction with asynchronous learning options.²² We decided to leverage animation over static images to engage users, especially given that this this is a refresher training and this information should not be new to the intended audience.²³

Training Study Group: Testing of Opioid Exposure Training Videos

Following video development, a pilot study was conducted with 18 first responders to evaluate knowledge gains and retention following completion of the online training. This testing was conducted remotely due to the social distancing recommendations of the COVID-19 pandemic. First responders were recruited using snowball sampling, starting with contacts and acquaintances of the research team, and respondents were asked to complete online questionnaires. Demographic information was collected, including information about current occupational roles, prior opioid training in the workplace, and familiarity with e-learning. The respondents were then asked to complete a pre-video knowledge check consisting of eight questions about a variety of topics, including potential opioid exposure routes, the signs and symptoms of opioid exposure, and the appropriate actions that first responders should take to prevent occupational exposures. The survey participants were then asked to watch the three videos consecutively and answer the same eight questions in a randomized order for the post-video knowledge check. Participants also answered questions on the quality and relevancy of the video series to their roles as first responders. Lastly, all responders were asked to participate in a three-month follow-up survey, which evaluated knowledge retention by re-asking them the eight knowledge-check questions.

Statistical Analysis

Overall test scores were calculated from the pre-test, post-test, and three-month follow-up knowledge-check questions for each participant for comparison using one-sided Wilcoxon Rank Sum Test scores.²⁴ This specific test of differences was used due to the relatively low number of respondents (n = 18, n = 10) in the two comparison groups. A one-sided test with a significance level of 0.05 was conducted between pairs of scores (pre-test to post-test, pre-test to follow-up, and post-test to follow-up) to determine whether the group post-test and follow-up test scores were greater than the pre-test and post-test group scores.

Results

User-Centered Design Focus Group: Formative Research Interviews With First Responders

We conducted initial research with the user-centered design focus group (UCDFG), a group of nine first responders, including law enforcement, fire service, and EMS personnel, as well as emergency management professionals with previous first responder experience (Table 1). Interviewees were located across the United States but primarily in major cities heavily impacted by opioids, including Denver, New York City, and Seattle. This group generally mirrored the overall first responder population in gender (78% male compared to 75%–95% across the national population) and in age, as national-level data show that the vast majority of first responders are twenty-five to fifty-four years of age.²⁵ However, the group, which was 100% white, was not representative of the broader first responder population (65.4%–78.6% white), which may be a by-product of the relatively small sample size. The UCDFG received structured interviews that focused on several topic categories, including previous experiences with opioid exposure in the workplace, previous trainings on occupational opioid exposures, and workplace training preferences.

Table 1.

Demographic Characteristics of UCDFG and Study Group Participants.

	UCDFG(n = 9)	Study group(n = 18)
	n (%)
Gender
Male	7 (78)	11 (61)
Female	2 (22)	7 (39)
Race
White	9 (100)	14 (78)
Black or African American	0 (0.0)	2 (11)
Asian	0 (0.0)	1 (6.0)
Other	0 (0.0)	1 (6.0)
Age
18 to 24	0 (0.0)	1 (6.0)
25 to 44	5 (56)	13 (72)
45 to 64	4 (44)	3 (17)
65 and older	0 (0.0)	1 (6.0)
Self-identified occupation
Firefighter	1 (11)	3 (17)
Paramedic	1 (11)	3 (17)
EMT	1 (11)	3 (17)
Firefighter/paramedic	1 (11)	3 (17)
Firefighter/EMT	1 (11)	0 (0.0)
Police	0 (0.0)	5 (28)
Physician	1 (11)	0 (0.0)
Emergency management	2 (11)	0 (0.0)
Other^a	1 (11)	1 (5.6)
Prior online opioid training
Yes	–	17 (94)
No	–	0 (0.0)
Unsure	–	1 (6.0)

UCDFG = user-centered design focus group; EMT = emergency medical technical.

^aOther includes one individual in the focus group who was a firefighter, police officer, EMT, and FEMA manager; and one individual in the study group who only listed their primary occupation (as an engineer) and not their specific role as a first responder.

Table 2.

Test Scores Before Training and After Training.

	Mean score % ± SD	Median score % (range)	p value^a
Before training	78.3 ± 9.2	80.0 (60.0–90.0)	–
After training	88.9 ± 8.3	90.0 (80.0–100)	.003

SD = standard deviation.

^ap value was calculated by comparing the after-training score with the before-training score.

One of the main and consistent findings from the interviews was a clear preference for training that could be used during daily ten-to-fifteen-minute briefings provided to incoming shifts. According to the responders we spoke to, daily briefings can include presentations about single topics and/or group discussions. The UCDFG also indicated that if this training could additionally be repurposed for asynchronous learning for those unable to attend briefings, it would be seen as an added benefit.

The interviews also revealed several additional consistent themes. First, responders shared that emphasizing universal precautions and situational awareness in any training curriculum is important, as opioid exposures frequently accompany other threats, such as needlestick injuries. As daily briefings are short and additional opportunities for training are sparse, such overlap could also efficiently reinforce other related concepts. Second, several interviewees also stated that reasons for PPE use and how it protects responders from opioid exposure should be reinforced, as nitrile glove and respirator use is reportedly low in occupational groups. Third, several respondents emphasized that in their opinion, there is no such thing as too much training, as material in their first responder departments is often taught just once but never revisited or reinforced. Finally, interviewees expressed comfort and familiarity with online trainings, and thought they integrate well into existing procedures.

Development of Opioid Exposure Training Videos

The data from the UCDFG were synthesized and training options were evaluated. We determined that short and engaging videos, structured as stand-alone topics and incorporating an optional peer discussion component, could be an optimal training tool for the first responder population. In addition, such a training video series could be easily refreshed and updated with new chapters. In addition, the training could also be viewed as stand-alone products for those unable to attend briefings.

A series of three training videos for first responders was developed and published on YouTube, containing content based on federal agencies and industry group recommendations and subject matter expertise guidance, leveraging best practices in adult education for course design.^a Three main topics are covered in the video training series: (1) routes of opioid exposure, (2) PPE and situational awareness, and (3) recognizing and responding to occupational opioid exposure events. The videos are fully animated and incorporate voice-overs (Figure 1).

Figure 1.

Screenshots from the animated opioid exposure training videos.

The videos are structured to reinforce a short list of learning goals for each topic: first presenting discussion questions and encouraging viewers to pause and discuss, then reviewing the course material, and lastly returning to the discussion question answers.

Testing of Opioid Exposure Training Videos

Following video development, we tested knowledge gains and retention to assess whether the training improved the capacity of first responders to recognize and manage opioid exposure risks (Table 2). Overall, eighteen first responders participated in the video series testing, belonging to three occupational roles: 50 percent were emergency medical technicians (EMTs) or paramedics, 28 percent were firefighters, and 22 percent were police officers. The study group, like the focus group, was generally representative of the average age and gender of the national first responder population.²⁵ It was also more racially diverse than the focus group and better aligned to the target population of responders.

All but one responder (94%) stated that they had previously received workplace opioid exposure training, while the remaining respondent reported that they were unsure if they had received training. When asked about their familiarity with online training courses on a five-point semantic differential scale (with 1 being not at all familiar and 5 being very familiar), the average response was 4.4, indicating that most respondents were very familiar with e-learning. Of these respondents, six had previously taken online courses, and specific training platforms that responders had previously used included Blackboard, SmarterU, and Target Solutions (now Vector Solutions). Most testing participants also responded that they believed they could safely respond to incidents involving opioids at work.

The average score in the pre-video knowledge check was 78.3 percent across the eighteen first respondents, with a median of 80.0 percent (range: 60.0%–90.0%). Generally, participants scored lowest on questions that asked about what PPE was most appropriate to use in specific scenarios or situations, while they scored highly on questions about what actions to take in an exposure situation. After watching the videos, the median score in the post-video knowledge check across the eighteen first responders was 90 percent (range: 80.0%–100%; mean 88.9%), a median improvement of 10 percent from the pre-video knowledge check. Using a one-sided, paired Wilcoxon Signed Rank Test with a significance level of .05, the median score in the post-test group was found to be greater than the median score in the pre-test group (p = .003). In our testing sample, seventeen of eighteen individuals stated that they had previously received opioid exposure training or had familiarity with online training courses; due to this imbalance in participants with and without previous training exposure, it was impossible to evaluate whether previous training is correlated with score improvement.

At the three-month follow-up period, ten out of eighteen (55.5%) respondents participated in the follow-up knowledge check, significantly reducing the size of the study sample. In a one-sided, paired Wilcoxon Signed Rank Test, there was no evidence of a greater score in the follow-up knowledge check score compared to the pre-knowledge check (p = .397) or compared to the post-knowledge check group (p = .978).

Finally, the participants were asked to rate the training videos on a series of five-point semantic differential scale questions related to the relevancy of the videos to their job duties and various aspects of the videos themselves (e.g., topics, engagement, presentation, and graphics). Overall, the videos were very well-received, with an average rating of 4.9 on all questions (range: 4–5). Specifically, when asked if these videos made responders more confident to carry out their duties, all eighteen participants answered that they agreed or strongly agreed.

Limitations

The primary limitation of our study was the overall small study size, which was limited by funding. The focus group was very small and not necessarily representative of the broader population of first responders in the United States. Using a larger and more diverse focus group may have made the training video content more relevant to a wider group. Furthermore, due to our study size, the already high level of knowledge of our cohort pre-test, and loss to follow-up in our test population, the significance of gains seen from our training product was not replicated from the immediate post-training test to the longer-term knowledge test. A second limitation that we identified was the lack of background knowledge on the quality of opioid protection programs that the study participants had previously been exposed to or participated in. This training product is designed to supplement existing programs, and having a better baseline understanding of existing training is necessary for evaluation. Responders coming from departments with a strong culture of training and safety may have achieved higher scores on the knowledge tests while not necessarily being the intended audience for this product. Follow-up studies should include questionnaires regarding funding and culture for training and safety (especially related to opioid exposures) as part of pre-testing activities.

Discussion

The Preventing Opioid Exposure Training for First Responders YouTube series was developed with input from the first responder community and well-received by a small cohort of testers involved in testing the initial product. There were statistically significant gains in knowledge immediately following initial training; however, it is not clear that these gains are retained over time. Previous studies have demonstrated that first responders experience rapid knowledge decay following training, with some suggestion that training repetition is more important than days since training for knowledge retention.^26,27 A larger study is needed to better assess the efficacy of the training product for medium- and long-term knowledge gains, but also to determine how the POET training integrates with the overall training suite that first responders are receiving to understand true potential efficacy.

Future work should focus on a larger scale study, with better integration with existing programs and a more diverse cohort. As a refresher training to aid knowledge retention, understanding how this program supplements existing trainings is crucial. Frequent refreshers and drills have been demonstrated to improve the preparedness of emergency response teams in the workplace, but to understand the efficacy of POET in this context the entire training program would have to be studied as a whole.²⁸ Future studies should collect more comprehensive data on baseline training to understand how knowledge gaps emerge over time and ensure that we are accurately targeting those gaps with this training. It would also be ideal to investigate not only how participants retain knowledge through testing, but if true behavioral change is also observed. To accomplish this type of study, we would need to partner with first responder organizations interested in improving their safety and training programs.

In addition, future work will focus on how to improve long-term knowledge retention in this training community. Loss of learned knowledge over time is a well-studied phenomenon, dating back to the 1880s with Ebbinghaus’ Forgetting Curve.²⁹ Recently, progress has been made in using mobile applications to increase knowledge retention using microlearning.³⁰ We will explore how to expand our curriculum using microlearning and other e-learning approaches to improve knowledge retention and design a follow-up study of sufficient power to detect long-term knowledge retention of training in the first responder community. Learning retention could further be improved by testing different behavioral change theories to determine if specific approaches resonate with our training population and drive learning retention and, ideally, behavior change.

Notably, YouTube has been used as a mainstream supplemental education resource for blended learning and self-directed learning with previous successes.^31–34 The first responder community overwhelmingly provided positive feedback in response to the training during the review process and expressed a desire to share the training videos with their community. A secondary distribution mechanism for this training will be organic searching and sharing, with users accessing the information they need just-in-time. First responders increasingly encounter synthetically produced opioids, such as fentanyl and various chemical analogs of fentanyl, that are far more potent than heroin or prescription opioids. Therefore, it is incredibly important to provide as many training opportunities and tools as possible to help first responders safeguard themselves and their colleagues. POET provides an on-demand resource that can be accessed anytime, anywhere to refresh on the basic principles of worker safety, which is a beneficial addition to the training tool kit for first responders as we work toward continuing to improve the delivery of the material of optimal retention.

Disclaimer

The findings and conclusions in this study are solely those of the authors and do not necessarily represent the views of the National Institute of Environmental Health Sciences (NIEHS) or the National Institutes of Health (NIH).

Footnotes

Declaration of Conflicting Interests

The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: The authors wish to disclose that The Preventing Opioid Exposure Training for First Responders YouTube series was developed under a Small Business Innovation Research grant from the NIEHS to Gryphon Scientific with the eventual goal of commercialization. All authors are employed at Gryphon Scientific, which may benefit financially from commercialization of this product in the future. Dr. Margaret Rush is a partial owner of Gryphon Scientific.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was funded by the National Institute of Environmental Health Sciences under grant number 1R43ES030582-01.Note

ORCID iDs

Audrey A. Cerles

Margaret Andrews Rush

Note

Author Biographies

Audrey A. Cerles, MS, is a microbiologist with expertise supporting risk assessments in various fields, including emerging infectious diseases and emerging biotechnologies. Cerles holds a BA degree in microbiology from Cornell University and an MS degree in food safety and defense from Michigan State University, where she conducted research into food additive contamination. Since joining Gryphon Scientific in 2013, Cerles has also worked on developing various training programs related to insider threat awareness and first responder safety.

Nathan N. L. Dinh is a graduate from the University of Richmond with a BS degree in Biochemistry & Molecular Biology. While at the university, he conducted research on nanotechnology for ceramic glazes and worked with the NIH on policy analysis. Following graduation in 2020, Nathan joined Gryphon Scientific as a Research Assistant and has collaborated on various projects focused on COVID-19 and biosecurity issues.

Laurel MacMillan, MS, is a statistician with extensive experience solving difficult biosecurity and biosurveillance problems with advanced computation and inference methods. MacMillan holds a BA degree in mathematics from the University of Virginia and an MS degree in statistics from American University where she conducted NSF-funded research focused on the development of optimal statistical methods to test sequential data in multiple hypothesis problems. Since joining Gryphon Scientific in 2016, MacMillan has worked on a diverse range of applied statistical research for the U.S. government, including differential diagnosis of infectious diseases, spatial modeling of zoonotic disease risk in the United States, and the evaluation of biological changes resulting from acute radiation exposure.

Daniel C. Kemp is a biochemist with experience in emerging biotechnologies, biomedical chemistry, and biosurveillance. Kemp holds a BS degree in biochemistry from the University of Virginia, where he conducted research to synthesize novel ligands for metal-center complexes to create catalysts used for carbon dioxide reduction, as well as improve yield and efficiency for existing synthetic procedures. He also conducted research with Mote Marine Laboratory to assess the impacts of probiotic supplements on the growth and survival rate of hatchery-reared common snook, as well the health impacts of petroleum toxins on hatchery-reared red drum. After joining Gryphon Scientific in 2019, Kemp has aided in efforts including the cataloguing of medically useful chemical compounds, communication of novel findings relevant to novel coronaviruses in the areas of zoonotic transmission and occupational health, creation of safety training programs for first responders, and the review of current and previous gaps in national biosecurity.

Margaret Andrews Rush is an infectious disease biologist whose career has focused on helping clients harness the power of their scientific data to answer critical research questions in public health, with a focus on global health. Rush developed her analytics expertise while completing her PhD at Harvard focused on malaria drug discovery. After graduation in 2009, she joined Gryphon Scientific where she is a partner and leads projects to collect, curate, analyze, and model complex data for clients including CDC, USAID, FEMA, DOD, and NIH. Her training in biological sciences for public health and expertise in data science is complimented by a strong background in training on health and emergency preparedness.

References

Centers for Disease Control and Prevention. Opioid overdose, www.cdc.gov/drugoverdose/epidemic/index.html (2021, accessed June 17, 2021).

Mattson

Tanz

Quinn

, et al. Trends and geographic patterns in drug and synthetic opioid overdose deaths—United States, 2013-2019. MMWR Morb Mortal Wkly Rep 2021; 70: 202–207.

U.S. Drug Enforcement Administration. NFLIS Brief: Fentanyl, 2001-2015. Springfield, VA: U.S. Drug Enforcement Administration, 2017.

U.S. Drug Enforcement Administration. 2020 National Drug Threat Assessment. Springfield, VA: U.S. Drug Enforcement Administration, 2021.

Zawilska

Kuczyńska

Kosmal

, et al. Carfentanil—from an animal anesthetic to a deadly illicit drug. Forensic Sci Int. 2021; 320: 110715.

National Center for Health Statistics. Provisional drug overdose death counts, www.cdc.gov/nchs/nvss/vsrr/drug-overdose-data.htm (2021, 30 June 2021).

Manchikanti

Vanaparthy

Atluri

, et al. COVID-19 and the opioid epidemic: two public health emergencies that intersect with chronic pain. Pain Ther 2021; 10: 1–18. DOI: 10.1007/s40122-021-00243-2.

Centers for Disease Control and Prevention (CDC) and The National Institute for Occupational Safety and Health (NIOSH). Illicit drugs, including fentanyl: preventing emergency responders’ exposures to illicit drugs, www.cdc.gov/niosh/topics/fentanyl/risk.html (2020, accessed 30 April 2021).

Centers for Disease Control and Prevention. Reported law enforcement encounters testing positive for fentanyl increase across US, www.cdc.gov/drugoverdose/data/fentanyl-le-reports.html (2016, accessed 24 July 2018).

10.

Drug Enforcement Administration. Fentanyl: a briefing guide for first responders. Springfield, VA: U.S. Drug Enforcement Administration, 2017.

11.

Chiu

Hornsby-Myers

de Perio

, et al. Health effects from unintentional occupational exposure to opioids among law enforcement officers: two case investigations. Am J Ind Med 2019; 62: 439–447.

12.

American College of Medical Toxicology (ACMT) and the American Academy of Clinical Toxicology (AACT). ACMT and AACT position statement: preventing occupational fentanyl and fentanyl analog exposure to emergency responders, http://ww1.prweb.com/prfiles/2017/07/18/14520974/Fentanyl_PPE_Emergency_Responders_.pdf (2017, accessed 30 April 2021).

13.

Lowenstein

Feuerstein-Simon

Sheni

, et al. Public libraries as partners in confronting the overdose crisis: a qualitative analysis. Subst Abus 2019; 18: 1–8. DOI: 10.1080/08897077.2019.1691129.

14.

Simmons

Rajan

Goldsamt

, et al. Implementation of online opioid overdose prevention, recognition and response trainings for professional first responders: year 1 survey results. Drug Alcohol Depend 2016; 169: 1–4.

15.

Simmons

Rajan

Goldsamt

, et al. Implementation of online opioid overdose prevention, recognition and response trainings for laypeople: year 1 survey results. Subst Use Misuse 2018; 53: 1997–2002.

16.

Huey

What do we know about in-service police training? Results of a failed systematic review. London: University of Western Ontario Sociology Publications, 2018.

17.

Latman

Wooley

Knowledge and skill retention of emergency care attendants, EMT-As, and EMT-Ps. Ann Emerg Med 1980; 9: 183–189.

18.

Rhue

VanDerveer

Wilderness first responder: are skills soon forgotten?

Wilderness Environ Med 2018; 29: 132–137.

19.

Howard

Perspectives on the U.S. opioid crisis, www.niehs.nih.gov/news/events/pastmtg/hazmat/assets/2018/wtp_fall_18_keynote_address.pdf (2018, accessed 28 July 2021).

20.

National Institute of Environmental Health Sciences Worker Training Program. Prevention of occupational exposure to fentanyl and other opioids awareness training tool, https://tools.niehs.nih.gov/wetp/public/hasl_get_blob.cfm?ID=11206 (2018, accessed 30 April 2021).

21.

The InterAgency Board. Recommendations on Selection and Use of Personal Protective Equipment and Decontamination Products for First Responders Against Exposure Hazards to Synthetic Opioids, Including Fentanyl and Fentanyl Analogues, https://www.interagencyboard.org/sites/default/files/publications/IAB%20First%20Responder%20PPE%20and%20Decontamination%20Recommendations%20for%20Fentanyl.pdf (2017).

22.

Knowles

Holton

Swanson

RA.

The adult learner: the definitive classic in adult education and human resource development. Routledge: Taylor & Francis, 2014.

23.

Höffler

Leutner

Instructional animation versus static pictures: a meta-analysis. Learn Instruct 2007; 17: 722–738.

24.

Hollander

Wolfe

DA.

Nonparametric statistical methods. New York, NY: John Wiley & Sons, 1973.

25.

Schafer

Sutter

Gibbons

Characteristics of individuals and employment among first responders. Washington, DC: U.S. Department of Labor, 2015.

26.

Anderson

Gaetz

Masse

First aid skill retention of first responders within the workplace. Scand J Trauma Resusc Emerg Med 2011; 19: 1–6.

27.

Anderson

Gaetz

Statz

, et al. CPR skill retention of first aid attendants within the workplace. Prehosp Disaster Med 2012; 27: 312–318.

28.

Ruttenberg

Raynor

Tobey

, et al. Perception of impact of frequent short training as an enhancement of annual refresher training. New Solut 2020; 30: 102–110.

29.

Murre

Dros

Replication and analysis of Ebbinghaus’ forgetting curve. PLoS One 2015; 10: e0120644.

30.

Shail

MS.

Using micro-learning on mobile applications to increase knowledge retention and work performance: a review of literature. Cureus 2019; 11: e5307.

31.

Goolsby

Strauss-Riggs

Klimczak

, et al. Brief, web-based education improves lay rescuer application of a tourniquet to control life-threatening bleeding. AEM Educ Train 2018; 2: 154–161. 2018/07/28. DOI: 10.1002/aet2.10093.

32.

Grønlien

Christoffersen

Ringstad

, et al. A blended learning teaching strategy strengthens the nursing students' performance and self-reported learning outcome achievement in an anatomy, physiology and biochemistry course—a quasi-experimental study. Nurse Educ Pract 2021; 52: 103046.

33.

Müller

Tipold

Ehlers

, et al. TiHoVideos: veterinary students’ utilization of instructional videos on clinical skills. BMC Vet Res 2019; 15: 326–309.

34.

Tamler

Green

Skamagas

, et al. Durability of the effect of online diabetes training for medical residents on knowledge, confidence, and inpatient glycemia. J Diabetes 2012; 4: 281–290.