Oncology boot camp: A preparatory curriculum for advanced pharmacy practice experience students

Abstract

Introduction

The primary objective of this study was to evaluate the effectiveness of an autonomous oncology boot camp on Advanced Pharmacy Practice Experience (APPE) student knowledge. Secondary objectives included assessing student perception of the virtual learning experience and overall comfort level with the material.

Methods

APPE students rotating through our institution between November 2019 and March 2020 were voluntarily enrolled in a 4-hour oncology-focused boot camp, which included five PlayPosit (Denver, CO, USA) interactive video lectures embedded with case-based application questions followed by one comprehensive web-based Quandary (Victoria, BC, Canada) action-maze case. Student learning was measured by a pre- and post-intervention exam. A web survey tool (Qualtrics, Provo, UT, USA) collected student perceptions evaluating their comfort with oncology-specific drug knowledge and APPE rotations tasks.

Results

Fifty students enrolled in the oncology boot camp, with 100% completing the pre- and post-intervention assessments. Overall, pre-intervention exam scores (mean: 55.4%, SD: 21.8%) improved by 23.2% following the boot camp (mean: 78.6%, SD: 19.2%; p < 0.001). Students performed better on all 10 exam questions, with 6 questions showing a statistically significant improvement (p < 0.05). Forty-five students (90%) completed the perception surveys. Of those, 93% agreed that it effectively reinforced important oncology knowledge, 91% supported the autonomous design, and 82% would recommend the oncology boot camp for future students.

Conclusion

The boot camp proved to be a beneficial educational tool that enhanced student knowledge and confidence in navigating common oncology concepts. Students valued the ability to independently complete the activities and supported its continuation.

Keywords

Boot camp oncology pharmacy education advanced pharmacy practice experience curriculum

Introduction

The pharmacists’ role in the management of cancer continues to evolve as providers are challenged with medication shortages, rapid deployment of newly-approved biosimilars and novel antineoplastics, development and implementation of precision medicine, and rising costs of immunotherapy, oral oncolytics, and cellular-based therapies.^1–3 Oncology pharmacists serve as integral members of the oncology interdisciplinary team and are responsible for providing comprehensive medication management, patient counseling, therapeutic plan development, formulary management, drug therapy evaluation, safe compounding and preparation of chemotherapy, and treatment-related toxicity monitoring.^4–8 Similar to many other subspecialties, student pharmacists are unlikely to receive adequate classroom education on these complex and ever-changing topics. Students entering an oncology Advanced Pharmacy Practice Experience (APPE) face a steep learning curve exacerbated, in our experience, by the deluge of newly approved anti-cancer drugs and the rapidly shifting onco-hematologic clinical landscape. For example, in 2020 alone, the U.S. Food and Drug Administration approved 53 new drugs, including 20 with oncology-related indications.⁹

Experiential learning in the medical education curricula builds upon the knowledge and skills gained during didactic teaching and remains a vital component of equipping future health care providers. In pharmacy education specifically, APPEs serve as the culmination of two to three years of intense didactic training where students demonstrate the fundamental capabilities required to be a competent pharmacist.¹⁰ Currently, there are no reported studies evaluating APPE-readiness in cancer-specific settings for pharmacy learners. In our experience, students frequently enter oncology-related APPEs with poor self-confidence stemming from minimal real-world experience with cancer-related drugs, diseases, and associated complications. As a result of this limited exposure, incoming APPE students may require a significant amount of time during the first week(s) of rotation to relearn basic oncology principles and commonly utilized antineoplastic agents.

The Ericsson theory for achieving high levels of performance involves active engagement in deliberate practice (DP).¹¹ These DPs include 1) tasks with well-defined goals, 2) motivation to improve, 3) feedback, and 4) ample opportunity for repetition and refinement. Many medical education programs across North America co-opted and incorporated these DP concepts into boot camp courses designed to help transition student learners into new clinical roles.¹² When looking at the differences in a boot camp versus a skills course, a boot camp is an intense, educational experience usually implemented before starting a new role, while a skills course involves improving competency during training.¹³ Medical-themed boot camps are increasingly utilized to reinforce information learned during didactic education as students transition to experiential learners. One meta-analysis evaluating the effectiveness of educational boot camps for medical students transitioning to post-graduate internships found significantly “large” improvements in clinical skills, confidence, and knowledge.¹²

Several studies have evaluated student oncology knowledge following various learning activities; however, an oncology APPE boot camp has not been studied.^14–16 Furthermore, a paucity of data currently exists in the literature surrounding the role and effectiveness of novel teaching strategies for oncology-related APPE activities.¹⁷ Moreover, student confidence with oncology knowledge as they enter APPE rotations in unknown. Resulting from the perceived need for additional oncology instruction and the increasing complexity associated with the provision of pharmaceutical care for the oncology patient, a boot camp was developed to further prepare students to successfully complete an oncology-based APPE rotation. This is, to our knowledge, the first study assessing the impact on pharmacy student knowledge and perceptions of an APPE boot camp in an oncology setting.

Aim

In an attempt to improve the oncology rotation at a large community hospital and outpatient infusion clinic, a 4-hour oncology boot camp was implemented and used as an academic leveling tool. The primary aim of this study was to evaluate the effectiveness of a newly-implemented, virtual, and self-paced oncology APPE boot camp to enhance student knowledge and confidence in navigating the dynamic and multifaceted subspecialty of hematology/oncology practice.

Methods

This study was approved by the Institutional Review Board of Union University. All participants were provided the pertinent details of the research activities. Investigators obtained informed consent from the participants of the boot camp study. Two student cohorts (class of 2020 and 2021) were included in this analysis.All Union University College of Pharmacy APPEs consist of one calendar month blocks in the third or fourth professional (P3 or P4) years. Prior to study initiation, a needs assessment was completed by the precepting oncology pharmacists to identify important oncology-related concepts required for APPE students to successfully navigate the first week(s) of rotation. Findings from this assessment guided the development of the oncology boot camp. Topics important to the oncology preceptors included: common dosing equations and calculations; basic pharmacology and counseling points for frequently used chemotherapy, immunotherapy, and oral oncolytic agents; typical monitoring parameters for patients receiving chemotherapy; and how to properly find and utilize pertinent drug information resources and practice guidelines.

One week prior to the rotation, investigators communicated with incoming students and provided specific instructions for the boot camp study. In addition, students used ExamSoft (Boca Raton, FL, USA) one week before starting the rotation to complete a pre-intervention assessment containing 10 knowledge/skills questions. Students were also sent an online perceptions survey (Qualtrics, Provo, UT, USA) to evaluate self-reported baseline comfort with oncology concepts. Investigators stored all material electronically on Canvas (Salt Lake City, UT, USA) for convenient student access. Canvas allowed investigators to view student activity and duration spent on each module. All APPE students rotating through our institution between November 2019 and March 2020 were eligible for enrollment in the oncology boot camp. The learning activities were designed to be engaging and self-paced, combining virtual PlayPosit (Denver, CO, USA) lectures with interactive questions. The boot camp included 5 enriched online video lectures and 1 web-based clinical case (Table 1).

Table 1.

Overview of the modules and associated concepts incorporated in the oncology boot camp curriculum.

Module	Topic
Introduction to Oncology30 minutes	Review basic principles and cancer terminology
	Outline TNM cancer staging and ECOG performance status
	Review pertinent laboratory requirements
	Utilize oncology-related resources (NCCN, ASCO, Trissel’s compatibility)
	Practice chemotherapy dose calculations
	Examine the role of the oncology pharmacist
Introduction to Chemotherapy40 minutes	Review the cell cycle as it relates to chemotherapy
	Describe the mechanism of action of common chemotherapeutic agents
	Identify common and noteworthy toxicities associated with chemotherapy
	Review common indications for chemotherapy
Review of Chemotherapy Toxicities35 minutes	Use Chemotherapy Man to identify organ systems affected by conventional chemotherapy
Review of Chemotherapy Toxicities35 minutes	With any given organ system, list potential chemotherapy agents that may elicit adverse events
Introduction to Immunotherapy30 minutes	Review the function of the immune system as it relates to tumorigenesis
	Identify FDA-approved checkpoint inhibitors and describe their mechanism of action
	Evaluate immune-related adverse events
	Recommend treatment of immune-related adverse events based on current NCCN guidelines
Introduction to TKIs30 minutes	Review the mechanism of action for commonly used TKIs
	Identify pertinent TKIs and their indications
	List common treatment-related adverse events associated with TKIs
	Outline important counseling points for common TKIs
Quandary Action-Maze Patient Case30–45 minutes	Evaluate a newly diagnosed lung cancer patient
	Using the NCCN guidelines, determine first-line treatment
	Calculate a BSA and appropriate doses for the preferred regimen
	Determine appropriate antiemetic therapy

ASCO: American Society of Clinical Oncology; BSA: body surface area; ECOG: Eastern Cooperative Oncology Group; FDA: Food and Drug Administration; NCCN: National Comprehensive Cancer Network; TKI: tyrosine kinase inhibitor.

Pre-recorded lectures were augmented usingan interactive and customizable video platform called PlayPosit and seamlessly embedded with case-based application questions and a user-friendly interface. Each video contained 5 to 8 intermittently dispersed quiz questions pertaining to the topic. Some questions were case-based involving application and analysis, others involved simple recall or comprehension. The video topics included: 1) Introduction to Oncology; 2) Introduction to Chemotherapy; 3) Review of Chemotherapy Adverse Effects; 4) Introduction to Immunotherapy; and 5) An Overview of Tyrosine Kinase Inhibitors. The APPE students were required to watch all of the posted videos and answer the embedded questions. After completing the video lectures, students completed one web-based action-maze designed by the research team using Quandary software (Victoria, BC, Canada). This multifaceted case emphasized important components found within each video module. The interactive action-maze scenarios allowedstudents to apply their knowledge by answering case-based questions using cancer-related guidelines and drug information sources. Students were able to work through the case from their computer and receive immediate feedback. Students were allotted sufficient time to watch the videos and complete the case.

Upon completion of the boot camp, students retook the initial exam and perceptions survey evaluating the effectiveness of the learning experience. The primary outcome was to evaluate improvement in overall student knowledge. Secondary outcomes included assessing student perception of the innovative educational tool and overall comfort level with the material. Student learning was evaluated by assessment performance. All 10 ExamSoft assessment questions were multiple choice or free response (e.g., dose calculations) and written by the precepting oncology pharmacists. ExamSoft was chosen because students were familiar and comfortable with this password-protected testing platform and it provided extra security by limiting student access to other resources while taking the exam. Student perceptions of the teaching and learning methods were evaluated by an anonymous 14-question web survey tool. Students were given one day to complete the exam and survey following the boot camp. A 5-point Likert scale survey collected student perceptions evaluating their comfort with oncology-specific drug knowledge and APPE rotations tasks, with 1 being “extremely comfortable” and 5 being “extremely uncomfortable.”

Survey elements

Pre- and post-boot camp surveys were provided to all participants. The questionnaire asked participants to indicate the extent of their agreement with statements in relation to comfort level with the following activities: 1) recalling common chemotherapeutics, their drug classes and mechanisms of action; 2) recalling specific toxicities associated with common chemotherapeutics; 3) recalling common tyrosine kinase inhibitors and their toxicities; 4) recalling common immune checkpoint inhibitors and their toxicities; 5) performing basic oncology-related calculations (e.g., body surface area, absolute neutrophil count, and the Calvert formula); and 6) utilizing oncology-related guidelines and tools (e.g., NCCN guidelines, MASCC scoring, etc.). Student feedback on the training methods and workshop design were also obtained. In addition, two open-ended questions were included to elicit further insight about the boot camp and the student’s experience.

Statistical analysis

Statistical analysis was performed using SPSS (version 25). Pre-and post-assessment scores were analyzed by comparing the mean test scores via paired t test. Individual exam questions were analyzed using either Fisher’s exact test or Chi-squared analysis, where appropriate. All p-values were two sided with statistical significance determined to be p < 0.05.

Results

Fifty students (100%) completed the pre- and post-intervention assessments. Overall, Table 2 shows pre-intervention exam scores (mean: 55.4%, SD: 21.8%) improved by 23.2% following the boot camp (mean: 78.6%, SD: 19.2%; p < 0.001). Students performed better on all 10 exam questions, with 6 questions showing a statistically significant improvement (p < 0.05). Pre- and post-intervention perceptions survey questions are listed in Tables 3 and 4. Forty-five students (90%) completed both the pre- and post-boot camp surveys. Of those who completed the survey, improvement was noted in all 6 preceptor-identified content areas (Table 3). In addition, 93% agreed that it effectively reinforced important oncology knowledge, 91% supported the autonomous design, and 82% would recommend the oncology boot camp for future APPE oncology students (Table 4). In addition, only 1 student disagreed with incorporating the boot camp for future APPE students.Although most of the student feedback was positive, highlighting the relevance, applicability, and value of the boot camp, we were interested in and specifically sought suggestions for improvement. Very few substantive comments were made regarding ways to improve the boot camp; however, 8 students (17.8%) identified the 4-hour boot camp as being too long and a potential area for improvement.

Table 2.

Comparison of student test performance on knowledge-based assessment questions before and after an oncology APPE boot camp.

ExamSoft assessment questions	Average pre-intervention score (N = 50 students)	Average post-intervention score (N = 50 students)	P-value*
1. Body surface area calculation	86% (43)	94% (47)	0.318
2. Paclitaxel dose calculation	84% (42)	92% (46)	0.357
3. Carboplatin dose calculation	26% (13)	72% (36)	<0.001
4. Role of the pharmacist	90% (45)	98% (49)	0.204
5. Carboplatin and paclitaxel adverse effects	34% (17)	56% (28)	0.027
6. Doxorubicin toxicities	48% (24)	68% (34)	0.043
7. Methotrexate supportive care and toxicities	68% (34)	78% (39)	0.260
8. Diarrhea-inducing agents	38% (19)	84% (42)	<0.001
9. Identifying checkpoint inhibitors	42% (21)	74% (37)	0.001
10. Tyrosine kinase inhibitor complications	38% (19)	70% (35)	0.001
Overall score	55%	79%	<0.001

Statistical significance at p < 0.05.

Table 3.

A perceptions survey administered before and after completing the oncology boot camp: Students who were “somewhat comfortable” or “extremely comfortable”.

Qualtrics survey questions	Pre-boot camp (median Likert score)^a	Post-boot camp (median Likert score)^a
1. Recalling common chemotherapeutics, their drug classes, and mechanisms of action	40.8% (3.24)	64.4% (2.44)
2. Recalling specific toxicity’s associated with common chemotherapeutics	43.8% (3.00)	68.9% (2.31)
3. Recalling common tyrosine kinase inhibitors and their toxicities	22.5% (3.55)	51.1% (2.60)
4. Recalling common immune checkpoint inhibitors and their toxicities	33.3% (3.31)	57.8% (2.51)
5. Performing basic oncology-related calculations (e.g., body surface area, absolute neutrophil count, and the Calvert formula)	53.1% (2.65)	82.2% (1.93)
6. Utilizing oncology-related guidelines and tools (e.g., NCCN guidelines and MASCC scoring)	46.9% (2.85)	77.3% (1.91)

Scale indicates 1 = extremely comfortable; 2 = somewhat comfortable; 3 = neutral; 4 = somewhat uncomfortable; 5 = extremely uncomfortable.

MASCC: multinational association of supportive care in cancer; NCCN: National Comprehensive Cancer Network.

Table 4.

Student responses to a satisfaction survey following the oncology boot camp.

Qualtrics survey questions	Strongly agree	Somewhat agree	Neutral	Somewhat disagree	Strongly disagree
1. The course was easy to navigate.	44.44 %	46.67%	8.89%	0.00%	0.00%
2. The interactive videos were an effective tool for learning.	44.44%	40.00%	11.11%	4.44%	0.00%
3. The interactive case was an effective tool for learning.	37.78%	46.67%	13.33%	2.22%	0.00%
4. The oncology boot camp effectively reinforced the information I learned in pharmacotherapy.	57.78%	35.56%	6.67%	0.00%	0.00%
5. I would recommend this course for future APPE oncology students.	57.78%	24.44%	15.56%	2.22%	0.00%

APPE: advanced pharmacy practice experience.

Discussion

The “boot camp” has become increasingly popular as a training experience and educational tool in recent years.¹² This may correlate with a growing understanding of current training limitations and new, innovative approaches to foster active learning and improve baseline knowledge. Active learning is designed to augment student retention of concepts by creating an atmosphere where higher-order Bloom’s Taxonomy levels are used, such as application, analysis, synthesis, and evaluation.^18–20 The primary purpose of this boot camp was to design an effective and autonomous educational tool that enhanced student knowledge and confidence with common oncology concepts. We opted for a fully autonomous and virtual boot camp that could be completed independently according to the student’s pace and schedule. We incorporated novel technology, including an interactive video platform (PlayPosit) and action-maze (Quandary), to help fulfill this vision. The overall assessment of the boot camp was favorable, both from the learner and preceptor perspectives. It created an educational time that was evidence-based, virtual, flexible, interactive, and capitalized on the opportunity to reach students immediately as they began their clinical rotation experience. The design of the boot camp involved contribution from oncology preceptors, faculty, and student learners. Many precepting pharmacists have demanding clinical schedules, so an intensive, autonomously designed educational boot camp that facilitates student learning while freeing up the preceptor for other responsibilities has relevance and is worth consideration. Most importantly, the post-assessment quiz and perceptions survey allowed preceptors to gauge, identify, and address any lingering deficiencies early in the APPE rotation, providing a more focused teaching experience for that specific student.

The mean pre-intervention assessment score of approximately 55% was not surprising to the investigators. The questions involved topics that were common in oncology practice, but not otherwise. For instance, using the Calvert Formula to calculate a carboplatin dose is a necessary skill for a student pharmacist on an oncology-related APPE rotation; however, this is not an equation and process most students are likely to have memorized beforehand. This was born out in the data, with only 1 in 4 students accurately recalling and applying the Calvert Formula before completing the boot camp and nearly 3 in 4 successfully employing the equation following boot camp activities. In addition, specific antineoplastic toxicity knowledge is likely to be lacking in students who have yet to complete an oncology-related APPE rotation. The overall improvement of approximately 23% was statistically significant and noteworthy. An improvement in all 10 questions demonstrated that the boot camp effectively improved student knowledge in key areas that were previously deficient. A perception survey was designed and disseminated to better understand the impact of the oncology boot camp among student participants. Student scores on all survey questions evaluating comfort with oncology-related concepts improved following completion of the boot camp. Furthermore, student survey responses confirmed that the session was 1) easy to navigate, 2) an effective tool for learning or reinforcing important concepts, and 3) a worthwhile activity that should be continued for future APPE students. Although not formally assessed, oncology APPE preceptors at our institution and faculty members were pleased with the boot camp and have since fully implemented it as an essential component of all future oncology APPE rotations.

This single-center pilot study has several limitations. For instance, the study was relatively small with only 50 participants and thus may not be reflective of all APPE student pharmacists. Since all students were from one institution, baseline knowledge was similar and may alter results and applicability of specific assessment questions. There was also no control group, so researchers could not determine if knowledge and confidence improvements would be similar with traditional learning activities. Another notable limitation is that investigators did not assess the change in student performance over the course of the rotation. Furthermore, the study evaluated only immediate post-boot camp knowledge and not overall rotation performance. A follow‐up study assessing the long-term retention of oncology-related information and confidence would be helpful to evaluate the longitudinal effect of our intervention.

Designing, creating, and implementing the oncology boot camp experience was both time and resource intensive, necessitating significant contributions from the team of precepting pharmacists. For instance, the design team worked on the initial development and implementation of the boot camp for over 4 months before enrolling the first cohort of students. Concise and timely communication with incoming students and preceptors was essential for a smooth and worthwhile experience. Overall, establishing the oncology APPE boot camp required a significant allotment of time and expertise from invested members of the research team. Once finalized, however, the autonomous boot camp required minimal maintenance or oversight from precepting pharmacists for the remainder of the study period. Moreover, as the boot camp focuses on core learning objectives that are relatively static and concrete, updating the virtual platforms, presentations, and cases following initial implementation required little time and effort.

Conclusion

This pilot study indicated that the oncology boot camp can be an effective and innovative educational tool to enhance student knowledge and confidence in navigating important onco-hematologic concepts on APPE rotations. Students valued the ability to independently complete the activities and supported its continuation. This study demonstrates the feasibility of a bridging course to help students transition and prepare for the demands of a specialized APPE rotation.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iDs

Bethannee Horn

Lyn Wells

Zachery Halford

References

Meier

Astier

Boşnak

, et al. The future of oncology pharmacy: European conference of oncology pharmacy 2018. Eur J Oncol Pharm 2019; 2: e20.

Ignoffo

Knapp

Seung

, et al. Trends in the delivery of care to oncology patients in the United States: emphasis on the role pharmacists on the healthcare team. J Oncol Pharm Pract 2021; 27: 5–13.

Vulaj

Hough

Bedard

, et al. Oncology pharmacist opportunities: closing the gap in quality care. J Oncol Pract 2018; 14: e403–e411.

Holle

Harris

Chan

, et al. Pharmacists' roles in oncology pharmacy services: results of a global survey. J Oncol Pharm Pract 2017; 23: 185–194.

Hematology/Oncology Pharmacy Association. Scope of hematology/oncology pharmacy practice, www.hoparx.org/images/hopa/resource-library/professional-tools/HOPA13_ScopeofPracticeBk.pdf. (accessed 20 April 2021).

Holle

Boehnke Michaud

Oncology pharmacists in health care delivery: vital members of the cancer care team. J Oncol Pract 2014; 10: e142-5–e145.

Segal

Bates

Fleszar

, et al. Demonstrating the value of the oncology pharmacist within the healthcare team. J Oncol Pharm Pract 2019; 25: 1945–1967.

International Society of Oncology Pharmacy Practitioners. Position statement: role of the oncology pharmacy team in cancer care, https://www.isopp.org/sites/default/files/news/files/ISOPP%20Position%20Statement%20%20March%201%2C%202021.pdf. (accessed 24 April 2021).

Food a. Drug Administration. New Drug Therapy Approvals 2020. Advancing Health Through Innovation, https://www.fda.gov/drugs/new-drugs-fda-cders-new-molecular-entities-and-new-therapeutic-biological-products/new-drug-therapy-approvals-2020 (accessed 25 March 2021).

10.

Medina

Plaza

Stowe

, et al. Center for the advancement of pharmacy education 2013 educational outcomes. Am J Pharm Educ 2013; 77: 162.

11.

Ericsson

KA.

Deliberate practice and acquisition of expert performance: a general overview. Acad Emerg Med 2008; 15: 988–994.

12.

Blackmore

Austin

Lopushinsky

, et al. Effects of postgraduate medical education “boot camps” on clinical skills, knowledge, and confidence: a meta-analysis. J Grad Med Educ 2014; 6: 643–652.

13.

Berridge

Jain

Biyani

CS.

Defining boot camp: a supporting literature review. SE Asian Jnl Med Educ 2020; 13: 3–13.

14.

Bossaer

Panus

Stewart

, et al. Student performance in a pharmacotherapy oncology module before and after flipping the classroom. Am J Pharm Educ 2016; 80: 31.

15.

Vyas

Bhutada

Feng

Patient simulation to demonstrate students’ competency in core domain abilities prior to beginning advanced pharmacy practice experiences. AJPE 2012; 76: 176.

16.

Serag-Bolos

Chudow

Perkins

, et al. Enhancing student knowledge through a comprehensive oncology simulation. Am J Pharm Educ 2018; 82: 6245.

17.

Ledbetter

Lau

Enterline

, et al. A simulation activity to assess student pharmacists’ knowledge and perceptions of oncology pharmacy. Am J Pharm Educ 2020; 84: 7474.

18.

Bloom

Englehart

Furst

EJ.

Taxonomy of education objectives. Handbook 1: the cognitive domain. New York, NY: Longman, 1956.

19.

Prince

Does active learning work? A review of the research. J Eng Educ 2004; 93: 223–231.

20.

Lucas

Testman

Hoyland

, et al. Correlation between active-learning coursework and student retention of core content during advanced pharmacy practice experiences. Am J Pharm Educ 2013; 77: 171.