Abstract
Objective
The National Comprehensive Cancer Network guidelines recommend an interval between surgery and adjuvant radiation therapy of less than 6 weeks, but only 44% of patients meet this metric nationally. We sought to identify key components of an improvement process focused on starting adjuvant radiation therapy within 6 weeks of surgery.
Methods
This project used an A3 model to improve a defined process measure. We studied a consecutive sample of 56 patients with oral cavity carcinoma who were treated at our institution with upfront surgical resection followed by adjuvant radiation therapy. Twelve proposed interventions tested during the study period focused on 3 key drivers of delays: delayed dental evaluation and teeth extraction, delayed radiation oncology consults, and inadequate patient engagement. The primary outcome measure was the number of days from surgery to the start of radiation therapy.
Results
Prior to the intervention, 62% of patients received adjuvant radiation within 6 weeks of surgery. Following the intervention, 73% of patients achieved this metric. The percentage of patients with avoidable delays decreased from 24% to 9%. The percentage of patients with unavoidable delays was relatively constant before and after the intervention (15% and 18%, respectively).
Discussion
Defining disease-specific metrics is critical to improving care in our head and neck cancer patient population. We demonstrate several key components to develop and improve self-defined metrics.
Implications for Practice
As we transition to a system of value-based care, structured quality improvement projects can have a measurable impact on cancer patient process measures.
Keywords
The National Comprehensive Cancer Network (NCCN) guidelines recommend a preferred time interval between surgery and the start of adjuvant radiation therapy of 6 weeks or less. 1 Multiple studies have suggested that exceeding this time interval can have an impact on overall survival, particularly in patients with more high-risk factors.2-6 Despite the guidelines and the mounting evidence demonstrating a survival benefit, nationally, only 44% of patients with head and neck cancer are treated within this time interval, with lower rates of 38% at academic medical centers. 4 No study has adequately assessed the ideal quality benchmark given the clinical variability in patient presentations that can lead to delays.
Numerous barriers exist to efficiently bring patients from surgery to initiation of adjuvant radiation therapy in a timely manner: multiple specialty teams to coordinate, patient education and social barriers, wound-healing issues, and insurance complications. We used a human-centered design process to explore the current process and devise best practices that can help optimize timely delivery of adjuvant radiation therapy.
The purpose of our quality improvement project was to develop a structured multidisciplinary approach to decrease treatment delays for patients with head and neck cancer requiring adjuvant radiation therapy, implement interventions in our head and neck cancer program, and assess the project’s impact as measured by the percentage of patients who received adjuvant radiation therapy within 6 weeks of surgery. We hypothesized that poor coordination of care was a primary driver in delays for patients receiving adjuvant radiation therapy. We report on the process that we used to improve our performance and propose a potential national quality benchmark that accounts for uncontrollable patient-level variability.
Methods
This study was reviewed by the Stanford University institutional review board and determined to be an internal quality improvement project that does not meet the definition of human subject research as defined in federal regulations 45 CFR 46.102. To avoid risk of patient identification, exact dates of patient treatment are not used in this publication. Treatment dates described are in reference to an unspecified baseline date. Our study complied with the SQUIRE 2.0 reporting guidelines. 7
Setting
Stanford Health Care is an academic 477-bed tertiary care hospital located in Stanford, California. The Head and Neck Oncology Cancer Care Program (HN CCP) is a multidisciplinary group consisting of 7 surgeons, 3 radiation oncologists, 2 medical oncologists, 1 dental oncologist, 5 advanced practice providers (physician assistants [PAs] and nurse practitioners [NPs]), and 6 nurse coordinators. Our quality improvement team was enrolled in the Stanford Clinical Effectiveness Leadership Training program and consisted of a head and neck surgeon, a radiation oncologist, an otolaryngology resident, the lead physician assistant, the director of cancer quality, a senior quality consultant, and a clinical administrative associate.
Intervention
The project used an A3 framework for process improvement, a form of Plan-Do-Study-Act (PDSA) methodology 8 (Supplemental Figure S1, available in the online version of the article). The analysis of our performance gap was performed by doing detailed chart review of all patients with oral cavity cancer who were delayed during a 20-month time interval. The primary cause for each delay was recorded and was later used to complete the fishbone diagram. In addition, the quality improvement team interviewed 10 different members of the care team to determine additional potential delays. These team members are responsible for different aspects of coordinating adjuvant therapy and included radiation oncology and surgical advanced practice providers, nurse coordinators, schedulers, an infusion center coordinator, and our clinic manager. These interviews were qualitative and were designed to elicit any workarounds used by team members to prevent potential delays that could be avoided by an improved process. The actual and potential causes for patient delays are outlined in a fishbone diagram, which groups the causes for delays into 6 primary areas ( Figure 1 ). Analysis of the fishbone diagram revealed that 3 key drivers were responsible for most patient delays. The first 2 key drivers were selected based on the most frequent cause of delays seen on chart review, and the last was based on qualitative interviews with team members. These included (1) delayed dental extractions, (2) delayed referral to radiation oncology, and (3) poor patient and team engagement.

Fishbone diagram demonstrating the actual and recalled causes for delays between surgery and the start of adjuvant radiation. FNA, fine needle aspiration; ITA, infusion center; PET, positron emission tomography; XRT, radiation therapy.
We developed 12 interventions that addressed our 3 key drivers ( Table 1 ). These interventions were developed by brainstorming with our multidisciplinary team while focusing on specific causes of patient delays seen in the chart review. We used an iterative process through our monthly meetings to identify and refine interventions that were effective at improving our performance. Each month, all of the interventions were sequentially reviewed by the team members and a decision was made to continue, revise, or abandon each intervention. Changes were communicated to the care teams the following week in the clinic and nurse huddles, as well as through email.
Key Drivers of Delays with Initial Proposed Interventions. a
The interventions were maintained, modified, or abandoned over time.
Dental extractions
To improve the timeliness of dental evaluations, we eliminated referring patients back to their community dentist. All patients with oral cavity cancer received a Panorex exam at their initial new patient visit, which was reviewed by our dental oncologist. Based on the Panorex, the dentist determined if the patient needed a formal consultation, if surgical extractions would be needed, or if the patient was cleared for radiation therapy. This allowed for effective scheduling of intraoperative extractions concurrently with the primary tumor resection.
Radiation oncology referrals
The timing of radiation oncology consults was shifted to earlier in each patient’s treatment pathway. The referral was placed as soon as the patient met any guideline recommendation for needing adjuvant treatment, which led to many consults being placed at the initial new patient visit prior to surgery. If there was any question of the need for adjuvant therapy, the patient was discussed at a multidisciplinary tumor board prior to surgery and a consult placed immediately if the radiation oncologist felt it was necessary.
For some patients, it was noted that the pathology report from surgery was the deciding factor on the need for radiation therapy. Pathology reports were not being sent to the care team while the patient was admitted to the hospital, causing them to not be acted on until the first postoperative visit. For patients with prolonged hospital stays, this could be 3 or more weeks after surgery. Therefore, we implemented a system by which the ancillary care providers use a 10-day postoperative reminder to trigger a pathology review and act on pathology results earlier.
Patient and team engagement
To increase patient engagement, the team designed a Clinical Visit Summary form ( Figure 2 ), which was completed during the patient visit. The form was a care pathway checklist that reminded the care team of the necessary components for ensuring adequate start times. Importantly, the form encouraged a discussion with the patient about the location of adjuvant radiation therapy, which was particularly important if a patient lived at a distance, so transportation and housing could be arranged.

Most recent version of the Clinic Visit Summary form used by the nurse coordinators to help with managing the care plan.
Study of the Intervention
Group meetings with involved team members were held monthly to evaluate the utility and effectiveness of the current intervention strategy, to improve retention rates, and to review the prior month’s data. This committee consisted of providers from each specialty, including physicians, nurse coordinators, physician assistants, dentists, and schedulers. At these meetings, we performed individual analysis of patients who had missed the benchmark to identify strategies to improve the intervention. In addition, each of the interventions related to the clinic nursing staff was reviewed in a weekly nurse huddle. Both meetings were an opportunity to provide feedback to the quality improvement team to determine which interventions were being effectively implemented.
For each patient not meeting the 42-day metric, a detailed analysis of the chart and processes of care was performed. In all instances, the root cause of delay was relayed back to the treating team and fed back to the quality improvement team to allow them to take these into account. The progress of the quality improvement project was reported to the entire care team at the tumor board conference once per month for rapid communication of areas for improvement.
Measures and analysis
The primary outcome measure was the number of days from surgery to the start of radiation therapy. All patients were classified as having met the metric if they started radiation in 42 days or less from the surgery date. The secondary outcome was the proportion of patients meeting the 42-day metric. Data were recorded in a run chart that plotted each patient against the number of days between surgery and adjuvant treatment ( Figure 3 ). The performance over a 20-month period prior to the intervention was compared to a period of 8 months after the intervention.

Run chart demonstrating patient outcomes before and after the intervention. Each dot represents a single patient and the number of days between surgery and radiation therapy start. Square dots in red are patients who were delayed for avoidable causes. Triangular dots in green are patients who were delayed with unavoidable causes. Circular dots in blue represent patients without delays.
Patient delays were recorded and categorized into 6 different causes: delayed dental evaluation and treatment, delayed radiation oncology consult, poor care coordination, delayed wound healing, new finding on exam or treatment planning positron emission tomography (PET) requiring biopsy, and patient indecision. Poor care coordination was defined as errors in communication between members of the care team. The first 3 of these causes were deemed to be avoidable. The latter 3 were considered unavoidable by the care team. One patient who was delayed due to rapid recurrence of her tumor prior to the start of adjuvant treatment was excluded from the analysis.
Results
Baseline
We identified 56 patients who underwent surgery and adjuvant radiation therapy at our institution for oral cavity carcinoma between February 2014 and July 2016. Table 2 shows the demographics of the patients treated during this interval. Of the patients treated prior to the intervention, 62% received adjuvant radiation therapy within 6 weeks.
Demographics of Patients Treated before and after the Intervention.
Feedback and Modification of Intervention
We developed 12 proposed interventions to address the key drivers. Over the course of the project, 7 were useful for achieving the goals of the projects. Multiple points of feedback allowed us to modify or abandon our interventions over time. Table 1 details the final state of each proposed intervention and includes observations on implementation.
Our initial attempts at having our new patient coordinators screen each patient’s dental history were inefficient since they were not always completed. This function got assigned to the nurse coordinator who was already performing an audit of what records were available. To minimize unnecessary new patient visits, the dental oncologist agreed to review each Panorex to determine whether a formal visit needed to be scheduled. If extractions were necessary, the surgical scheduler was responsible for informing the dental team of the surgical date.
The Clinic Visit Summary form was modified multiple times during this process to reflect the input of various teams. With each iteration, the form grew in scope until it became a more comprehensive care pathway checklist. Once the teams understood the process better, the use of the forms was abandoned by the care team for 2 surgeons as they incorporated the changes into their workflow without needing the form. In addition, it was felt to be burdensome for teams as it was no longer focused solely on timely adjuvant radiation therapy.
Quantitative Results
During the project, the primary outcome from each patient’s treatment was measured and recorded on the run chart ( Figure 3 ). Each point on the run chart is a single patient, measured by the number of days between surgery and the start of adjuvant radiation therapy. Following the intervention, the percentage of patients treated within 6 weeks increased from 62% to 73%. Table 3 shows the number of patients by type of delay. The percentage of avoidable delays decreased from 24% to 9% following the intervention. The percentage of unavoidable delays remained relatively stable from 15% to 18%.
Patient Outcomes before and after Intervention. a
Causes of delays were determined based on chart review and then grouped based on the provider’s ability to control for the delay.
Discussion
Our project demonstrated the utility of a structured quality improvement project in decreasing the time from surgery to the start of adjuvant therapy. In our study, we increased the percentage of patients receiving adjuvant XRT within 6 weeks from 62% to 73%. Our project highlights the importance of early referrals to radiation oncology and timely involvement of our colleagues in dentistry. Based on our study of unavoidable delays, our data suggest that the optimal rate of compliance with this metric at similar institutions may be closer to 80%, which is a helpful guide when determining national benchmarks for performance improvement.
Following the intervention, 2 patients were delayed for avoidable reasons. The first patient started radiation at 43 days due to a delayed postoperative visit, at which time the first consult to radiation oncology was made. The second patient started radiation at 64 days. She was initially sent for radiation therapy at an outside facility; however, since she was pacemaker dependent, the facility was not comfortable with treating her and she needed to be sent back to our institution for treatment.
The particular strength of this project was the inclusion of a multidisciplinary team across different departments and service lines. We had strong physician leadership and support for this project based on the evidence that this quality metric had a direct impact on patient outcomes. By developing a rapid cycle of feedback to provider teams, we could immediately notify provider teams about issues and make modifications to our interventions.
The primary interventions in our study were to ensure presurgical dental evaluation, have timely placement of radiation oncology consults, and increase earlier patient engagement. Each of these interventions contributed to the improved outcomes. Presurgical dental evaluations allowed us to ensure that dental extractions occurred at the time of surgery. If the decision for extractions is not made until after surgery, patients usually require at least 1 to 2 weeks to have a dental evaluation and extractions performed and an additional 2 to 3 weeks of healing before radiation can start. Early initiation of radiation oncology consults alerted the multidisciplinary team of pending patients and allowed the radiation oncology team to track patients through surgery and proactively schedule follow-up visits and their simulation scan. Finally, early patient engagement allowed patients to make earlier decisions on where they wanted to receive radiation therapy and gave them a target start date that they could plan for as well.
Unavoidable delays accounted for 15% to 18% of the delays in this study. We classified these delays into 3 categories: delayed wound healing, new finding requiring biopsy, and patient indecision. Delayed wound healing occurred in 2 patients who developed an infection (n = 1) and fistula (n = 1) following surgery. Large reconstructions of the head and neck are reported to have a fistula rate of 3% to 20%, although occurring more frequently in patients who have received prior radiotherapy.9,10 This can pose a significant source of delays, particularly since there is no standardized method for determining when a wound is adequately healed for radiation therapy. At our institution, treatment-planning PET scans are frequently used to help target radiation therapy. These scans are performed after surgery and have the potential to identify lesions that require biopsy prior to continuing with adjuvant therapy. The biopsies often require image guidance (ultrasound or computed tomography), which can add further delays due to scheduling. The final category of delays was due to the additional time patients needed to make decisions on if they wanted adjuvant treatment and which treatment center. This was particularly true for patients with early stage tumors and negative margins but adverse pathological features such as perineural invasion, as well as patients traveling a distance to our center who were considering a local treatment option.
We were unable to find any articles in the literature focused on quality improvement in timely delivery of radiation therapy, which underscores the importance of this work. We provide specific interventions and recommendations on implementation. In addition, by analyzing type of patient delays, we could propose a national benchmark for programs that accounts for unavoidable delays due to patient-level factors.
The project required increased utilization of resources in different aspects of care. The increased number of Panorex exams required coordination with radiology to ensure timely and consistent access to the Panorex machine. Review of the Panorex by our dental oncology team required a specific message to be sent requesting review of the images and then the additional time required for review by the dentist. Lowering the threshold for consultation of radiation oncology increased the number of new patients seen by them, and not all new consults ended up receiving treatment. Additional clinic time was required for discussion with patients about the implications of delaying radiation therapy and the steps necessary to ensure that it occurred on time.
One of the limitations to our study is that our interventions were designed to work within our current environment and resource set. Access to a Panorex machine and inclusion of a dedicated dental oncologist on our team were both critical to our success. Many institutions that do not have partnering dental colleagues would have difficulty obtaining the easy access to care that we were able to offer our patients. Another aspect of our environment was the strong support of our radiation oncologists, who were willing to see patients prior to surgery and help track patients requiring adjuvant treatment. Finally, we had access to resources for data collection, analysis, and reporting. Following the initial implementation of this project, we began a larger quality improvement initiative, which requires a 25% full-time equivalent (FTE) for data management. While this larger initiative reports on 5 different metrics in addition to our radiation delays, this is a substantial investment for an institution.
Sustainability of the work is one of the most difficult aspects of quality improvement projects. To help ensure a continued focus on this project, we used 2 overlapping mechanisms (quality committee and tumor board) that would frequently review the data and allow us to make changes to the interventions. Our monthly quality improvement committee included members from each job function in the program to help understand problems and disseminate solutions. Presentation of metrics at our tumor board allowed for the physicians involved to remain aware of the metric and our performance. Institutional awareness of this project has likely contributed to our positive results, and we anticipate ongoing review of our outcomes will promote team engagement. We also faced the challenge of trying to standardize our process across different care teams. A few teams understood the key drivers and wanted to implement minor changes to their current processes, as opposed to adopting a new standardized system.
There are several key lessons that we learned during this project. Data collection is the foundation of quality improvement projects yet can be difficult, particularly without dedicated resources. In our system, identification of patients required manual review of all new patient consults since we currently lack a centralized database of all new patients with cancer. Patients need to be tracked during their course of treatment. Communication across different provider teams can be difficult, and there were multiple instances where the proposed changes were not fully understood by all the teams. Frequent coaching, reminders, and check-ins were required when implementing change. Finally, patient presentations are very diverse. Many factors such as insurance coverage of procedures, distance from our center, lack of understanding of needed care, and poor social support all contributed to the hurdles that we needed to overcome.
Implications for Practice
A structured quality improvement project can effectively address important quality metrics in head and neck cancer care. As we transition to a system of value-based care, these types of projects will become increasingly important for cancer programs to implement and can have a measurable impact on patient process measures. A shorter time interval between surgery and radiation therapy appears to show a clinical benefit based on retrospective single institution and national database studies.3,6 Future research should prospectively assess how a quality improvement project that decreases this time interval affects patient outcome measures such as recurrence rates and overall survival.
Author Contributions
Disclosures
Supplemental Material
OTO768254_suppl_mat – Supplemental material for Reducing the Time from Surgery to Adjuvant Radiation Therapy: An Institutional Quality Improvement Project
Supplemental material, OTO768254_suppl_mat for Reducing the Time from Surgery to Adjuvant Radiation Therapy: An Institutional Quality Improvement Project by Vasu Divi, Michelle M. Chen, Wendy Hara, Deepa Shah, Kristina Narvasa, Andrea Segura Smith, Jennifer Kelley, Eben L. Rosenthal and Julie Porter in Otolaryngology–Head and Neck Surgery
Footnotes
Acknowledgements
The authors would like to acknowledge Louise Davies, MD, for her valuable contributions in composing the manuscript.
No sponsorships or competing interests have been disclosed for this article.
Supplemental Material
Additional supporting information is available in the online version of the article.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
