Abstract
Objective
To determine if the introduction of a systematic preoperative sinus computed tomography (CT) checklist improves identification of critical anatomic variations in sinus anatomy among patients undergoing endoscopic sinus surgery.
Study Design
Single-blinded prospective cohort study.
Setting
Tertiary care hospital.
Subjects and Methods
Otolaryngology residents were asked to identify critical surgical sinus anatomy on preoperative CT scans before and after introduction of a systematic approach to reviewing sinus CT scans. The percentage of correctly identified structures was documented and compared with a 2-sample t test.
Results
A total of 57 scans were reviewed: 28 preimplementation and 29 postimplementation. Implementation of the sinus CT checklist improved identification of critical sinus anatomy from 24% to 84% correct (P < .001). All residents, junior and senior, demonstrated significant improvement in identification of sinus anatomic variants, including those not directly included in the systematic review implemented.
Conclusion
The implementation of a preoperative endoscopic sinus surgery radiographic checklist improves identification of critical anatomic sinus variations in a training population.
Endoscopic sinus surgery (ESS) is one of the most common procedures performed by otolaryngologists, with significant ramifications associated with intraoperative complications. Furthermore, due to the close proximity to vital structures, there is a 1% risk of major complication. 1 Patients undergoing ESS receive preoperative computed tomography (CT) imaging to identify critical anatomic relationships to reduce complications. Knowledgeable interpretation of radiologic imaging has become paramount in performing ESS. Studying and analyzing preoperative imaging arms the surgeon with the anatomic knowledge to successfully navigate the nasal cavity and paranasal sinuses, reducing the risk of potential complications. However, minor and major complications can and still occur due to anatomic variants, altered anatomy due to previous surgery, severe polyposis, and decreased visibility due to bleeding. The surgeons’ familiarity with each patient’s unique anatomy and pathology, as gleaned from preoperative imaging, is an important part of reducing the risk associated with ESS.
Checklists are well known to reduce complications and improve outcomes in surgical care.2,3 Soler et al developed a preoperative checklist for ESS to decrease the frequency of avoidable surgical errors.4,5 A portion of this checklist recommends identifying and discussing critical anatomic variants prior to the surgery. After instituting this checklist at the University of Utah, we observed that surgeons in training had a difficult time accurately and systematically identifying important anatomic variations, which suggested that another subcategory within Soler’s checklist was needed for reviewing the CT scan to identify the major anatomic variations.
While exhaustive checklists have been described by radiology colleagues, 6 a more streamlined preoperative checklist of sinus imaging was introduced by Weitzel et al. 7 Based on the mnemonic “CLOSE” ( Table 1 ), this checklist systematically reviews sinus CT scans for evaluation of common rhinologic anatomic concerns during ESS. Yao et al 8 demonstrated that residents felt that the use of a sinus CT checklist was useful, but the article does not demonstrate objective improvement in identification of sinus anatomic variants with the checklist. Our objective was to evaluate whether the implementation of the CLOSE criteria checklist would improve the identification and recall of the critical sinus anatomy during the comprehensive ESS preoperative checklist.
CLOSE Preoperative ESS Radiographic Checklist. a
Abbreviations: CT, computed tomography; ESS, endoscopic sinus surgery.
Republished with permission from Weitzel EK, Floreani S, Wormald PJ. Otolaryngologic heuristics: a rhinologic perspective. ANZ J Surg. 2008; 78(12):1096-1099.
Methods
Approval from the University of Utah Institutional Review Board was sought and obtained prior to initiating the study. The inclusion criteria were ESS wherein residents reviewed and commented on critical sinus CT landmarks preoperatively as part of an ESS checklist. Any cases for which a resident did not participate or review CT scans preoperatively were excluded. Data were gathered with no resident-specific identifiers, as only year of training was included to perform a subgroup analysis.
A deception consent was used to keep the residents blinded to participation while data were being collected prior to and after instituting the CLOSE mnemonic. At the beginning of the study, residents consented to participate in a study that (they thought) involved a questionnaire regarding educational conferences. Following completion of data gathering and statistical analysis, the participating residents were debriefed with the true nature of the study, and all consented to continued participation.
The preoperative sinus surgery checklist developed by Soler was performed prior to ESS. 4 The second item on this checklist consists of reviewing pertinent CT scan findings: Did the surgical team verbally review among themselves important anatomic variations at start of case? Residents, who were blinded to their participation in the study, were interviewed by the supervising attending surgeon during the preoperative timeout to determine their knowledge base of the patient-specific imaging prior to starting the procedure. During this interview, the supervising surgeon documented if the resident appropriately identified up to 10 potential CT findings ( Table 2 ). These sinus CT findings were determined to be a priority by the authors as critical anatomic variants to identify for ESS. These variants are not meant to be exhaustive of all potential sinus anatomy pitfalls; rather, they represent common potential sinus variants that could increase surgical risk and influence outcomes.
Critical Sinus Anatomic Variants Assessed Pre- and Postintervention. a
Corresponding CLOSE item in parentheses.
After collection of preimplementation data, the residents were taught in a formal teaching setting to use an ESS radiographic checklist by employing the CLOSE mnemonic. 7 The collection of correctly identified critical sinus CT findings was continued following the implementation of the ESS radiographic checklist.
The percentage of anatomic findings correctly identified was used to compare groups before and after implementation of the ESS radiographic checklist, with a 2-sample independent t test and a 2-sample paired t test and with statistical significance set at a P value <.05.
Results
A total of 57 preoperative encounters were analyzed: 28 before implementation of the ESS radiographic checklist and 29 after. Throughout the study, 9 senior residents (fourth- and fifth-year residents) and 9 junior residents (second- and third-year residents) participated in the study, and 3 supervising faculty surgeons collected the data.
The critical anatomic findings were grouped by their corresponding CLOSE item. A few of the critical anatomic findings did not have a corresponding CLOSE item, and these were evaluated individually. Preimplementation junior and senior residents identified 29% and 22% of critical anatomic findings, respectively, with a total identification of 24%. There was no statistical difference between the preimplementation groups overall for identification of critical anatomic findings. However, junior residents were significantly more likely to identify the lamina papyracea, Onodi, and sphenoid variants when compared with the senior residents ( Table 3 ).
Preimplementation of CLOSE Checklist: Comparison of Junior and Senior Residents. a
Data presented as percentage correct. Numbers in parentheses indicate encounters.
P ≤ .05.
P ≤ .001.
Postimplementation identification percentages improved significantly to 94% for junior residents and to 83% for senior residents ( Figure 1 ), with no significant difference between them in either overall or individual-item percentages ( Table 4 ). The overall change in identification percentages in the pre- and postimplementation groups ( Table 5 ) demonstrates a significant improvement in identification of critical sinus anatomy. Analysis of the individual findings demonstrates a significant improvement with implementation of the checklist with all items, except lamina papyracea ( Figure 2 ).
Percentage correct for junior and senior residents. post, postintervention; pre, preintervention.
Postimplementation of CLOSE Checklist: Comparison of Junior and Senior Residents. a
Data presented as percentage correct. Numbers in parentheses indicate encounters. Dashes (—) indicate not applicable.
Change in Sinus Anatomy Identification: Pre- and Postintervention for All Residents. a
Data presented as percentage correct. Numbers in parentheses indicate encounters.
P ≤ .001.
Percentage correct for the mnemonic “CLOSE” and items 1, 8, and 9 for all residents. post, postintervention; pre, preintervention.
Discussion
The implementation of preoperative surgical checklists is not a new concept: they have been used across many different surgical specialties with decreased surgical complications and improved detection of potential safety hazards.2,3 In our experience, we found that implementation of an ESS preoperative surgical checklist 4 improved identification of potential problem areas prior to surgery. The most commonly identified deficiency revolved around recognizing key anatomic variants on the CT scan. We therefore adapted the ESS preoperative surgical checklist and composed an additional sub-subcategory to specifically address surgical anatomy with the CLOSE mnemonic. Our findings demonstrate that the implementation of a systematic review of preoperative imaging with an ESS radiographic checklist improves identification of critical sinus surgical landmarks.
Particularly interesting is the finding of our improved identification of sinus findings that were not part of the CLOSE mnemonic. This suggests that the implementation of a systematic review improves the trainees’ attention to detail, even for items not directly included in the review. Preimplementation junior residents were more likely than senior residents to identify some critical sinus variants, which may suggest some complacency with experience and training. This difference was not appreciated following the implementation of the checklist. This suggests that the current system of surgical training—which relies on many years of training, repetition, intuition, and using recall knowledge—would have benefited the senior surgical residents. These findings highlight the differences among those industries that consistently use checklists, such as the airline industry and the medical field. We believe that the adoption of preoperative checklists prior to ESS should be seriously considered and should include a sinus CT imaging checklist, such as the CLOSE criteria, to improve identification of sinus anatomic variants.
In the book The Checklist Manifesto, Atul Gawande reports that with complicated tasks, a small error can lead to a dramatic change in outcome. Gawande suggests that the institution of checklists, as seen in the airline industry, lowers the risk of these small errors occurring and improves the overall outcomes. 9 In a similar fashion, our overall goal of using an ESS radiographic checklist is to reduce errors and minimize avoidable complications related to ESS. Although the checklist increased performance of tasks that may avoid errors, this study provides no direct evidence that it reduces complications. It is beyond the scope of this study to demonstrate reduced complication rates, and it is ethically impractical to randomize patients to a less prepared surgical team. Nonetheless, it is our belief and highly probable that improved identification of critical sinus anatomy prior to ESS will reduce surgical complications.
Another potential limitation of this study is that it was performed with physicians in training, and it is unknown if these findings can be extrapolated to practicing otolaryngologists doing ESS. Further studies would be beneficial to demonstrate improvement of surgical outcomes and reduction in mortality and morbidity associated with better preoperative identification of critical sinus anatomy. It would also be helpful to study if trainees continue to use the ESS radiographic checklist following training and maintain improved identification.
Conclusion
The introduction of a systematic approach to reviewing preoperative sinus CT scans with an ESS radiographic checklist improves identification of critical anatomic sinus variations in a training population.
Author Contributions
Disclosures
Footnotes
Sponsorships or competing interests that may be relevant to content are disclosed at the end of this article.
This article was presented at the Annual Meeting of the American Rhinologic Society: 2016 ARS @ AAO-HNS; September 16-17, 2016; San Diego, California.