Abstract
The “Cochrane Corner” is a quarterly section in the Journal that highlights systematic reviews relevant to otolaryngology–head and neck surgery, with invited commentary to aid clinical decision making. This installment features a Cochrane Review, titled “Functional Endoscopic Balloon Dilation of Sinus Ostia for Chronic Rhinosinusitis,” that finds no convincing evidence supporting balloon dilation compared to conventional surgical modalities for managing refractory disease.
Endoscopic sinus surgery for chronic rhinosinusitis is one of the most common currently performed outpatient otolaryngologic procedures, with an estimated 257,000 ambulatory sinus surgeries performed annually in the United States. 1 Since the introduction of the sinus endoscope, which redefined sinus surgery, several advances in instrumentation have become part of the armamentarium for endoscopic sinus surgery, including high-definition video cameras, powered instrumentation such as the microdebrider, and image guidance systems. More recently, balloon dilation of sinus ostia has emerged as another, but controversial, tool in this armamentarium.
Cochrane Abstract: Functional Endoscopic Balloon Dilation of Sinus Ostia for Chronic Rhinosinusitis, by Ahmed J, Pal S, Hopkins C, Jayaraj S 2
Disclaimer
This is an abstract of a Cochrane Review published in the Cochrane Library 2010 Issue 5 (see www.thecochranelibrary.com for information). Cochrane Reviews are regularly updated as new evidence emerges and in response to feedback, and the Cochrane Library should be consulted for the most recent version of the review.
Background
Dilation of sinus ostia using a high-pressure balloon has been introduced as a treatment for chronic rhinosinusitis (CRS) refractory to medical treatment. The efficacy of this technology, however, has not been systematically reviewed.
Objectives
To assess the effectiveness of balloon sinus ostial dilation as a treatment for patients suffering with CRS refractory to medical treatment.
Search Methods
We searched the Cochrane Ear, Nose and Throat Disorders Group Trials Register; CENTRAL; PubMed; EMBASE; CINAHL; Web of Science; BIOSIS Previews; Cambridge Scientific Abstracts; ISRCTN and additional sources for published and unpublished trials. The date of the most recent search was 20 December 2010.
Selection Criteria
Randomized controlled trials in patients of any age with rhinosinusitis lasting longer than 12 weeks who have failed a prolonged course of medical treatment. Studies compared either balloon sinus ostial dilation or a hybrid procedure (balloon dilation in conjunction with functional endoscopic sinus surgery (FESS)) versus conventional surgery (eg, FESS) or a waiting list control.
Data Collection and Analysis
Two authors independently selected studies for inclusion, extracted data and assessed risk of bias.
Results
One study (34 patients) met our inclusion criteria although it was not yet a peer reviewed publication. The study randomized patients with chronic frontal sinusitis who had failed a prolonged course of medical treatment into two groups: balloon dilatation of the frontal recess (plus conventional FESS of other involved sinuses) versus conventional FESS (Draf type 1/2a procedures on the frontal sinuses). At 12 months follow up there was no statistically significant difference in radiological resolution of frontal sinuses between the two groups. The percentages of directly observed patent frontal recesses at 12 months were 75% in the balloon dilation group versus 63% in the FESS-only group. The authors state that this was statistically significant but details of the analysis were not presented. Indeed the study as a whole suffers from a bias in the way its outcome measures were reported.
No major complications were reported. Three patients in the FESS-only group required further revision frontal sinus surgery compared to one in the balloon dilation group, although synechiae were more common in the latter.
Authors’ Conclusions
At present there is no convincing evidence supporting the use of endoscopic balloon sinus ostial dilation compared to conventional surgical modalities in the management of CRS refractory to medical treatment. With the escalating use of balloon sinuplasty, there is an urgent need for more randomized controlled trials to determine its efficacy over conventional surgical treatment modalities.
Comments on Cochrane Review
Comments by Bhattacharyya
At first glance, those who wish to minimize the findings of this Cochrane Review may suggest that such a review is premature since balloon dilatation is a new technique. However, as noted in the review, balloon dilatation was approved in the United States in 2005, and after initial publication of safety and feasibility studies, a consistent density of publications has emerged regarding patients treated with this technology from several different authors and different institutions.
Unfortunately, as this Cochrane Review poignantly elucidates, the level of evidence concerning balloon dilatation for chronic rhinosinusitis is limited. All published studies that collected clinical or radiological outcome data, respectively, were limited to uncontrolled, nonrandomized case series ( Table 1 ) and were therefore excluded from the review. Other studies were excluded for lack of randomization, lack of a control group, or because of their retrospective nature. This is unfortunate because despite many studies with large sample sizes and likely considerable costs for conduct, we lack a level of evidence for balloon dilatation that is fit to be included in a Cochrane Review.
Studies Excluded from Consideration in the Systematic Review
Information extracted from Ahmed et al. 2
Generally, with the evolution of new technology, health care stakeholders would like to see a steady rise in evidence level as that technology is acquired and assimilated into the surgeons’ repertoire. 3 After early studies of safety and feasibility, cohort studies demonstrating a meaningful clinical benefit are hopefully, and naturally, followed by randomized controlled trials (RCTs) demonstrating efficacy. In the case of balloon dilatation techniques, randomized trials could be performed to demonstrate efficacy of balloon techniques relative to continued medical management alone or relative to the currently accepted surgical standard of conventional endoscopic sinus surgery.
Some may argue that randomized trials were not required for powered instrumentation or for image guidance systems to be adopted as tools in endoscopic sinus surgery. Although this is true, the introduction of balloon technology has been perceived as a relative paradigm shift engendering controversy, as the review authors point out. This shift perhaps is most clearly evidenced by the promulgation of different terminology and the renaming of sinus surgery using this technique as balloon catheter dilation, functional endoscopic dilatation of the sinuses, balloon catheter sinusotomy, and balloon sinuplasty. The coinage and use of such terms imply a substantial change in the fundamental nature of the surgery being performed and beg for evidence to justify them.
Why are we lacking RCTs of balloon dilatation for chronic rhinosinusitis? The reasons may be myriad for this evidence gap, but the consequences of such an absence are striking. It is quite common for new technology to be understandably more expensive. Health care stakeholders such as third-party payers, hospitals, and patients have grown increasingly analytical about new technologies primarily because the advent of new technologies does not always guarantee better results, and in the case of similar results to existing therapies, their increased costs may not be justified. 3 Furthermore, such evidence gaps limit the conclusions that can be incorporated into guideline statements for treatment and also often leave determinations about insurance coverage and reimbursement to third-party payers.
Based on the multiple cohort studies published and cited in this review regarding balloon dilatation of the sinus ostia, it is clear that many surgeons feel strongly in favor of this technology. Clearly, the findings of this Cochrane Review might serve as a “call to task” for those in favor of this technology, hopefully stimulating ascension to the next level of evidence allowing stakeholders to draw more definitive conclusions about the value of balloon technology for the treatment of chronic rhinosinusitis going forward. Ahmed and colleagues 2 indicate that the next stage of their review is expected in December 2012, and the potential appearance of a second Cochrane Review with no includable studies would constitute a further palpable absence of evidence.
Comments by Burton
Those who introduce new surgical techniques or devices should demonstrate that these are effective and safe. Is this unreasonable? Few would argue—I hope—that it is a sensible and reasonable request. They may argue about the best way of demonstrating effectiveness and safety.
Many surgeons are innovators, and history demonstrates the merits of such innovation. But the maverick surgeon, working alone in an unregulated environment, “trying out” new ideas on passing patients, is a character from the past. Surgical innovations in the 21st century must be evaluated critically, rigorously, and scientifically.
A series of meetings comprising surgeons, methodologists, trialists, and others was held in Oxford in 2008-2009 and became known as the Balliol Colloquia. I was a participant. The group produced a series of 3 papers, published in The Lancet in September 2009,4-6 on “Surgical Innovation and Evaluation,” examining the stages of surgical innovation and the challenges of evaluating such innovation. The third paper sought to clarify the way ahead and was titled, “No Surgical Innovation without Evaluation: The IDEAL Recommendations.”
IDEAL stands for Idea, Development, Exploration, Assessment, and Long-term study. It comprises a recommended framework to encourage concurrent evolution, evaluation, and reporting of new surgical techniques. Recently, Blazeby et al 7 applied the IDEAL framework retrospectively to a prospectively created database of patients undergoing minimally invasive esophagectomy. They demonstrated that the recommendations could be applied in practice and identified areas where IDEAL could be improved, specifically in regard to regulation and training.
The “evidence” required at different stages of the IDEAL process varies. At stage 1, the “Idea”/proof-of-concept stage, structured case reports will suffice. But in stage 2a (Development), when the technique is undertaken only by the innovator and some early adopters, prospective studies are required. During the following stage (2b—Exploration), this should include a research database, and at this time point, an exploratory RCT may be undertaken. An RCT should certainly be the default option for evaluation at stage 3—Assessment. It is accepted that other study designs may be appropriate, but a variety of modifications of the standard RCT (eg, expertise-based randomized trials) might be of particular relevance in surgical research. RCTs should not be summarily dismissed as “impossible in surgery.” The final stage—stage 4—is the Long-term study phase and can include the use of registries and routine databases to collect data on long-term effects and identify rare adverse events.
The authors of the Cochrane Review were right to seek evidence from RCTs to evaluate the efficacy of balloon dilatation for CRS. Dr Bhattacharyya is right to ask why they did not find any. Is it because the procedure is still “under development” (stage 2a)? The widespread worldwide promotion and marketing of the technique, as well as the technology required to perform it, suggests otherwise. It requires rigorous and appropriate evaluation, and I agree that those who have developed the concept and explored its application should be charged with evaluating it against continued medical treatment or more conventional surgical techniques in an RCT. Careful consideration needs to be given to the appropriate choice of outcomes, but the Cochrane Review provides some guidance on this issue. When high-quality, well-reported RCTs are published, they will be incorporated in the updates of the Cochrane Review.
Comments by Rosenfeld
The paucity of evidence supporting balloon dilation ( Table 1 ) echoes nicely the insights of Marple and coworkers 3 : “Historically the use of higher evidence standards in surgical subspecialties has lagged behind that of medical disciplines. This is an inherent problem mostly due to US Food and Drug Administration (FDA) requirements and not necessarily surgeons’ lack of ability to conduct high-quality research.” About 95% of medical devices cleared by the FDA are via the 510(k) clearance mechanism, which is used when a proposed device has “substantial equivalency” to an approved device. 3 The “approved device” with regards to balloon dilation is traditional instrumentation for functional endoscopic surgery. This comparator, however, has also not been shown superior to medical therapy for CRS. 8
When efficacy data are lacking, management decisions should be guided by safety, cost, and experience. Balloon dilation appears safe, but cost is substantial, and experience varies greatly with sparse data on long-term outcomes. Nonetheless, physicians are often partial to gizmos, which can be so appealing based on first principles that they “make sense” without evidence or even with evidence to the contrary. 9 Moreover, gizmos (like balloons) can impart a halo effect, whereby the perception of efficacy based on technological sophistication creates cognitive bias that can distort outcomes through rose-colored glasses.
Ultimately whether or not to use balloons, or any other new technology, should be based on shared decision making with patients about the supporting evidence or lack thereof. There is nothing wrong with safe, judicious, and cost-effective use of new technologies, but we must all shun unfounded claims of better outcomes based solely on gizmo appeal or halo effects when unbiased RCTs are lacking.
Author Contributions
Disclosures
Footnotes
Sponsorships or competing interests that may be relevant to content are disclosed at the end of this article.