Abstract
During the past 6 years, the technique of renal sympathetic denervation has been proposed as a treatment for drug-resistant hypertension, and several studies have been published that claimed to provide supportive evidence of its efficacy. There is no question that resistant hypertension is a major medical concern: hypertension has been authoritatively designated as the greatest threat to the global burden of disease, and approximately 8% of hypertensive individuals have resistant hypertension. However, the first studies reporting the efficacy of renal sympathetic denervation were not methodologically capable of supporting such claims. Subsequently, in March 2014, the most appropriately designed clinical trial failed to provide supportive evidence of the intervention’s efficacy. This Commentary presents a cautionary case study to exemplify the need for optimal methodologic rigor in all experimental clinical research and highlights the pitfalls should claims from less than optimally rigorous studies be afforded more weight than is scientifically appropriate.
Keywords
Introduction
Writing many years ago in the voice of his beloved character Sherlock Holmes, Sir Arthur Conan Doyle wryly observed that it is a huge mistake to theorize before one has acquired the data, since to do so inevitably leads one to twist facts to suit theories rather than developing theories to suit facts. While his writings may not be commonly thought of as addressing the constructs now termed study design, experimental methodology, and inferential statistics (components of research that are inextricably linked by being integrative, interactive, and ideally seamless 1 ), Doyle’s observation would serve admirably as the opening statement in any introductory class on these topics and would also serve students well by being repeated at the end of the final lecture. With regard to the specific topic of this Commentary, renal sympathetic denervation (RDN) as a proposed interventional treatment for drug-resistant hypertension (hereafter called resistant hypertension), one could not imagine a more prescient admonition.
Regulatory agencies have a challenging two-edged mandate in that they are charged with both the promotion and the protection of public health. 2 They wish to bring appropriately safe and efficacious medical products to patients who need them as expeditiously as possible to promote public health, and regulatory science 3 can play an important role in this regard. However, regulatory agencies must also temper such legitimate and laudable desires with “calm, clinical, and comprehensive evaluations of approaches to any medical need.” 4 This Commentary is a cautionary tale not only for clinical researchers addressing legitimate and very important medical needs but also for regulatory agencies that must evaluate evidence presented to them by clinical researchers and determine if such research meets the evidentiary standards necessary for marketing approval. Our key message is simple, but we believe it is one that illuminates a grave and often overlooked principle: optimal study design in experimental clinical research is abandoned at one’s peril.
Central Characters in Our Case Study
To make the following discussion of this case study accessible to all readers, it may be helpful to reiterate the major characters. The first two characters are resistant hypertension and the proposed interventional treatment of RDN. The others include study design, experimental methodology, and inferential statistics and, in particular, two fundamental aspects of the ability of this triumvirate to facilitate scientific investigation: randomization, and appropriately controlled control treatment arms.
Hypertension and Resistant Hypertension
Papers published in The Lancet in 2013 designated high blood pressure (BP) as the greatest threat to the global burden of disease 5 –7 : it continues to be the most common diagnosis in adult primary care practice and the most salient cardiovascular disease risk factor. The American Heart Association’s Statistics Committee and Stroke Statistics Subcommittee recently reported that, on the basis of data from 2007 to 2010, 33% of adults ≥20 years of age in the United States have hypertension, that is, almost 80 million people (the prevalence is similar for both sexes). 8 Commenting on hypertension as the major cause of death and morbidity worldwide, Dolan and O’Brien 9 observed, “Ischaemic heart disease, ischaemic, non-ischaemic and haemorrhagic stroke, hypertensive heart disease, atrial fibrillation and flutter, peripheral vascular disease, aortic aneurysm and chronic renal disease (to which we must now add cognitive impairment and dementia) are all attributed to hypertension.”
There are multiple classes of antihypertensive medications, and many hypertensive individuals are prescribed regimens including drugs from more than one class. Resistant hypertension is defined as BP levels above specified targets (which can vary from treatment guideline to treatment guideline and by comorbid conditions such as diabetes and chronic kidney disease) despite adherence to at least three optimally dosed antihypertensive medications of different classes, including a diuretic, or BP that requires four or more antihypertensive medications to achieve control. 10,11 Detailed discussion of this topic requires addressing many considerations, including the following: adherence by physicians to treatment guidelines and by patients to prescribed regimens 12 ; the duration of maximal triple-drug therapy before considering a patient to have resistant hypertension (some authors have suggested that such a treatment regimen should have been in place for at least three months 13 ); discriminating among true resistant hypertension, controlled resistant hypertension, and pseudo–resistant hypertension 11 ; and the benefits of ambulatory BP monitoring in both the determination of resistant hypertension and the evaluation of proposed treatments. 13 Such discussion need not be exhaustively reexamined here: the key observation is that true resistant hypertension is present in approximately 8% of all hypertensive patients. 11 Given the enormous prevalence of hypertension worldwide, therefore, resistant hypertension is indeed a medical condition of great concern.
RDN
As Parati and Esler 14 observed, “The sympathetic nervous system was brought to public awareness in the early decades of the twentieth century by Cannon, through his research on, and popularization of the concept of, the ‘fight and flight’ response to stress.” They also noted that, in the past three decades, “the sympathetic system has moved towards centre stage in cardiovascular medicine,” and the importance of aberrations in this system has been demonstrated in hypertension and various other conditions, including heart failure. In both disease states, there is “persistent and adverse activation of sympathetic outflows to the heart and kidneys.” 14 With regard to hypertension, sympathetic innervation of the kidneys plays an important role in its pathogenesis through modulation of renin secretion, glomerular filtration rate, and renal absorption of sodium. 15
These observations, taken together with evidence of beneficial lowering of BP and reduced mortality with earlier surgical procedures that disrupted the sympathetic nervous system (albeit with severe and incapacitating adverse effects 15 ), have led to the development of radiofrequency-based renal nerve ablation technology in the past decade. Percutaneous RDN is a procedure in which radiofrequency energy is applied with an endovascular catheter inserted via the femoral artery to both renal arteries sequentially, with resultant ablation of the sympathetic nerve fibers.
Randomization, Study Design, and Inferential Statistics
Randomization is the process of randomly assigning participants to one of the treatment groups in a clinical trial so that the potential influences that cannot be controlled for (eg, height, weight) or cannot be determined by simple visual inspection (eg, specific metabolic pathway characteristics) are likely to be as frequent in one treatment group as they are in the other. 1 That is, randomly assigning participants to different treatment groups is done with the intent of avoiding any selection bias in participant assignment. Inferential statistics (discussed shortly) requires such random assignment to allow differences in responses between groups to be causally connected to the treatments administered. 1
Consider one simple but powerful clinical trial study design of relevance to current discussions: the randomized, blinded, concurrently controlled, parallel-group design involving a test treatment arm (the treatment arm in which participants receive the intervention of interest) along with a concurrent control treatment arm (which, in biopharmaceutical trials, may comprise a placebo or an active control). “Blinding” refers to the participants’ and investigators’ not knowing which treatment the participants receive. The descriptor “double-blind” (the greatly preferred condition) conveys that neither participants nor investigators know the group assignments, whereas “single-blind” typically conveys that only participants do not know their group assignments.
The discipline of inferential statistics provides accepted ways to analyze data from an appropriately designed clinical trial that permit statements, at various levels of confidence, of whether the data collected in the trial provide compelling evidence of a systematic influence on the data. 1 Imagine a clinical trial designed as a randomized, blinded, concurrently controlled, parallel-group study in which half of the participants are placed in the treatment group and receive RDN, and the other half are placed in the control group and therefore do not receive RDN. The systematic influence of interest is whether participants receive RDN, and the related question of interest is as follows: Is there compelling evidence that, on average, individuals receiving RDN showed a greater reduction in BP than individuals in the control group? If the study’s design was appropriate, the experimental methodology used during the conduct of the trial was of optimal quality, the statistical analysis conducted was an appropriate one for that study design, and the answer to the latter question is “Yes, there is compelling evidence,” it is permitted to make the inference that RDN would be a beneficial treatment not just for the specific set of participants who received it in this trial but also for the general population with similar clinical characteristics to those participating in the trial.
If such inference is to be permissible, two critical requirements of experimental methodology must be realized: randomization and appropriately constructed control conditions. Addressing the latter first, the control treatment group must receive an intervention that is as similar as possible in all respects to that received by the test treatment group: while not receiving the test treatment, all other aspects of both groups’ experiences (eg, number and timing of visits to the experimental site, blood draws and physical examinations, and physiologic measurements made to evaluate the treatment’s safety and efficacy) should be identical. In biopharmaceutical trials (certainly for orally administered agents), this is typically readily achieved by making the tablets received by the treatment group and the control group identical in size, shape, color, and taste, often by using over-encapsulation. In medical device and interventional procedure trials, it can be more challenging to construct identically structured control group circumstances, but clinical trialists must endeavor to do so should they wish to make legitimate inferences regarding the responses of trial participants and those of the general population: they should also make all (legitimate) limitations of study designs clear everywhere and on every occasion that their results are presented and discussed. 16
It should be noted here that useful information can certainly be gained from nonrandomized studies, particularly early in a medical product’s development program: the discussions here are not meant to denigrate such trials, which pave the way for later therapeutic confirmatory trials that have the ability to provide compelling evidence of a treatment’s efficacy. However, the results of such studies should be presented with appropriate clinical decorum: the scientific methodology of providing compelling evidence of efficacy and inferring how a treatment would likely benefit the general population with the clinical condition or disease of interest requires subsequent studies incorporating randomization.
A Set of Renal Sympathetic Denervation Studies
While other RDN studies have been reported, 17,18 a series of three SYMPLICITY HTN studies forms the basis of our case study. An initial report and two follow-up reports have been published for SYMPLICITY HTN-1, and an initial report and one follow up report have been published for SYMPLICITY HTN-2. The results from SYMPLICITY HTN-3 were published on-line on March 29, 2014, by the New England Journal of Medicine. Each of these studies is considered in turn.
SYMPLICITY HTN-1
Krum et al 19 reported the results of “a multicentre safety and proof-of-concept study” in The Lancet in 2009. Fifty participants with resistant hypertension (office systolic BP [SBP] ≥ 160 mm Hg on ≥3 antihypertensive drugs, including a diuretic) were recruited at five investigational sites in Europe and Australia. Forty-five received RDN, and five did not (for reasons discussed in due course). Reduction in office BP at 1, 3, 6, 9, and 12 months was a primary endpoint. Changes in BP for the two groups are presented in Table 1. The authors’ interpretation of their results was that RDN causes “substantial and sustained blood-pressure reduction.” 19 They also appropriately noted that “prospective randomised clinical trials are needed to investigate the usefulness of this procedure in the management of resistant hypertension.”
Results From the Original Report of SYMPLICITY HTN-1
BP, blood pressure; DBP, diastolic blood pressure; RDN, sympathetic renal denervation; SBP, systolic blood pressure.
Two subsequent reports based on the SYMPLICITY HTN-1 study were published. In 2011, the Symplicity HTN-1 Investigators published data in the journal Hypertension for the cohort of 45 individuals originally treated with RDN and a larger group of individuals (n = 153) who had subsequently been treated with RDN out to two years. 20 The greatest mean reduction in office BP, –32/–14 mm Hg (SBP and diastolic BP, respectively), occurred at 24 months. The authors’ interpretation of these data was that RDN “results in a substantial reduction in BP sustained out to ≥ 2 years of follow up.” 20
On November 6, 2013, The Lancet published on-line the final three-year report from SYMPLICITY HTN-1, which was subsequently published in print format in February 2014. 21 Data from 83 of the 153 individuals in the two-year follow-up report were available. The mean reduction in office BP at 36 months was –32.0/–14.4 mm Hg (SBP and diastolic BP, respectively). Moreover, reductions of 10 mm Hg in SBP were seen in 93% of patients at that time. The authors’ interpretation of these data was that “changes in BP after RDN persist long term in patients with treatment-resistant hypertension.” 21
SYMPLICITY HTN-2
Esler et al 22 published results from SYMPLICITY HTN-2 in 2010 in The Lancet, calling this trial a “randomised controlled trial.” Participants were randomly allocated (in a 1:1 ratio) to the test treatment group or the control treatment group. Those in the test treatment group (n = 49) maintained their previous pharmacologic treatment and received RDN; those in the control treatment group (n = 51) simply maintained their previous treatment. Mean baseline BPs for the two groups were almost identical. Mean office BP at 6 months was reduced in the treatment group by –32/–12 mm Hg (SBP and diastolic BP, respectively), while mean changes from baseline BP for the control group were –1/0 mm Hg. With regard to individual decreases in SBP of ≥10 mm Hg, such reductions were seen in 84% of those in the treatment group and 35% of those in the control group. The authors concluded that RDN can be used to “substantially reduce blood pressure” in treatment-resistant hypertension. 22
A follow-up study was reported by Esler et al 23 in 2012 in Circulation. Data were presented for two sets of individuals: participants who had initially received RDN, and for whom 12-month data were available, and participants who were initially in the control treatment group but were then offered RDN and therefore provided 6-month data. For the former group of participants, the mean 12-month reduction in office SBP was –28.1 mm Hg; for the latter group, the mean 6-month reduction was –23.7 mm Hg. The authors concluded that RDN provides “sustained reduction of blood pressure” to 1 year. 23
Limitations of SYMPLICITY HTN-1 and SYMPLICITY HTN-2
Before discussing SYMPLICITY HTN-3, let us consider the methodologic limitations of SYMPLICITY HTN-1 and SYMPLICITY HTN-2. SYMPLICITY HTN-1 did not incorporate randomization: of 50 participants enrolled in the study, five did not qualify to receive RDN “for anatomical reasons (mainly on the basis of dual renal artery systems).” 19 That is, the five participants in the control group were explicitly different from the 45 participants in the experimental treatment arm, an occurrence that is the antithesis of randomization. It is true that formal inferential testing was not reported; however, inspection of the study’s results, presented in Table 1, could certainly lead a reader less trained in experimental methodology to “perceive” a dramatic treatment effect. In fairness, it should be noted that the original study, a “multicentre safety and proof-of-principle cohort study,” 19 had a perfectly appropriate design for a treatment at the initial stage of investigation, and its report was also appropriately conservative in tone, stating (as previously noted) that “prospective randomised clinical trials are needed to investigate the usefulness of this procedure in the management of resistant hypertension.” 19 However, it seems that the methodologic limitations of the initial design were not heeded in the two follow-up reports: the claims made in the 24-month report 20 and the 36-month report 21 of this study became bolder and reached conclusions that overlooked the study design limitations.
With regard to SYMPLICITY HTN-2, the presentation of this study as “a randomised controlled trial” 22 is true by strict interpretation of the term “randomised,” but is misleading with regard to the descriptor “controlled.” The control group cannot be regarded as an adequately controlled control group: their experience in the trial (simply maintaining their previously prescribed pharmacologic treatment regimen) cannot be equated to the experience of the test treatment group, which comprised maintaining their previously prescribed pharmacologic treatment regimens and also undergoing the invasive procedure of RDN. Additionally, it can reasonably be argued that the authors’ conclusion that RDN can substantially reduce BP in resistant hypertension should be modified to one stating that RDN enhances the effects of the referent treatment, that is, the “standard of care” pharmaceutical regimen that was received by participants in both treatment arms: while such enhancement would likely be additive in nature, a synergistic effect cannot immediately be ruled out.
Finally here, the power of and the insights provided by ambulatory BP monitoring were not awarded their deserving gravitas throughout the RDN saga described to date. As noted earlier, ambulatory BP monitoring is advantageous in both the determination of resistant hypertension (in both clinical practice and in terms of determining eligibility for a trial of treatments for the condition) and the evaluation of proposed treatments, be they behavioral, pharmacologic, or interventional. Obtaining BP information across a 24-hour period (with perhaps four readings taken each hour) provides much richer information than do a (very) small number of BP readings taken in a physician’s office or at an investigational site conducting a clinical trial. Several publications highlighted that when BP decreases following RDN were evaluated using ambulatory BP monitoring, the decreases were of considerably less magnitude than those yielded by traditional clinical BP measurement, and various authors expressed concern at interpretations being made. 13,24 –27
SYMPLICITY HTN-3
Kandzari et al 28 published the rationale and design of SYMPLICITY HTN-3 in Clinical Cardiology in 2012. As the authors noted, “SYMPLICITY HTN-3 is a pivotal study designed as a prospective, randomized, masked procedure, single-blind trial evaluating the safety and effectiveness of catheter-based bilateral renal denervation for the treatment of uncontrolled hypertension despite compliance with at least 3 antihypertensive medications of different classes (at least one of which is a diuretic) at maximal tolerable doses.” The primary efficacy endpoint was the change in office systolic BP from baseline to six months, and a secondary efficacy endpoint was the change in ambulatory systolic BP from baseline to six months.
This trial’s design had notable differences from SYMPLICITY HTN-1 and SYMPLICITY HTN-2. First, 24-hour ambulatory BP monitoring was used to confirm resistant hypertension status: the mean 24-hour SBP had to be >135 mm Hg. Additionally, approximately five times as many participants were enrolled as in SYMPLICITY HTN-2, and this was the first blinded (single-blinded) randomized trial with an appropriately structured control treatment arm: the control treatment group experienced a sham procedure (renal angiography) that was as similar as possible to the RDN procedure without ablation of the renal nerves actually occurring. In this case, the randomization ratio was 2:1, with twice as many participants being randomized to the RDN treatment arm as to the control arm. This is a perfectly legitimate strategy in that it allows the collection of more safety data for the treatment of interest, RDN, than would be gained from a 1:1 randomization ratio (although it should be noted that it lowers statistical power to detect a difference in response between the two treatment groups should one truly exist).
Press Release Concerning SYMPLICITY HTN-3 in January 2014
On January 9, 2014, a press release announced that SYMPLICITY HTN-3 had failed its primary efficacy endpoint. 29 In the press release, one of the trial’s co–principal investigators noted, “While it’s disappointing that the trial did not meet its primary efficacy endpoint, this is the most rigorous renal denervation clinical trial conducted to date, and the first of its kind to include a sham-control group.” The press release also noted that enrollment in clinical trials for regulatory approvals (SYMPLICITY HTN-4 in the US, HTN-Japan, and HTN-India) had been suspended. Importantly, and appropriately, it was also announced that participants in SYMPLICITY HTN-3 would be followed as originally planned for 5 years to evaluate long-term safety. This is important because sensitive imaging technology has demonstrated that the damage to the renal vasculature as a result of RDN may be considerable, and the consequences beyond 6 months are not known. 30
Publication of the Full Results of SYMPLICITY HTN-3
The results of SYMPLICITY HTN-3 can be summarized succinctly. For the primary efficacy endpoint, change in office systolic BP at six months, the mean change in the test (RDN) treatment group was –14.13 mm Hg; for the sham control treatment group, it was –11.74 mmHg, yielding a point estimate of the treatment effect of 2.39 mm Hg. This finding was confirmed by even smaller decreases when reporting systolic BP reductions assessed via ambulatory BP monitoring: the mean decrease was 1.96 mm Hg. The authors concluded, “This blinded trial did not show a significant reduction of systolic blood pressure in patients with resistant hypertension 6 months after renal-artery denervation as compared with a sham control.” 31
Ramifications of Study Result Interpretations From SYMPLICITY HTN-1 and SYMPLICITY HTN-2
With the publication of the results of SYMPLICITY HTN-3, various groups of individuals now face challenges of differing import. First, approximately 5000 to 10,000 patients (estimates differ among sources) have received RDN worldwide. If they are concerned that they received a procedure that has been shown to be ineffective, how best are they to be counseled and provided appropriate physiologic and psychological care when they express their concern to their physicians? How also can they be reassured that satisfactory investigatory procedures are in place to detect the potential long-term adverse effects of RDN?
Second, in the face of this evidence and also the fact that RDN has been approved in various regulatory jurisdictions worldwide, how are physicians who would have continued to perform this intervention to reassess the evidence upon which their clinical interventions to date were conducted?
Third, two guidelines on RDN by professional societies were released in 2013: a position paper by the European Society of Hypertension 32 and an expert consensus document by the European Society of Cardiology. 33 The authors of these documents now face a difficult question: how does one begin to reconcile guidance on the use of an interventional technique with the observation that, according to the strongest level of evidence to date, the technique is ineffective? One anticipates that these guidelines will be revisited and rewritten.
Shortly after the January 2014 press release announcing that SYMPLITICY HTN-3 had failed its primary efficacy endpoint, in the UK, the British Hypertension Society, the British Cardiovascular Intervention Society, the British Society for Interventional Radiology, the National Institute for Clinical Outcomes Research, the British Cardiovascular Society, and the Renal Association called for a temporary moratorium on renal denervation procedures for all patients with hypertension as part of routine care in the National Health Service and private practice in the UK until the data from SYMPLICITY HTN-3 were fully available. 34 Now those data have been published, and we anticipate further statements from the joint UK societies.
Certain regulatory agencies, such as those in Europe that permitted RDN to be used in clinical practice ahead of compelling evidence of its benefit—evidence that has not been provided by SYMPLICITY HTN-3, the best designed and conducted trial to date—will now hopefully engage in serious and circumspect examinations of their processes; in contrast, and to its credit, the US Food and Drug Administration adopted a more cautious approach that prevented the procedure’s approval in its jurisdiction.
Concluding Comments
We believe that this case study powerfully illuminates the consequences of taking shortcuts in experimental clinical research. While acknowledging that genuinely caring physicians may wish that proposed effective treatments become available to their patients as quickly as possible, and that compassionate regulators can share these sentiments, methodological rigor is the sine qua non of scientific pursuit, including the evaluation of the efficacy of new medical products.
We readily acknowledge the possibility that other procedures for the ablation of renal sympathetic nerves may have beneficial effects in resistant hypertension, and possibly other cardiovascular disease states too. However, for the sake of patients worldwide, we implore global scientific and regulatory communities not to leap to conclusions before receiving methodologically rigorous evidence: as noted at the start of this Commentary, it is a huge mistake to theorize before one has acquired the data, since to do so inevitably leads one to twist facts to suit theories rather than developing theories to suit facts.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
