Abstract
In the accompanying article, Anand et al. [1] describe the unfortunate conflict reached when the WAVE trial needed supplemental funding to deal with delayed enrollment and Canadian Institutes of Health Research (CIHR), the funding agency, wanted access to confidential interim results to inform its response to the request. The investigators declined to reveal interim results, citing the widely held view that doing so would jeopardize the ultimate interpretation of trial data. They argued that their independent Data and Safety Monitoring Board (DSMB) existed for the purpose of evaluating the trial progress and making recommendations about continuation (or not), and that CIHR should respect the DSMB recommendation.
The WAVE experience is worrisome, but access to interim trends by the funder or others is not the real issue in my view. The problem is a failure to plan for departure from the initial strategy for trial implementation.
My knowledge of WAVE is limited to what Anand et al. present; nor do I have actual knowledge of CIHR procedures for making funding decisions, although they probably resemble NIH procedures with which I am familiar. An initial decision to fund a trial is presumably based on assessment of relative priorities among far more meritorious applications than funds are available for. Background data and scientific rationales have roughly the same opportunity to convince. Later, when the original funding is used up before a trial is finished, as frequently happens with large trials, the assessment is inherently different. Funders can be reluctant to (effectively) reward the investigators for failing to do what they promised in terms of timely completion, when doing so would naturally disadvantage the current cohort of applicants. (Would CIHR have been more receptive to the request if a DSMB member had offered to underwrite it by reducing funding of his or her own new CIHR grant?) Meanwhile, the researchers point out that abandoning a trial just short of completion breaks the implied promise to volunteers about their contribution to generation of knowledge, and that, as a practical matter, the ‘large’ investment already made would go for nothing without the ‘small’ further investment being requested.
A more satisfactory approach would involve futility analyses, built into the protocol design. This would require difficult negotiation about reasonable stopping guidelines, possibly in terms of conditional power calculations. At a specified time (or times) after trial initiation, the statistician would present the futility analysis result to the DSMB, who would not divulge the actual conditional power but rather recommend stopping for futility or not. In making their recommendation the DSMB would take into account any data they deem relevant, that is, respecting the futility stopping guideline to the same extent as they would respect an early stopping guideline for efficacy.
Anand et al. argue, among other things, that an isolated primary analysis (or conditional power) for comparing groups on a single basis is not really a satisfactory way to characterize what new knowledge a clinical trial delivers. I certainly agree with that, and that we cannot hope to predict what particular analyses will prove to be the most interesting. On the other hand, in other conversations, trialists make sharp distinctions between planned and exploratory analyses. Ultimately the experimenters have to commit to one or a few primary questions to be addressed. Justification for requesting more funds can only very rarely be accepted on the grounds that secondary objectives should be promoted to primary ones. Indeed, it is unclear that DSMB members, no matter how expert, will be able to anticipate what a Steering Committee would choose to do if they had access to all the interim data.
There are important trials for which early stopping for futility is of no interest. An example given by the authors involves ‘a trial of a commonly used therapy’ that might turn out to be both ineffective and toxic. In such cases, the researchers and funders ought to agree at the outset that the primary objective can be met only by completing the trial as designed. This simplifies the discussion of funding extension considerably, because the funder’s choices are either to provide more funds or ‘to perpetuate the generation of unclear results’, as the authors assert.
In conclusion, I appreciate the editor’s invitation to comment on this interesting case. Confidentiality of interim trial results, including numerical results of conditional power calculations, is paramount, so I agree with the Steering Committee’s refusal to provide them to CIHR. This does not mean, however, that a funder should always accept a recommendation of a DSMB for a funding extension. I am not sympathetic to the proposition that funders should defer to DSMBs to the extent of unilateral changes in the key study objectives. In reality, more attention to contingency plans would avoid many or even most problems of this nature.