Bringing data monitoring committee charters into the sunlight

Introduction

Randomized controlled trials (RCTs) are essential for assessing benefits and risks of many medical interventions. ¹ In addition to oversight by an institutional review board, many RCTs require a data monitoring plan to protect participants from unnecessary risks and to ensure that the data are collected and analyzed rigorously. Many such trials use a data monitoring committee (DMC), sometimes called a data and safety monitoring board or committee, to conduct this work.^2,3 Although the National Institutes of Health (NIH) and Food and Drug Administration (FDA) policies specify which trials require a DMC, many other trials also use DMCs to implement their data monitoring plans.^4–7 While most clinical researchers are generally familiar with the concept of DMCs, their actual workings remain mysterious to many. ⁸ To explain what DMCs have done, some trials have published information about DMC deliberations.^9,10

ClinicalTrials.gov is the preeminent searchable database of clinical trial information. Over the years, as policies have evolved, ClinicalTrials.gov added the capacity to upload certain full-text study documents to complement the structured trial information that data providers must submit about their trials. Currently, all trial registrants are able to submit a full-study protocol, the statistical analysis plan, and the informed consent form as PDF documents as well as summary findings.^4,11 The result has been systematic transparency into previously “secret” documents. The DMC charter, a document which typically describes the procedures by which DMCs operate, contains details about the DMC’s organizational structure, membership, meeting frequency, sequential monitoring guidelines, and the overall contents of DMC reports for interim review, however, is generally accessible only to those directly involved in the trial. In our experience, DMC charters vary greatly in substance and quality. To increase the transparency of DMC operations, the quality of future charters, and the credibility of a trial, we urge the National Library of Medicine to modify ClinicalTrials.gov to allow voluntary upload of DMC charters. This simple technical revision would then allow clinical trialists to submit the DMC charters for their trials, leading to improved insight into current DMC practices.

Background

RCTs remain the gold standard for evaluating the impact of interventions for human health. Until the launch of ClinicalTrials.gov in 2000, however, the public had little access to important information about the existence, design, status, and even findings of most trials. Public information was not available concerning which trials had been initiated; for what populations, interventions, and conditions; what primary outcome measures had been prespecified; when or whether the trial had been completed; and, if published, whether or not the reported analyses were consistent with the prespecified protocol. Following instances in which publications provided misleading information about the trial design and the summary results, editors of medical journals were influential in advocating for prospective trial registration. ⁴ This action was followed by a set of legal and funder-specific policies requiring trial registration and reporting of summary results. ⁴

Since its inception, ClinicalTrials.gov has accommodated this evolving set of policies by adding features, including a results database in 2008, and quality control processes that expanded and improved the information available about each trial. ¹² The sponsor or principal investigator is required to submit information about the trial in structured fields; the staff of the National Library of Medicine (NLM) performs quality control review. ClinicalTrials.gov now provides summary information from the trial protocol that is prepared prior to trial initiation; the website includes updates to study status and changes to the protocol as reflected by amendments made during the trial. ¹² When the trial has ended, structured tables summarize participant flow, baseline characteristics, and the results of all primary and secondary outcome measures, as well as the frequency and type of serious and frequent adverse events.^5,11

In addition to the structured fields, since June 2017 ClinicalTrials.gov has accepted the full-text trial protocol, statistical analysis plan (SAP), and informed consent form as PDF files. ClinicalTrials.gov currently has information on over 400,000 clinical trials. The data include over 55,000 sets of trial results, over 22,000 full-text protocols and SAPs, and over 5500 informed consent forms. While these documents can be provided to ClinicalTrials.gov at any time, various Federal policies specify required submission timelines for certain clinical trials.^4,11

The availability of full-study protocols, SAPs, and informed consent forms provides researchers with the capacity to learn more about individual trials and to identify better (and worse) practices to improve the quality of future trials. Representative samples of trial protocols and informed consent forms can be compared with standards to identify practice variation and areas needing improvement.^13,14

The importance of DMCs

An important area of clinical trial functioning and oversight, however, remains out of public view: how data accumulating during a trial are monitored to protect the trial participants as well as the integrity of the data and the analyses, a responsibility often held in part by a DMC that has unique access to unblinded data while the trial is ongoing.^8,15

A well-constructed DMC charter outlines the principles by which the DMC functions for a specific trial or set of trials; it does not list a set of rules, but provides guidelines to help the DMC carry out its mission. Charters describe DMC membership, the relationship to the trial sponsor, considerations regarding conflict of interest, and overall functioning, including the DMC’s responsibilities in monitoring trial progress, data quality, and emerging trends for evidence of early overwhelming benefit, trial futility, or potential harm to participants. Charters should specify how, and to whom, the DMC makes its recommendations. Many charters include statistical monitoring procedures used to guide the DMC, especially for issues related to early termination for benefit or potential harm. While the DMC charter cannot capture every aspect of DMC deliberations, it should use best principles to provide a framework for operations. ¹⁵ The full DMC charter is usually a document separate from the protocol. Table 1 shows the suggested content of a DMC charter. ²

OPEN IN VIEWER

Table 1.

Contents of a DMC charter. ²

The Data Monitoring Committee Charter TITLE OF PROTOCOL: PROTOCOL NUMBER: SPONSOR OF PROTOCOL: DATE OF DOCUMENT: TABLE OF CONTENTS 1. INTRODUCTION 2. PRIMARY RESPONSIBILITIES OF THE DMC 3. ORGANIZATIONAL DIAGRAM 4. MEMBERSHIP OF THE DMC  4.1. Members  4.2. Conflicts of Interest 5. TIMING AND PURPOSE OF THE DMC MEETINGS  5.1. Organizational Meeting  5.2. Early Safety/Trial Integrity Reviews  5.3. Formal Interim Efficacy Analyses 6. PROCEDURES TO ENSURE CONFIDENTIALITY; PROPER COMMUNICATION  6.1. Closed Sessions  6.2. Open Session  6.3. Open and Closed Reports  6.4. Minutes of the DMC Meeting  6.5. Recommendations to the Steering Committee (SC) 7. STATISTICAL MONITORING GUIDELINES 8. CONTENT OF THE DMC’S OPEN AND CLOSED REPORTS  8.1. Open Statistical Report: An Outline  8.2. Closed Statistical Report: An Outline

In April 2007, ClinicalTrials.gov added an optional data field to include during trial registration, addressing whether the registered trial involves a DMC. Over the past 5 years, 80% of registrants have responded to this optional field, allowing broad statements about DMC use. For example, Califf et al. ⁵ reported that about 40% of trials responding to this question used a DMC, with important differences by funder (more use in NIH-funded trials than in industry trials), intervention type, and disease category. Current data from Clinical Trials.gov indicate that nearly 14,000 active trials with sites only in the US report having a DMC. It is unknown how many of these DMCs have a charter nor what the content or quality of those charters is.

Lack of public access to DMC charters

Several guidelines describe the importance of DMCs and discuss how DMCs should function;^2,6,7,15 however, little public information is available about how DMCs actually function. ⁸ The specific manner in which the DMC operates can be essential for protecting trial participants as well as for ensuring the trustworthiness of the trial’s analyses. Protocols may indicate the existence of a DMC and its membership but often does not describe details of monitoring or interim analysis. For some trials, this could leave those interpreting trial findings without sufficient information to understand the results fully. This lack of transparency may be especially problematic for trials that terminate early.

Although regulators may review DMC charters for certain trials, and charters have sometimes been made available to other clinical trialists, for most trials, it is difficult to obtain details about how a DMC functioned. Often the sponsor’s role or the interim analysis plans are opaque even long after the trial has been completed and the results reported. Access to such details would be informative about whether the trial used DMC best practices. Such details could inform judgments about whether or not the trial participants, and the trial data and analyses, were appropriately protected. For example, did the DMC have proper multidisciplinary representation, such as inclusion of both clinical and statistical experts, or regional representation in international trials? Was the range of clinical expertise broad enough to allow informed judgment about safety? Did the meeting format protect the committee’s independence and enhance its effectiveness? During the trial, were DMC members and those who reported data to the DMC the only ones with access to unblinded data about both harms and benefits? Did the DMC have access to unblinded efficacy as well as safety data at each meeting? Were a priori guidelines included for establishing convincing evidence of benefit or harm, or for considerations of futility? Was the DMC allowed to ask the reporting statisticians for additional analyses without informing the sponsor or the investigators and did the statistical reporting group have the authority to produce those analyses without informing the sponsor or investigators? Were DMC recommendations formulated independently, using a process that emphasized consensus development?

Recommendation

In order to bring sunlight into the world of DMC oversight, we make the following recommendations: first, we urge the National Library of Medicine to modify ClinicalTrials.gov to allow the upload of a PDF of a DMC charter for any registered clinical trial. We believe that the mechanism currently in use for the upload of other study documents is well suited to this task. We also recommend modifying the search interface to allow easy identification and retrieval of trials with DMC charters. Second, we urge the “Responsible Party” for the ClinicalTrials.gov record (i.e. the sponsor or the designated principal investigator) to make use of the opportunity to upload a DMC charter for any trial that has one. The resulting cache of DMC charters—even if only a small percentage of trialists make use of the optional upload—would provide a unique window into current practice and could clarify many questions about specific trials. We anticipate that researchers will be able to use this resource to gain understanding into specific trials, to identify current and “best” practices, and to identify areas ripe for quality improvement. In addition, we anticipate that availability of existing charters could help trialists by informing the development of new charters.

Our proposals would create a mechanism by which DMC charters could be made publicly available. At this time, no policy (or policy proposals that we are aware of) would mandate such an action. For this to be successful, the trial community would have to decide to participate. One critical question that will have to be addressed by individual trial sponsors is the timing of upload. There are several considerations. On one hand, upload at the time of trial onset has the advantage of full transparency regarding trial conduct and could be especially relevant for current participants; on the other hand, if the charter contains details of interim analysis plans, which is usually the case, others could use such information to “second guess” the trial’s ultimate outcome. Posting the charter when the DMC’s work is complete, generally after the trial has ended, would avoid this potential problem. In a clinical trial with a complex design or controversial results, timely access to the DMC charter could improve interpretability of trial results, especially in settings where important design changes regarding alpha-spending occurred during trial conduct. In such cases, it is important to know if the DMC, when unblinded to efficacy and safety data, was involved in decisions to alter trial design.

In addition to timing, the inclusion (or redaction) of DMC member names must be considered. DMC charters typically list the DMC members; the acknowledgment section of many clinical trial publications also names the DMC members and the statistical team preparing DMC reports. We believe that the specific expertise and experience of the DMC members is part of the trial’s documentation and should be known once the trial is over. DMC members should be acknowledged for their activity and their home institutions should view their work as contributing positively to science. Making the names publicly available only after the trial is completed prevents any potential problems that could ensue should individual DMC members be identified while a trial is ongoing.

The analysis of accumulating information should inform future policy recommendations. We consider ClinicalTrials.gov the most logical place to make DMC charters public because it is the established archive for structured and searchable trial information. ¹⁶ It already contains the trial protocol, informed consent form, SAP, and a summary of the final results. Experience with providing access to this information leads us to believe that increasing access to DMC charters will have a beneficial effect on transparency, accountability, and quality of trials.

Many trialists recognize a growing need to expand and train the next generation of potential DMC members. ¹⁵ In addition to existing textbooks on practical aspects of DMCs,^2,3 the Society for Clinical Trials’ website provides free video lectures that describe DMC structure and function. ¹⁷ Earlier papers have stressed the need not only for better training for DMC members but also for improvement in the reports the DMC reviews.^18–20 Better training and improved interim reports, however, are not sufficient for high-quality functioning of a DMC; charters, too, should reflect best practices.

Summary

Five documents are important to understanding a pivotal Phase 3 RCT: the protocol, the model informed consent form, the SAP, the DMC charter, and the paper describing the final results. All except the charter are currently publicly available. The protocol, the model informed consent form, and the SAP are posted on ClinicalTrials.gov, and the final results should be published in a scientific journal and summarized on ClinicalTrials.gov. Sharing these documents has improved the quality of clinical research. Trial documents are also critical in publication review by journals and in subsequent interpretation of trial results by health care professionals. Moreover, they support the public’s trust in the credibility and integrity of the clinical trial research enterprise. Adding the fifth document, the DMC charter, to ClinicalTrials.gov would complete the five-legged description of an RCT. We believe that public access to the DMC charter of a trial will enhance the credibility of the trial and ultimately improve the quality of DMC charters as well as clinical trials in general.