Abstract
Background
Randomized clinical trials often encounter slow enrollment. Failing to meet sample size requirements has scientific, financial, and ethical implications.
Aims
We report interventions used to accelerate recruitment in a large multicenter clinical trial that was not meeting prespecified enrollment commitments.
Methods
The Carotid Revascularization Endarterectomy vs. Stenting Trial began randomization in December 2000. To accelerate enrollment, multiple recruitment tactics were initiated, which included expanding the number of sites, hiring a recruitment director (May 2003), broadening eligibility criteria (April 2005), branding with a study logo, Web site, and recruitment materials, increasing site visits by study leadership, sending e-mails to the site teams after every enrollment, distributing electronic newsletters, and implementing investigator and coordinator conferences.
Results
From December 2000 through May 2003, 14 sites became active (54 patients randomized), from June 2003 through April 2005, 44 sites were added (404 patients randomized), and from May 2005 through July 2008, 54 sites were added (2044 patients randomized). During these time intervals, the number of patients enrolled per site per year was 1.5, 3.6, and 5.6. For the single years 2004 to 2008, the mean monthly randomization rates per year were 19.7, 38.1, 56.4, 53.0, and 54.7 (annualized), respectively. Enrollment was highest after recruitment tactics were implemented: 677 patients in 2006, 636 in 2007, and 657 in 2008 (annualized). The prespecified sample size of 2502 patients, 47% asymptomatic, was accomplished on July 2008.
Conclusions
Aggressive recruitment tactics and investment in a full-time recruitment director who can lead implementation may be effective in accelerating recruitment in multicenter trials.
Introduction
Successful randomized clinical trials (RCTs) require efficient organization, management, and marketing to ensure adequate enrollment (1–4). Failing to meet sample size requirements has scientific, financial, and ethical implications. Statistical power is compromised, which undermines testing the trial's primary hypothesis. Inefficient enrollment delays trial completion, leading to additional costs. Studies may be shut down by funding or regulatory entities before reaching meaningful conclusions (5,6). Underenrollment can be unethical. Participants consent to the risks and inconveniences mandated by the study, and family members share in the inconveniences. If enrollment is not adequate for the study to achieve its aims, the participants have been subjected to the risks and inconveniences unnecessarily. These consequences can lead to mistrust of research by potential participants and so jeopardize enrollment in future studies (7).
The Carotid Revascularization Endarterectomy vs. Stenting Trial (CREST) was designed to contrast the relative safety and efficacy of carotid artery stenting (CAS) vs. carotid endarterectomy (CEA) (8,9). The protocol was approved by the institutional/ethics review boards (IRB) at participating sites. All participants provided written informed consent.
Aims
Enrollment in CREST was slower than required to achieve the predetermined sample size of 2500. Roadblocks included the extended process for credentialing of operators for the CAS procedure (10), delays by the Center for Medicare and Medicaid Services (CMS) to reimburse CAS (11), and suspension of enrollment from February to June 2002 due to a protection device failure. We describe recruitment tactics used in a large multicenter National Institutes of Health (NIH)-funded stroke prevention trial that was not meeting prespecified enrollment commitments (8–11).
Methods
In April 2003, the Investigational Device Exemption (IDE) was transferred to the CREST Administrative Center at the University of Medicine and Dentistry of New Jersey (UMDNJ). The IDE created the need for additional staff. Two senior clinical research associates were hired to carry out site start-up and management. In May 2003, a recruitment director was appointed to coordinate enrollment efforts across all centers. A recruitment center, composed of the study national coprincipal investigator (PI) and the recruitment director, was funded and established, centralizing the coordination of recruitment efforts.
CREST initially received NIH and Food and Drug Administration (FDA) approval for 40 centers. Study leadership determined early in the enrollment phase that additional sites would be required. Approvals for 50 sites, then 60, 70, 110, 115, and 119 were obtained from October 2000 to June 2009. Development of a recruitment plan was required by each site.
In 2004, results of the Asymptomatic Carotid Stroke Trial (12) reported a benefit of CEA in asymptomatic patients similar to that shown in the Asymptomatic Carotid Atherosclerosis Study (13). Because the majority of CEAs in the United States were being performed for asymptomatic disease, study leadership proposed to the Data and Safety Monitoring Board, NIH, and FDA and they approved that the CREST protocol be amended to include asymptomatic patients (8). The first asymptomatic patient was enrolled in April 2005. The recruitment director worked with each site to update recruitment tactics and materials for this new pool of potential patients.
Monitoring of enrollment was increased by the recruitment director. Status of low-enrolling sites, obstacles to recruitment, tactics for improvement, and timelines for enrollment targets were reported weekly to the recruitment committee. Action items for sites not meeting monthly recruitment goals for three consecutive months were discussed and implemented. Such actions might include telephone calls by the national PI to the local site PI followed by formal letters outlining the site's enrollment rate and requesting a timeline for expected improvement. After six-months of no improvement, a site could face suspension from enrollment until a written site-specific recruitment plan was presented to and approved by the committee.
The recruitment director and the recruitment committee developed new tactics (Table 1) that were tailored depending on the characteristics of the site (academic or community-based), the medical specialty (and referral population) of the site PI, the size and location of the site (e.g. Canada), or upon the target population (e.g. women and minorities). Recruitment materials required approval of each site's IRB and the UMDNJ IRB. The study logo was used on all CREST materials. Recruitment supply packages with enrollment tools (Table 1) such as the posters, study brochures, laminated pocket cards, and coordinator protocol fact sheets were developed and shipped to the sites on a monthly basis and/or as needed.
Summary of multiple tactics used simultaneously and repeatedly to accelerate recruitment
The e-mail congratulating individual CREST teams for each patient randomized was sent from the recruitment director and followed by an e-mail from the national PI. The recruitment director also made personal calls on a rotating basis to coordinators to ascertain and address recruitment issues or obstacles. Over 120 site visits for the randomized phase of the study were made by either the national PI, the national co-PI, the recruitment director, or the senior clinical research associates.
The Recruiter, a bimonthly newsletter, recognized enrollment achievements with photos of site teams. Updates regarding study protocol, training, data compliance, and additional recruitment tips were included. An enrollment summary report by site was attached, allowing sites to compare their randomization rates.
The planning of the annual CREST Principal Investigators' conferences and the annual coordinator conferences as well as more frequent telephone conferences focused on protocol updates, protocol compliance, training, and the status of the trial. New recruitment tools, women and minority recruitment, issues or obstacles to enrollment, and patient retention were also included.
The CREST Web site included a section dedicated to the general public and provided a study overview. Participating sites with PI and coordinator contact information were listed in a directory, allowing potential participants to locate centers closest to their area. Another section solely for investigators and coordinators required a login and password. The ‘Recruiting Tools’ section provided links for download of recruitment materials such as the template letters, press releases and newspaper advertisements, slide sets, and the coordinator fact sheet. Information regarding the informed consent process, recruitment tactics, tips for improving enrollment of women and minorities, and a product overview by the manufacturer of the carotid stent and embolic protection systems were also included.
Results
Patients were randomized from 21 December 2000 to 18 July 2008 (Table 2). From December 2000 through May 2003, the month the recruitment director was hired, 14 sites became active, and 54 patients were randomized. The mean monthly randomization rate was 1.8. From June 2003 through April 2005, the month asymptomatic patients became eligible, 44 active sites were added, and 404 patients were randomized. The mean monthly randomization rate was 17.6. From May 2005 through July 2008, 54 active sites were added, and 2044 patients were randomized. The mean monthly randomization rate was 52.4. During these three time periods, the number of patients enrolled per site per year was l.5, 3.6, and 5.6. The costs of the enhanced enrollment activities from the beginning of 2003 until completion of enrollment were considerable with personnel costs exceeding $770 000 (Table 3).
Notable study activities influencing recruitment and number of active sites, yearly enrollment, monthly average rate of enrollment, and average enrollment per site per year from 2000 to 2008
Annualized.
Sx, symptomatic; Asx, asymptomatic; NA, not applicable (asymptomatic patients were not eligible for enrollment into CREST until April 2005).
Selected costs for enrollment in CREST (May 2003–July 2008)
The numbers in this column are estimates of selected costs for enrollment activities in CREST. They do not represent all enrollment costs.
For the five-years (2004–2008) following the hiring of the recruitment director, the mean randomization rates per month were 19.7, 38.1, 56.4, 53.0, and 54.7 (annualized). Enrollment was highest during the years 2006 (n = 677), 2007 (n = 636), and 2008 (n = 657, annualized), following implementation of new recruitment tactics and inclusion of asymptomatic patients. Actual vs. estimated randomizations for the enrollment period are provided in the Fig. 1. The maximum number of patients randomized in a single month was 76 in August 2006 (97 active sites). Ten subjects were enrolled in a single day on 8 June 2007 (108 active sites).
CREST actual vs. estimated randomizations (n = 2502) (last patient randomized 18 July 2008).
Of the 119 approved sites, 117 enrolled at least one participant. The median number of enrollments per site was 16, interquartile range 6–27. Three sites individually enrolled >100 participants. The leading center was a university hospital (Oregon Health Sciences University, Portland, OR, Wayne Clark, MD, PI), second was a community hospital (Central Baptist Hospital, Lexington, KY, William Brooks, MD, PI), and third was a city hospital in Canada (CHA, Hôpital de l'Enfant-Jésus, Québec, QC, Ariane Mackey, MD, PI). Eight sites enrolled >50 participants. Forty-five sites enrolled <10; as a result of site recruitment monitoring, 59 sites received warning letters for underenrollment (28 received more than one warning letter), 11 sites were suspended (one site more than once), eight sites were reinstated after they provided a written plan to improve enrollment, and three sites were permanently suspended from further enrollment. Among the 2502 randomized participants, 1321 (53%) were symptomatic, 872 (34.9%) were female, and 234 (9.4%) were minority by race ethnicity or race-ethnicity.
Discussion
Our results suggest that increasing the number of clinical sites, establishing a full-time recruitment director, broadening study eligibility, and initiating multiple tactics to enhance enrollment were complementary and effective in achieving the required sample size of 2502 in CREST.
The expansion of approved sites from 40 to 119 was likely the most important tactic for successful enrollment. The original 40 sites were selected because they were thought to be the best with regard to quality and enrollment. The data (Table 2) suggest, however, that the addition of new sites never resulted in a decline in enrollment as judged by enrollment per site per month. The new sites were as productive as the original sites despite the rigorous site selection process in place in 2000. We suggest that the enthusiasm of new teams, the availability of a new treatment [CAS – reimbursable within CREST (14)], and the efforts and resources of the recruitment director and the CREST team combined for effective site expansion.
The effectiveness of a full-time recruitment director is also supported by the results (Table 2, Fig. 1). The pace of enrollment increased shortly after the recruitment director was hired, and increasing enrollment continued before eligibility of asymptomatic patients could be established study-wide. In addition, site selection and start-up were accelerated, as well as IRB approval and site implementation of recruitment tools. Ongoing daily feedback to the sites following an enrollment became feasible. In CREST, the model was to centralize supplementary personnel and mechanisms for recruitment. In other trials, the utility of where such efforts should be concentrated will depend on the design of the trial. For example, in the NINDS tissue plasminogen activator (tPA) Stroke trial (15), there were a smaller number of sites, eight, and so it was feasible (and effective) to have a recruitment-dedicated professional at each site. In a trial with larger numbers of sites, resources may be insufficient for the NINDS tPA model, and so the centralized model may be more effective.
Widening eligibility in CREST to include asymptomatic patients also drove enrollment. The principle argument for inclusion was to increase the generalizability of the CREST results, to inform clinical decision-making for asymptomatic patients. Adding asymptomatic patients was expected to increase enrollment, and a rise in enrollment would have occurred even without increasing the number of sites or hiring a recruitment director. However, we were encouraged in that enrollment for symptomatic patients did not decline and the rate of enrollment of the asymptomatic patients increased after 2005, suggesting that site expansion and the recruitment director efforts continued to add value.
Multiplicity in tactics and tools is likely more important than the best tactics and tools. Many of the recruitment interventions we report are not unique, having been used in other trials. Rather, we suggest that our accelerating enrollment was due in substantial proportion to the number of tactics and tools that could be exported to the sites. Patients and sites from CREST, and in other RCTs, are heterogeneous. Individual tactics and tools will vary in effectiveness so the more, the better. We also suggest that implementation of multiple mechanisms for recruitment requires full-time personnel in a large multicenter RCT. Even a simple patient brochure is surprisingly time-consuming in inception, design, regulatory approval, and distribution.
Repeated communication to the centers in a multicenter trial is essential. Not uncommonly, the individual centers are enrolling only one patient per month, or less. In that context, the personnel in the typical multishift clinic or hospital, and even some investigators, can forget enrollment criteria or that a particular trial is enrolling. The impression of the recruitment director, the CREST PI and co-PI, and other team members was that repetition was exceptionally effective. The message is also important. Recognizing individual investigators and coordinators at the sites for a job well done is not soon forgotten, particularly when that recognition is timely. Soliciting feedback from the sites, which can be followed by actions from study leadership, cements strong relationships. In CREST, protocol amendments were instituted, which were based upon feedback from the clinical sites, including changes to case report forms, follow-up visit schedules, and types of visits. Confidence that their concerns were being heard and acted upon allowed for candid discussions with investigators and coordinators regarding site-specific recruitment challenges and initiatives to improve enrollment.
Large RCTs commonly experience slow enrollment or underenrollment. A review of 114 trials reported that less than one-third were successful in obtaining targeted enrollment within the original prespecified enrollment period, and about one-third required extensions (16). Characteristics of the study site, whether academic or community-based, will often dictate the appropriate resources for effective enrollment (17). In CREST, sustaining an average yearly randomization rate of 607 over the last four-years of the enrollment period attenuated the negative budgetary consequences of enrollment extending beyond mid-2008. More importantly, achieving the prespecified sample size maintained the scientific integrity of the trial. In so doing, successful enrollment also fulfilled the ethical obligations to the patients and to their family members for the risks and inconveniences of participation.
Of note, for trial design and peer review of grant proposals, the original estimates of site productivity provided by CREST leadership based upon the site estimates were inaccurate. For example, the annual rate of enrollment per site per year was projected to be 25 with 40 enrolling centers. The best annual rate of enrollment per site per year ever sustained for CREST was 6.6 with 103 active sites, only 26% of the predicted rate (Table 2). We hypothesize that overestimating enrollment is a common limitation of RCT proposals to NIH and other funding institutions. The investigators initiating the proposals are likely to anticipate better peer review if they incorporate fewer sites, shorter enrollment periods, and less expensive total costs. Potential sites are likely to overestimate as well. The site investigators wish to be selected, particularly if participation is tied to CMS and other third-party reimbursement. Our results suggest that critical examination of recruitment methodology and capability should receive greater prioritization at peer review of large multicenter RCT proposals.
It should be noted that in CREST, data regarding site-level or study-wide variables that may have affected recruitment were not systematically collected (e.g. turnover of approved personnel, temporary shutdown of enrollment due to technical problems, availability of certified physicians to perform the procedure, times for IRB approval of protocol changes, and recruitment materials). Screening logs were maintained early, but quality varied, and so the logs were not prioritized. Implementing CMS reimbursement varied considerably by region, state, and hospital. The dates for implementation of the inclusion of asymptomatic patients also varied considerably by site.
Conclusions
Increasing the number of enrolling centers, adding a recruitment director, widening eligibility, and expanding recruitment tactics were the drivers for achieving the prespecified sample size in CREST. From 1977 to 2007, the NINDS at the NIH funded 28 phase 3 stroke trials randomizing over 44 000 participants (18). Future trials may require greater numbers of participants (18). Scientific, ethical, and financial success will depend upon improved strategies and tactics to maximize enrollment into these RCTs.