Good intentions and ICH-GCP: Trial conduct training needs to go beyond the ICH-GCP document and include the intention-to-treat principle

Abstract

Background:

It is assumed investigators and statisticians fully understand the importance of avoiding missing outcomes and the intention-to-treat principle during design and analysis phases of a randomised controlled trial in order to obtain the most valuable and reliable results. However, many personnel undertaking day-to-day trial conduct and data collection commonly rely exclusively for guidance on the widely implemented, indeed regulated, International Conference on Harmonisation–Good Clinical Practice document as the guideline and standard for trial conduct.

Purpose:

This article describes adverse consequences of omission of intention-to-treat principles from training for trial personnel and explores the need for training in addition to the International Conference on Harmonisation–Good Clinical Practice guideline document.

Methods:

Data from the Breast Boost Study were used to illustrate a comparison of actual results, where vigilant senior investigators re-enforced intention-to-treat requirements throughout all aspects of trial conduct with results that could easily have occurred if study personnel did not understand the importance of intention-to-treat principles. Experience as a co-ordinating centre for an international trial (Trans-Tasman Radiation Oncology Group 08.06 Breast STARS) acted as an audit of data-management culture regarding intention-to-treat in Australia and New Zealand.

Results:

Despite the Breast Boost Study exceeding planned accrual, it was demonstrated that the study, which found a statistically significant result, could have reported a negative or inconclusive result under the scenario of trial conduct personnel having lack of understanding of the importance of avoiding losses to follow-up. Trans-Tasman Radiation Oncology 08.06 co-ordination experience verified that data-management culture in Australia and New Zealand does not adequately recognise intention-to-treat principles, and this is reflected in trial conduct.

Limitations:

Trial data described are limited to two trials and in the Australian and New Zealand setting.

Conclusion:

To be both scientifically and ethically valid, guidelines for trial conduct should include and stress the importance of the intention-to-treat principle and in particular avoiding missing outcomes. Our discussion highlights the vitally important role played by personnel involved in day-to-day trial conduct. Inclusion of scientific principles in guideline documents and/or training which goes beyond International Conference on Harmonisation–Good Clinical Practice to include intention-to-treat is essential to achieve robust research results. Related aspects of randomised trial consent and ethics are discussed.

Keywords

Intention-to-treat ITT follow-up missing outcome guidelines GCP good clinical practice clinical trials regulation loss to follow-up

Introduction

It is important to assure the results of randomised clinical trials are valid by avoiding missing outcomes. The costs involved with running randomised trials are large and increasing,^1,2 and a goal of obtaining unbiased results is an ethical responsibility to the patients who agree to allow their treatment allocation to be randomised.

The special strength of the randomised trial is avoiding bias by using random assignment of treatments. The preferred analysis strategy is intention-to-treat (ITT), defined as including all randomised patients according to allocated treatment regardless of compliance.³ Bias can be introduced during study conduct by missing outcomes.⁴ All other consequences of missing outcomes are adverse and include loss of power,⁵ inconclusive results,⁶ which can potentially preclude further studies,^7,8 and under-reporting of treatment-related toxicities.⁹ Including additional subjects in the study does not resolve bias and does not always resolve other problems.¹⁰

The gold standard of trial conduct, therefore, is to avoid missing outcomes altogether. It is understood that some loss to follow-up cannot always be avoided, and where this is significant, a number of approaches are suggested, for example, outcome imputation, inverse probability weighting and/or sensitivity analyses.^6,10–12 For all such options, assumptions are inevitably involved which can be controversial and may themselves introduce bias.¹³ There is no universally accepted method, and Altman⁵ suggests exploring different approaches. A recent statistical note by Vickers and Altman¹⁴ concludes, ‘Above all, analysis of missing data teaches us the importance of avoiding missing data in the first place: an informed guess, even using a technique as sophisticated as multiple imputation is still a guess’.

Loss to follow-up is not just a hypothetical possibility but is recognised as a serious pervasive problem.¹⁵ A systematic review conservatively estimated that under plausible assumptions regarding outcomes of losses to follow-up, the results reported in top medical journals may no longer be statistically significant for possibly one-third of trials.¹⁶

Investigators and statisticians involved with randomised trial design and analysis will generally understand ITT concepts, but what may be overlooked is that those involved with trial conduct and patient contact include a large number of other health professionals (e.g. data managers, trial co-ordinators, registrars, staff specialists and others). We suggest that many day-to-day trial conduct personnel do not realise the importance of ITT concepts or may not even be aware of them and that this impacts the extent of missing outcomes. This is illustrated using data from two studies where the St George Cancer Care Centre Clinical Trials Unit provides the central trial co-ordination supervised by author P.H.G.: the Breast Boost Study and a Trans-Tasman Radiation Oncology (TROG) 08.06 trial. We assess and discuss whether the current approach to training the entire randomised trial enterprise using only the Good Clinical Practice document E6 (International Conference on Harmonisation–Good Clinical Practice (ICH-GCP)) is adequate with respect to ITT concepts, or whether this could be part of the problem.¹⁷

Methods

St George Breast Boost Study

This randomised trial assessed the value of a boost dose of radiotherapy with a reduced whole breast dose for local control in breast conservation for in situ and invasive breast cancer.¹⁸ The study was primarily conducted at St George Cancer Care Centre with a small recruitment of patients from two other local centres. The primary outcome of the study was time to local recurrence of breast cancer. Patients were requested to undertake clinical examination and complete a Quality of Life Questionnaire (QoL) annually until at least year 10. Accrual commenced in 1996, but follow-up for the primary endpoint is continuing long term, currently ranging from 9 to 17 years.

The data presented are summaries of numbers of local recurrence events, time at risk, log-rank test results and hazard ratios from the Cox proportional hazard model. Analysis of actual trial data is presented, and analyses were also undertaken based on hypothetical scenarios which simulate withdrawal from the study (at which time the patient was censored) as a result of the patient’s non-compliance regarding completion of questionnaires. The analysis was undertaken using STATA 11.

TROG 08.06 Study Co-Ordinating Centre experience

The TROG 08.06 Study is centrally co-ordinated by St George Cancer Care Centre and conducted at 24 centres in all states of Australia and also in New Zealand. The study provides a randomised comparison of an aromatase inhibitor commenced prior to or after radiotherapy. Follow-up and primary endpoints for this study are similar to the Breast Boost Study. Our experiences with follow-up are described.

Results

St George Breast Boost Study

In brief, 688 patients were recruited, 564 at St George Cancer Care and 124 from two other centres. During long-term follow-up, a growing number of patients indicated a request to be free of study requirements, often stating that they wished to be taken out of the study. The simple response to this request by study personnel, based on ICH-GCP training, was to withdraw patients and cease follow-up. Following some such events, investigators emphasised ITT principles and instructed all study personnel to invest considerable time and effort towards maintaining contact with patients in order to maintain access for at least the primary outcome data, local recurrence. Following discussion, almost all patients who had initially requested withdrawal were in agreement with a modified contact involving annual phone calls. Commonly, these patients had decided they would no longer complete QoLs. However, only 3 of 688 patients were unavoidably lost to follow-up for local recurrence.

Table 1, row ‘a’ presents the analysis of local recurrence as a result of full follow-up on all but three patients. The hazard ratio is 1.90 (95% confidence interval (CI) = 1.0–3.6, p = 0.049).¹⁹ Table 1, rows ‘b’ and ‘c’ show analyses based on two illustrative scenarios of losing outcome data, that is, patients were censored at the time the patient last completed a questionnaire or 12 months thereafter, respectively. The comparisons between treatments were no longer statistically significant, and had these scenarios occurred, the study would have demonstrated either no difference between treatments or inconclusive findings. These scenarios are intended as simple illustrations of what could easily have occurred under standard trial conduct training that does not emphasise the importance of follow-up to the validity of trial results.

Table 1.

An illustration of local control results.

	Boost (n = 342)			No boost (n = 346)			Total (n = 688)			Analysis results
	Events	Time at risk (years)	Median follow-up (years)	Events	Time at risk (years)	Median follow-up (years)	Events	Time at risk (years)	Median follow-up (years)	Log rank	p-value	Hazard ratio (95% CI)
a.	26	2871	8.7	14	2913	8.9	40	5784	8.8	3.86	0.049	1.90 (1.0–3.6)
b.	17	2296	7.0	10	2265	6.9	27	4561	7.0	1.94	0.16	1.73 (0.79–3.78)
c.	22	2627	8.0	12	2590	8.0	24	5217	8.0	3.14	0.08	1.87 (0.93–3.8)

CI: confidence interval; QoL: Quality of Life Questionnaire.

(a) full data collection with n = 3 lost to follow-up; (b) patients censored at the time last QoL completed; and (c) patients censored 1 year after last QoL completed.

The results under these scenarios indicate that questionnaire compliance may not have contributed bias, and results do not demonstrate an appreciable difference in QoL compliance between treatment groups. This is suggested by Figure 1 which also demonstrates how easily large numbers of patients can be lost to follow-up without diligence and effort by study personnel.

TROG 08.06 Study Co-Ordinating Centre experience

Opening in September 2009, 820 patients had been recruited as of 30 June 2013. So far, there have been 18 cases where protocol treatment variations have occurred. For many of these, instruction from us as the central co-ordinating centre to the local participating centre has been required to attempt to avoid or remedy inappropriate discontinuation of follow-up. Withdrawal of patients from the study at more than half of the centres has occurred despite our trial protocol and other materials outlining goals of continued follow-up.

Discussion

Trial conduct guidelines and training

ICH-GCP is the regulatory standard for trial conduct in Australia and is the basis on which conduct monitoring and audits may be undertaken.²⁰ For pharmacological trials, ICH-GCP is generally the only trial conduct training personnel are required to undertake. TROG trial agreements with participating centres include a requirement that trials are conducted in compliance with ICH-GCP and annual scientific meetings include management workshops with ICH-GCP as a main focus. In our experience, trial personnel describe ICH-GCP as the bible of trial conduct and can hold the view that conduct is required to conform only to ICH-GCP and the trial protocol. This restricted view is re-enforced by the commercial ICH-GCP training industry which trains for ICH-GCP compliance but not beyond.

The ICH-GCP document does not describe ITT concepts nor does it mention the importance of avoiding missing outcomes. With respect to measures to minimise bias, the ICH-GCP lists randomisation and blinding, but does not mention outcome retention. It is of concern that ICH-GCP has been interpreted as not only mute on ITT but as being quite permissive with respect to withdrawals. Van der Weijden et al.²¹ refer to ‘… the discrepancy between the ease with which GCP accepts drop-outs and the rigidness with which ITT tries to keep them in …’. In our experience, ICH-GCP has been interpreted as not only allowing withdrawals but endorsing or even encouraging withdrawals for minor reasons expressed by patients. We have also heard statements along the lines of ‘common sense’, suggesting that where a patient does not comply with study treatment, their outcome data could not be useful anyway, indicating that ITT can be considered as counter-intuitive unless specific training is given. Under ICH-GCP, we understand that monitoring of withdrawals would involve checking completion of correct documentation but would not critically assess actions taken or not taken in an attempt to avoid loss to follow-up.

While all agree that the overall goals of the ICH-GCP are excellent, numerous papers in the scientific literature and international symposiums have outlined a broad range of problems and suggestions for amendments.^22–25 It has been suggested previously that it may be harmful to trial conduct that a number of scientific principles, including ITT, are not described in the ICH-GCP.²³ One of the goals of the ICH-GCP was to ensure the protection of rights of consenting patients. ICH-GCP emphasises ethical conduct towards participants, and the guideline in this respect has been instrumental worldwide towards protecting the rights of individual patients, particularly with respect to consent and treatment. With respect to omission of ITT, the ICH-GCP does not fully encompass the overall ethical responsibility owed to all consenting patients in achieving the best study results. Nowhere in a consent form do we declare that the study may lead to biased or inconclusive results.

The important scientific knowledge that can be gained from randomised trials is the reason society accepts treatment randomisation and is why we can ethically ask for consent. In most cases, it would also be the reason that patients agree to consent, yet ICH-GCP does not identify the patient’s right to be informed that withdrawal of access to outcome data will adversely affect this scientific knowledge. Our experience is that most patients, given such information, will agree to continued study team access to outcome data. We hold the view that it would be unethical to withhold this information from patients as this may lead to unnecessary missing outcomes and possibly biased or inconclusive study results. Ultimately, this could essentially amount to asking a patient’s consent to randomisation for no purpose at best, or possibly erroneous results at worst.

Those making treatment decisions based on results for studies described as randomised are likely to assume that the results are ‘unbiased’ due to the study design itself. A study design may be excellent and minimise intrinsic bias, and investigators and statisticians can report with honesty and full disclosure, but can only achieve unbiased results if all personnel involved with patient contact and generating the data are trained to incorporate ITT into their daily trial conduct. The goal should be full appreciation by all trial personnel of the importance of pursuing all outcome data in the same way that there is appreciation of the importance of obtaining consent for all patients and collecting all baseline data.

In our experience, despite study materials encouraging continuation of follow-up, various trial personnel continue to make statements and hold views that are inconsistent with ITT concepts. We conclude that there is an endemic flaw in trial conduct training if ICH-GCP is considered the comprehensive guideline document around which training programmes are designed.

Trial conduct training wider than ICH-GCP

The reason that regulators have not looked further than the ICH-GCP could be found in the ‘Introduction’ where it is stated that ICH-GCP is an ‘international ethical and scientific quality standard’.¹⁷ In our view, to be scientifically and ethically valid, a guideline document should describe ITT concepts, and the best solution would be to amend mandated guideline documents. If this cannot be achieved, regulators need to become aware that ICH-GCP is not a comprehensive document and look to include and regulate additional material for trial conduct.

The ICH website contains about 100 guideline documents with approximately 22 classified as Efficacy. Another efficacy document, E9, does mention ITT concepts in the context of statistical analysis.²⁶ ICH-GCP was written in 1996 and does not reference E9 which was written in 1998, and in practice, trial conduct training does not include other ICH documents. With respect to personnel accepting that ITT concepts are important, it may be helpful if parts of E9 are included in required training for trial conduct.

The missing outcome problem is receiving considerable concern in the research community generally. Recent papers include numerous practical suggestions to help reduce the problem,^10,11,15,27 and many directly target the issue we discuss, for example, educating patients as part of informed consent about the importance of not withdrawing, ensuring that the protocol addresses avoidance of missing outcome data and trial staff training. This literature may be read by researchers, but we suggest this is unlikely for trial personnel or regulators in some countries. We would hope that these suggestions will filter down into trial conduct in the future. Our perspective is similar to that of the United States Food and Drug Administration that bringing about needed changes in the way trials are conducted and managed will not be easy.¹⁵ They also suggest that a cultural shift will be required and that implementing any of their 18 recommendations will take time and extensive consciousness raising.

Right now for studies underway, trial personnel need to be empowered with a full understanding of the scientific principles that underpin randomised trials and their importance and relevance to day-to-day trial conduct. While this may seem obvious, in our experience, this additional training is not easy to achieve. Trial centres commonly see no need for something that is not mentioned in any regulation. Presenting ITT concepts to highly trained and professional trial conduct personnel can be challenging. There is recognition that ICH-GCP is prescriptive with respect to many aspects of trial conduct, and personnel can be resistant to accepting that a concept like ITT is important given that it is not even mentioned in ICH-GCP. We have experienced comments along the lines of ‘ITT cannot be correct, or it would have been included in ICH-GCP’ or ‘ITT is not consistent with statements in ICH-GCP about patient withdrawal’. In the present regulatory environment, ITT can be viewed as ‘interesting’, but it will not be considered as seriously as training in ICH-GCP. ICH-GCP can be used to argue against ITT concepts, and wherever a conflict may be interpreted, ICH-GCP will prevail. An example would be defining good trial conduct when a patient is considering withdrawal from follow-up or data access. It is recommended that patients have a right to informed withdrawal of consent so that they understand that withdrawal adversely affects study results and scientific knowledge.²⁸ If trained only in ICH-GCP, trial personnel are likely to incorrectly view this as inconsistent with the right to withdraw at any time or that patient’s rights are more important than the interests of science and society. Ultimately, even with increasing awareness of the missing outcome problem, we see the need for regulations to mandate training wider than ICH-GCP and to define specific training and conduct aimed at minimising missing outcomes.

Ethics of consent, confidentiality and ITT

This discussion raises another related aspect of trial conduct associated with full outcome data collection – confidentiality. Expressions of concern appear widespread relating to the potential for an excessive view of patient privacy to cause barriers to research projects.²⁹ This issue relates to all medical research including large-scale data collection important to epidemiological evidence. For randomised trials, it is important to consider that all patients have given specific consent. While these patients must be free to withdraw from study treatment, invasive endpoint assessments and so on, the ethics of patients withdrawing consent for study team to access outcome data that would be contained in medical records is a sensitive topic but one worthy of discussion.

The study team and the patient need to consider their ethical responsibilities to all the patients who have consented to take part in the research project. All have consented in good faith that the study has been designed, approved and conducted in a manner to give a result which answers the research question with scientific rigour.^30,31 We reiterate that nowhere in a consent form do we declare that the study may lead to biased or inconclusive results. Does prohibiting study access to some of the outcome data amount to loss of an ethical vigilance owed to the majority of randomised patients because of perhaps unreasonable ethical vigilance towards confidentiality for the few?

It has been suggested that trial participants should be educated about the importance of continued engagement in the trial and encouraged to continue to provide outcome information or access to records regardless of treatment status.²⁸ Is it reasonable to include in the consenting process an understanding that data for at least the primary outcome of the study will always be available to the study team? There are many studies where data for primary outcomes would form part of basic medical records. This would commonly be the case for oncology studies where cancer outcomes and survival are the primary endpoints. Perhaps the research community and ethics committees need to consider allowing study team to access data regarding the primary endpoint for all patients who initially consent to be randomised, provided it does not require non-routine or invasive investigations. If so, does the consent need to be explicit?

Conclusion

Application of ITT concepts minimises losses to follow-up but is not intuitive to many trial personnel and not included in ICH-GCP. Trial conduct guidelines and training must not be limited to ICH-GCP but include other good scientific principles. The impact of missing data and the importance of the ITT principle should be included as required training for trial personnel together with procedures for adapting trial conduct to maintain primary endpoint data collection where possible. We raise the question of how individual rights should be considered ethically in the context of ethical obligations to other study participants and the community with respect to access to study outcome data.

Footnotes

Acknowledgements

Trial registrations: A Randomised Comparison of Breast Conservation with or without Lumpectomy Radiotherapy Boost (Breast Boost Study): NCT00138814, and TROG 08.06 Breast Stars Trial: NCT00887380, www.clinicaltrials.gov.

Declaration of conflicting interests

The authors declare that there is no conflict of interest.

Funding

The TROG 08.06 study is supported by a Cancer Australia Grant (project grant 626201) and funding from AstraZeneca (study code D5390L00067). Trial conduct and analysis are independent of these funding sources.

References

Kramer

Schulman

. Transforming the economics of clinical trials. Discussion paper, Institute of Medicine of the National Academies, Duke University Medical Centre, April 2012, http://www.iom.edu/~/media/Files/Perspectives-Files/2012/Discussion-Papers/HSP-Drugs-Transforming-the-Economics.pdf (accessed 1 July 2014).

European Forum for Good Clinical Practice (EFGCP). Impact on clinical research of European legislation. Grant Agreement Number. HEALTH-F1-2007-201002, June 2009. Belgium: EFGCP.

Heritier

Gebski

Keech

. Inclusion of patients in clinical trial analysis: the intention-to-treat principle. Med J Aust 2003; 179: 438–440.

Moher

Hopewell

Schulz

. CONSORT 2010 Explanation and Elaboration: updated guidelines for reporting parallel group randomised trials. BMJ 2010; 340: c869.

Altman

. Missing outcomes in randomized trials: addressing the dilemma. Open Med 2009; 3(2): 51–53.

Cakir

Gebski

Keech

. Flow of participants in randomised studies. Med J Aust 2003; 178: 347–349.

Campbell

. Doing clinical trials large enough to achieve adequate reductions in uncertainties about treatment effects. JLL Bulletin: commentaries on the history of treatment evaluation, 2012, www.jameslindlibrary.org (accessed 1 July 2014).

Freiman

Chalmers

Smith

. The importance of beta, the Type II error and sample size in the design and interpretation of the randomised control trial. N Engl J Med 1978; 299(13): 690–694.

Madigan

Sigelman

Mayer

. Under-reporting of cardiovascular events in the rofecoxib Alzheimer disease studies. Am Heart J 2012; 164(2): 186–193.

10.

Little

D’Agostino

Cohen

. The prevention and treatment of missing data in clinical trials. N Engl J Med 2012; 367(14): 1355–1360.

11.

Mallinckrodt

Roger

Chuang-Stein

. Missing data: turning guidance into action. Stat Biopharm Res 2013; 5(4): 369–382.

12.

Seaman

White

. Review of inverse probability weighting for dealing with missing data. Stat Methods Med Res 2011; 22(3): 278–295.

13.

European Medicines Agency. Guideline on missing data in confirmatory clinical trials. Doc. Ref CPMP/EWP/1776/99, April 2009. London: European Medicines Agency.

14.

Vickers

Altman

. Statistics notes: missing outcomes in randomised trials. BMJ 2013; 346: f3438.

15.

O’Neill

Temple

. The prevention and treatment of missing data in clinical trials: an FDA perspective on the importance of dealing with it. Clin Pharmacol Ther 2012; 91(3): 550–554.

16.

Akl

Briel

You

. Potential impact on estimated treatment effects of information lost to follow-up in randomised controlled trials (LOST-IT): systematic review. BMJ 2012; 344: e2809.

17.

ICH Harmonised Tripartite Guideline. Guideline for good clinical practice E6(R1). June 1996, http://www.ich.org/fileadmin/Public_Web_Site/ICH_Products/Guidelines/Efficacy/E6_R1/Step4/E6_R1__Guideline.pdf (accessed 1 July 2014).

18.

Millar

EKA

Graham

O’Toole

. Prediction of local recurrence, distant metastases and death after breast-conserving therapy in early-stage invasive breast cancer using a five-biomarker panel. J Clin Oncol 2009; 27(28): 4701–4708.

19.

Graham

Millar

Browne

. Randomized trial shows reduced whole breast dose negates benefit of lumpectomy radiotherapy boost. In: Proceedings of the 29th ESTRO meeting, Barcelona, 12–16 September 2010 (Radioth Oncol 2010; 96(Suppl. 1): S145, oral presentation 383).

20.

Therapeutic Goods Administration. The Australian clinical trial handbook. Australian Government, March 2006, http://www.tga.gov.au/pdf/clinical-trials-handbook.pdf (accessed 1 July 2014).

21.

Van der Weijden

Slot

Rosema

. ITT in respect to GCP: a matter of diligence. J Clin Periodontol 2008; 35: 681–682.

22.

Grimes

Hubacher

Nanda

. The Good Clinical Practice guideline: a bronze standard for clinical research. Lancet 2005; 366: 172–174.

23.

Yusuf

. Damage to important clinical trials by over-regulation. Clin Trials 2010; 7: 622–625.

24.

McMahon

Conway

MacDonald

. The unintended consequences of clinical trials regulations. PLoS Med 2009; 6(11): e1000131.

25.

Sensible Guidelines Symposium. http://www.cannectin.ca/default.cfm?id=136 (2012, accessed 6 September 2013).

26.

ICH Harmonised Tripartite Guideline. Statistical principles for clinical trials E9. February 1998, http://www.ich.org/fileadmin/Public_Web_Site/ICH_Products/Guidelines/Efficacy/E9/Step4/E9_Guideline.pdf (accessed 1 July 2014).

27.

Little

Cohen

Dickersin

. The design and conduct of clinical trials to limit missing data. Stat Med 2012; 31: 3433–3443.

28.

National Research Council of the National Academies. The prevention and treatment of missing data in clinical trials. Washington, DC: National Academies Press, 2010, pp. 39–45.

29.

Wartenberg

Thompson

. Privacy versus public health: the impact of current confidentiality rules. Am J Public Health 2010; 100(3): 407–412.

30.

Australian Government and National Health and Medical Research Council (NHMRC). National statement on ethical conduct in human research (Updated May 2013). Canberra, ACT, Australia: NHMRC, 2007.

31.

Australian Government and National Health and Medical Research Council (NHMRC). Australian code for the responsible conduct of research. Canberra, ACT, Australia: NHMRC, 2007.