Abstract
To the Editors:
We read with great interest the excellent article by Antoniou et al, 1 describing the detailed steps for conducting a systematic review and meta-analysis. In this article, the authors provide a ten-step assessment of guiding principles to produce high-quality evidence-based medicine and to improve clinical decision making through systematic reviews of the effects of health care interventions. However, we want to contribute an addition to their study highlighting the importance of reporting the prediction interval (PI) in meta-analyses, an essential tool for interpreting evidence, further aiding in clinical practice. 2 PI is an important parameter in data analysis when assessing inconsistency among studies, presenting the heterogeneity on the same scale as the original outcomes compared to other tools such as I2 or τ 2 (also called the inconsistency measure and the among-study variance, respectively). 3 Furthermore, a confidence interval (CI) is inadequate for clinical decision making because it only summarizes the average effect for the average study and thus may lead to different conclusions, while PI estimates what true treatment effect can be expected in future studies. Interestingly, IntHoute and Ioannidis 2 reported that implementing PI in >400 published meta-analyses may have led to completely opposite effects in over 20% of them.
In the study of Antoniou et al, 1 the forest plot depicted in their Figure 4B represents a continuous outcome given as the mean ± standard deviation and the mean difference from a meta-analysis showing that there is marginally no statistical significance of the results. Implementation of the PI in the forest plot using STATA (v.13, College Station, TX, USA) shows that there is a wide variation of the treatment effect, therefore the results are not marginally but widely nonsignificant (Figure 1). Further enhancing its value, PI provides a predicted range for the true effect size in a new study and is especially accurate when heterogeneity is large (I2>30%), as in the forest plot of Antoniou et al (Figure 4B, I2=65%).
Implementation of the prediction interval (PI) in the forest plot. Diamond represents the pooled effect estimate (mean± CI) and the horizontal lines on both sides represent the width of the PI (CI: −8.18 to 1.61, PI: −22.85 to 16.28).
Regarding assessing quality, Antoniou et al 1 focused on the assessment of the constituent primary studies. They included an informative and detailed table with all the relevant assessment tools for all the different types of clinical studies. However, these kinds of tools are used to grade the studies that are used in a given meta-analysis and not for the critical appraisal of the systematic review itself, assessing parameters as the depth of the literature search, if the cause of included and excluded studies was adequately explained and if the methods used to combine the findings of the primary studies were appropriate. There are such tools designed to assess the quality of meta-analyses and systematic reviews as the AMSTAR score for use in meta-analyses including only randomized controlled trials and the AMSTAR-2 for meta-analyses including and nonrandomized studies.4,5
In conclusion, CI and PI convey different but complementary information. CI estimates the precision of the mean and indicates where the mean effect is likely to be. The 95% PI means that in 95% of cases the true effect of a new and unique study will fall within the PI values. PI is particularly helpful when excess heterogeneity exists and the combination of individual studies into a meta-analysis would not be advisable.
