Abstract
I read with great interest the meta-analysis of early versus late tracheostomy by Liu et al. 1 I believe this article highlights a serious issue with meta-analysis.
Meta-analyses are widely regarded as the highest standard for providing evidence and has indeed contributed many important, cumulative effects of research studies (eg, the relationship between infant sleeping position and sudden infant death syndrome). 2 They aim to provide an objective summary of the outcome of multiple tests of the same intervention or treatment.
However, they also have the power to magnify error to a level of significance and be misleading. As the number of studies included in a meta-analysis increases, the influence of a type II error increases and therefore false rejection of a null hypothesis becomes a serious risk.3,4 This risk increases by massing studies of high methodological heterogeneity, with variable aims and data completeness. Thus, the objectivity of the meta-analysis is vulnerable.
Liu et al 1 report the conclusion that the cumulative result of all 11 studies is that early tracheostomy is associated with a decrease in intensive care unit (ICU) length of stay. The largest, best-designed studies do not concur.5,6 The largest effect size, suggesting ICU stay is shortened in association with early tracheostomy, is produced by much smaller studies, which are less robustly designed and considerably underpowered. By including all of these studies, the accurate answer is at risk of being drowned out by background “noise.”
In this situation, Slavin’s principle of “best evidence” must be remembered when considering conclusions drawn from meta-analyses. 7 Here, surely the best evidence included in the meta-analysis clearly shows no decrease in ICU length of stay?
Disclosures
Footnotes
Sponsorships or competing interests that may be relevant to content are disclosed at the end of this article.