Quantifying heterogeneity in a meta‐analysis

TLDR

Heterogeneity in meta‑analyses, quantified by between‑study variance, complicates overall conclusions and is traditionally assessed by tests that depend on the number of studies. The authors develop measures of the impact of heterogeneity that are independent of study number and effect metric. They derive three statistics—H (the square root of χ² divided by its degrees of freedom), R (the ratio of the random‑effects to fixed‑effects standard error), and I² (a transformation of H that represents the proportion of total variation due to heterogeneity). The study concludes that H and I² are particularly useful and should be reported in place of the traditional heterogeneity test. © 2002 John Wiley & Sons, Ltd.

Abstract

Abstract The extent of heterogeneity in a meta‐analysis partly determines the difficulty in drawing overall conclusions. This extent may be measured by estimating a between‐study variance, but interpretation is then specific to a particular treatment effect metric. A test for the existence of heterogeneity exists, but depends on the number of studies in the meta‐analysis. We develop measures of the impact of heterogeneity on a meta‐analysis, from mathematical criteria, that are independent of the number of studies and the treatment effect metric. We derive and propose three suitable statistics: H is the square root of the χ 2 heterogeneity statistic divided by its degrees of freedom; R is the ratio of the standard error of the underlying mean from a random effects meta‐analysis to the standard error of a fixed effect meta‐analytic estimate, and I 2 is a transformation of H that describes the proportion of total variation in study estimates that is due to heterogeneity. We discuss interpretation, interval estimates and other properties of these measures and examine them in five example data sets showing different amounts of heterogeneity. We conclude that H and I 2 , which can usually be calculated for published meta‐analyses, are particularly useful summaries of the impact of heterogeneity. One or both should be presented in published meta‐analyses in preference to the test for heterogeneity. Copyright © 2002 John Wiley & Sons, Ltd.