Assessing heterogeneity in meta-analysis: Q statistic or I² index?

TLDR

In meta‑analysis, the Q test assesses homogeneity but only indicates presence or absence, whereas the I² index has been proposed to quantify the extent of heterogeneity. The study compares the performance of the Q test and the confidence interval of the I² index using Monte Carlo simulation. The authors conduct a Monte Carlo simulation to evaluate the two approaches. The simulation demonstrates that the I² index complements the Q test, but both suffer from low power when few studies are available.

Abstract

In meta-analysis, the usual way of assessing whether a set of single studies is homogeneous is by means of the Q test. However, the Q test only informs meta-analysts about the presence versus the absence of heterogeneity, but it does not report on the extent of such heterogeneity. Recently, the I(2) index has been proposed to quantify the degree of heterogeneity in a meta-analysis. In this article, the performances of the Q test and the confidence interval around the I(2) index are compared by means of a Monte Carlo simulation. The results show the utility of the I(2) index as a complement to the Q test, although it has the same problems of power with a small number of studies.