Statistical Primer: heterogeneity, random- or fixed-effects model analyses?†

Abstract

Heterogeneity in meta-analysis describes differences in treatment effects between trials that exceed those we may expect through chance alone. Accounting for heterogeneity drives different statistical methods for summarizing data and, if heterogeneity is anticipated, a random-effects model will be preferred to the fixed-effects model. Random-effects models assume that there may be different underlying true effects estimated in each trial which are distributed about an overall mean. The confidence intervals (CIs) around the mean include both within-study and between-study components of variance (uncertainty). Summary effects provide an estimation of the average treatment effect, and the CI depicts the uncertainty around this estimate. There are 5 statistics that are computed to identify and quantify heterogeneity. They have different meaning and give complementary information: Q statistic and its P-value simply test whether effect sizes depart from homogeneity, T2 and T quantify the amount of heterogeneity, and I2 expresses the proportion of dispersion due to heterogeneity. The point estimate and CIs for random-effects models describe the practical implications of the observed heterogeneity and may usefully be contrasted with the fixed-effects estimates.