Challenges in meta-analyses with observational studies

TLDR

Meta‑analyses of observational studies are common but are viewed as less reliable than those of RCTs because of higher bias risk and greater heterogeneity. This article aims to identify and recommend careful considerations for meta‑analyses of observational studies compared to RCT‑based meta‑analyses. The authors review quantitative synthesis issues, including heterogeneity and bias handling, and describe advanced synthesis methods that allow more flexible modelling than conventional meta‑analysis models. Using a mental‑health example of antipsychotic users, the authors show that heterogeneity and short‑exposure studies can inflate myocardial‑infarction risk estimates, underscoring the need to verify that included studies address the same clinical question before synthesis.

Abstract

Meta-analyses of observational studies are frequently published in the literature, but they are generally considered suboptimal to those involving randomised controlled trials (RCTs) only. This is due to the increased risk of biases that observational studies may entail as well as because of the high heterogeneity that might be present. In this article, we highlight aspects of meta-analyses with observational studies that need more careful consideration in comparison to meta-analyses of RCTs.We present an overview of recommendations from the literature with respect to how the different steps of a meta-analysis involving observational studies should be comprehensively conducted. We focus more on issues arising at the step of the quantitative synthesis, in terms of handling heterogeneity and biases. We briefly describe some sophisticated synthesis methods, which may allow for more flexible modelling approaches than common meta-analysis models. We illustrate the issues encountered in the presence of observational studies using an example from mental health, which assesses the risk of myocardial infarction in antipsychotic drug users.The increased heterogeneity observed among studies challenges the interpretation of the diamond, while the inclusion of short exposure studies may lead to an exaggerated risk for myocardial infarction in this population.In the presence of observational study designs, prior to synthesis, investigators should carefully consider whether all studies at hand are able to answer the same clinical question. The potential for a quantitative synthesis should be guided through examination of the amount of clinical and methodological heterogeneity and assessment of possible biases.