Using natural experiments to evaluate population health interventions: new Medical Research Council guidance

TLDR

Natural experimental studies are recommended for assessing the health impact of policies and large‑scale interventions, offering advantages over planned experiments but also being more prone to bias, and future experience may broaden their use. This paper introduces new Medical Research Council guidance to help researchers, users, funders, and publishers optimally employ natural experimental approaches for evaluating population health interventions. The guidance stresses the importance of understanding exposure processes, carefully selecting and combining methods, testing assumptions, and ensuring transparent reporting. It highlights that natural experiments can yield convincing evidence of impact even when effects are small or delayed.

Abstract

Natural experimental studies are often recommended as a way of understanding the health impact of policies and other large scale interventions. Although they have certain advantages over planned experiments, and may be the only option when it is impossible to manipulate exposure to the intervention, natural experimental studies are more susceptible to bias. This paper introduces new guidance from the Medical Research Council to help researchers and users, funders and publishers of research evidence make the best use of natural experimental approaches to evaluating population health interventions. The guidance emphasises that natural experiments can provide convincing evidence of impact even when effects are small or take time to appear. However, a good understanding is needed of the process determining exposure to the intervention, and careful choice and combination of methods, testing of assumptions and transparent reporting is vital. More could be learnt from natural experiments in future as experience of promising but lesser used methods accumulates.