Enough Already about “Black Box” Experiments: Studying Mediation Is More Difficult than Most Scholars Suppose

TLDR

The question of how causal effects are transmitted is fascinating and arises whenever experiments are presented, yet social scientists often underestimate the difficulty of establishing causal pathways rigorously. The authors argue that current statistical methods for mediation are flawed and that even sophisticated experimental designs cannot address mediation without strong assumptions, and that mediation research requires an extensive program of experimental studies. They conclude that the statistical methods used to study mediation are flawed and that even sophisticated experimental designs cannot speak to mediation questions without strong assumptions.

Abstract

The question of how causal effects are transmitted is fascinating and inevitably arises whenever experiments are presented. Social scientists cannot be faulted for taking a lively interest in “mediation,” the process by which causal influences are transmitted. However, social scientists frequently underestimate the difficulty of establishing causal pathways in a rigorous empirical manner. We argue that the statistical methods currently used to study mediation are flawed and that even sophisticated experimental designs cannot speak to questions of mediation without the aid of strong assumptions. The study of mediation is more demanding than most social scientists suppose and requires not one experimental study but rather an extensive program of experimental research.