Gain Scores Revisited: A Graphical Models Perspective

Abstract

For misguided reasons, social scientists have long been reluctant to use gain scores for estimating causal effects. This article develops graphical models and graph-based arguments to show that gain score methods are a viable strategy for identifying causal treatment effects in observational studies. The proposed graphical models reveal that gain score methods rely on a bias-removing mechanism that is quite different to regular matching or covariance adjustment. While gain score methods offset noncausal associations via differencing, matching or covariance adjustment blocks noncausal association via conditioning. Since gain score estimators do not rely on conditioning, they are immune to measurement error in the pretest, bias amplification, and collider bias. The graph-based arguments also demonstrate that the key identifying assumption for gain score methods, the common trend assumption, is difficult to assess and justify when the pretest causally affects treatment assignment. Finally, we discuss the distinct role of pretests in the context of Lord’s paradox.