Fixed and random effects models: making an informed choice

TLDR

The literature is confused about the properties of fixed and random effects models, prompting a review of their capabilities and limitations. The paper evaluates multilevel modeling options to guide researchers, advocating the within‑between random‑effects model as a starting point for all analyses. The authors use simulation studies to assess how fixed, random, and within‑between random‑effects models handle common mis‑specifications. The within‑between random‑effects model is the most general, and simulations show that omitting random slopes yields anti‑conservative errors while mis‑specifying intercept distributions causes only modest bias, supporting its use.

Abstract

This paper assesses the options available to researchers analysing multilevel (including longitudinal) data, with the aim of supporting good methodological decision-making. Given the confusion in the literature about the key properties of fixed and random effects (FE and RE) models, we present these models' capabilities and limitations. We also discuss the within-between RE model, sometimes misleadingly labelled a 'hybrid' model, showing that it is the most general of the three, with all the strengths of the other two. As such, and because it allows for important extensions—notably random slopes—we argue it should be used (as a starting point at least) in all multilevel analyses. We develop the argument through simulations, evaluating how these models cope with some likely mis-specifications. These simulations reveal that (1) failing to include random slopes can generate anti-conservative standard errors, and (2) assuming random intercepts are Normally distributed, when they are not, introduces only modest biases. These results strengthen the case for the use of, and need for, these models.