An Evaluation of a Cognitive Theory of Response-Order Effects in Survey Measurement

TLDR

Prior studies have shown that the order of response choices influences survey answers, yet no psychological theory explains these effects. The study proposes a psychological theory of response‑order effects based on diverse psychological research. The authors test the theory using a split‑ballot experiment from the 1984 GSS on Kohn’s parental values measure, examining predicted primacy effects strongest among low‑cognitive‑sophistication respondents, and also evaluate a form‑resistant correlation hypothesis. The predictions are confirmed, and while response‑order changes alter item correlations, the latent value dimension remains essentially unchanged.

Abstract

Previous research has documented effects of the order in which response choices are offered to respondents using closed-ended survey items, but no theory of the psychological sources of these effects has yet been proposed. This paper offers such a theory drawn from a variety of psychological research. Using data from a split-ballot experiment in the 1984 General Social Survey involving a variant of Kohn's parental values measure, we test some predictions made by the theory about what kind of response order effect would be expected (a primacy effect) and among which respondents it should be strongest (those low in cognitive sophistication). These predictions are confirmed. We also test the “form-resistant correlation” hypothesis. Although correlations between items are altered by changes m response order, the presence and nature of the latent value dimension underlying these responses is essentially unaffected.