Answering Without Reading: IMCs and Strong Satisficing in Online Surveys

Abstract

Some respondents of online surveys click responses at random. Screeners or instructional manipulation checks (IMC) have become customary for identifying this strong form of satisficing. This research first analyzes the factors that condition IMC failures using an online panel survey carried out in Spain (2011–2015). Our data show that the probability of passing a screener depends mainly on the screener’s difficulty, the individuals’ intrinsic motivations for answering the survey, and past failures. We then address the substantive consequences of omitting those who fail to pass IMCs. We find that this strategy introduces an additional source of bias in descriptive analyses. The article ends with a discussion of the implications that these findings have for the use of IMCs.