The (Non) Religion of Mechanical Turk Workers

Abstract

Social science researchers have increasingly come to utilize Amazon's Mechanical Turk (MTurk) to obtain adult, opt‐in samples for use with experiments. Based on the demographic characteristics of MTurk samples, studies have provided some support for the representativeness of MTurk. Others have warranted caution based on demographic characteristics and comparisons of reliability. Yet, what is missing is an examination of the most glaring demographic difference in MTurk—religion. We compare five MTurk samples with a student convenience sample and the 2012 General Social Survey, finding that MTurk samples have a consistent bias toward nonreligion. MTurk surveys significantly overrepresent seculars and underrepresent Catholics and evangelical Protestants. We then compare the religiosity of religious identifiers across samples as well as relationships between religiosity and partisanship, finding many similarities and a few important differences from the general population.