Using Mechanical Turk to Study Clinical Populations

TLDR

Recruiting participants with psychiatric symptoms is challenging, but online crowdsourcing platforms may help, yet no prior studies have examined their utility for psychopathology research. The study aims to assess the prevalence of psychiatric disorders and related problems, and evaluate the reliability and validity of self‑reported data from Amazon Mechanical Turk users. The authors surveyed MTurk participants, collecting self‑reports on psychiatric disorders and related problems, and analyzed these data for prevalence, reliability, and validity. Crowdsourcing software offers several advantages for clinical research while highlighting potential problems such as misrepresentation that researchers should address when collecting data online.

Abstract

Although participants with psychiatric symptoms, specific risk factors, or rare demographic characteristics can be difficult to identify and recruit for participation in research, participants with these characteristics are crucial for research in the social, behavioral, and clinical sciences. Online research in general and crowdsourcing software in particular may offer a solution. However, no research to date has examined the utility of crowdsourcing software for conducting research on psychopathology. In the current study, we examined the prevalence of several psychiatric disorders and related problems, as well as the reliability and validity of participant reports on these domains, among users of Amazon’s Mechanical Turk. Findings suggest that crowdsourcing software offers several advantages for clinical research while providing insight into potential problems, such as misrepresentation, that researchers should address when collecting data online.