Should We Trust Web-Based Studies? A Comparative Analysis of Six Preconceptions About Internet Questionnaires.

TLDR

The Internet’s rapid growth offers psychologists new research opportunities. The study evaluates and compares internet-based self‑report questionnaire methods with traditional paper‑and‑pencil approaches. The authors tested six preconceptions by comparing a large internet sample (N = 361,703) to 510 published traditional samples. Internet samples were diverse in gender, socioeconomic status, region, and age, and their findings generalized across formats, were unaffected by nonserious or repeat responders, and aligned with traditional methods, indicating that internet methods can contribute to many areas of psychology.

Abstract

The rapid growth of the Internet provides a wealth of new research opportunities for psychologists. Internet data collection methods, with a focus on self-report questionnaires from self-selected samples, are evaluated and compared with traditional paper-and-pencil methods. Six preconceptions about Internet samples and data quality are evaluated by comparing a new large Internet sample (N = 361,703) with a set of 510 published traditional samples. Internet samples are shown to be relatively diverse with respect to gender, socioeconomic status, geographic region, and age. Moreover, Internet findings generalize across presentation formats, are not adversely affected by nonserious or repeat responders, and are consistent with findings from traditional methods. It is concluded that Internet methods can contribute to many areas of psychology.