Electronic Survey Methodology: A Case Study in Reaching Hard-to-Involve Internet Users

TLDR

Using the Internet for quantitative research presents unique challenges; paper‑based survey quality criteria cannot be fully adapted; electronic surveys have distinct technological, demographic, and response characteristics that affect design, use, and implementation, requiring careful attention to design, privacy, sampling, distribution, response rates, and piloting. The article aims to present quality criteria for electronic survey design and use derived from recent literature. These criteria were developed through an investigation of recent electronic survey literature. The case study demonstrates that hard‑to‑reach online participants can be reached using the most important quality criteria, but also reveals conflicts among criteria and challenges researchers in cultures intolerant of intrusions.

Abstract

Abstract Using the Internet to conduct quantitative research presents challenges not found in conventional research. Paper-based survey quality criteria cannot be completely adapted to electronic formats. Electronic surveys have distinctive technological, demographic, and response characteristics that affect their design, use, and implementation. Survey design, participant privacy and confidentiality, sampling and subject solicitation, distribution methods and response rates, and survey piloting are critical methodological components that must be addressed. In this article, quality criteria for electronic survey design and use based on an investigation of recent electronic survey literature are presented. The application of these criteria to reach a hard-to-involve online population-nonpublic participants of online communities (also known as "lurkers")-and survey them on their community participation, a topic not salient to the purpose of their online communities is demonstrated in a case study. The results show that a hard-to-reach audience can be reached using the quality criteria that are most important for reaching these types of audiences. The results suggest how the use of some criteria may conflict and what researchers may experience when conducting electronic surveys in an online culture in which people are not tolerant of intrusions into online lives.