An Examination of the Convergence of Online Panel Data and Conventionally Sourced Data

Abstract

Samples drawn from commercial online panel data (OPD) are becoming more prevalent in applied psychology and management research, but they remain controversial due to concerns with data quality. In order to determine if OPD are systematically different from conventionally sourced samples, we conduct meta- analyses of online panel samples and compare means, internal consistencies, and effect size estimates with those found in conventionally sourced data. Although some meaningful differences were found, results based on 54 independent samples and 17,324 participants show that OPD largely demonstrates similar psychometric properties to conventionally sourced data and OPD correlations generally fall within the credibility intervals of existing meta-analytic results using conventionally sourced data. We suggest that, with suitable caution, OPD are appropriate for many research questions in the field of applied psychology.