Social Media Analyses for Social Measurement

TLDR

Social media content analysis has shown promise of aligning with survey-based measurements, but its reliability as a substitute for official statistics remains uncertain due to differing assumptions, data characteristics, and ethical considerations. The study calls for interdisciplinary dialogue to identify where social media analytics can align with or diverge from traditional survey methods. Estimates from social media and surveys align variably, depending on topic, population, platform, and extraction methods, and social media can predict social phenomena even without full population coverage.

Abstract

Demonstrations that analyses of social media content can align with measurement from sample surveys have raised the question of whether survey research can be supplemented or even replaced with less costly and burdensome data mining of already-existing or "found" social media content. But just how trustworthy such measurement can be—say, to replace official statistics—is unknown. Survey researchers and data scientists approach key questions from starting assumptions and analytic traditions that differ on, for example, the need for representative samples drawn from frames that fully cover the population. New conversations between these scholarly communities are needed to understand the potential points of alignment and non-alignment. Across these approaches, there are major differences in (a) how participants (survey respondents and social media posters) understand the activity they are engaged in; (b) the nature of the data produced by survey responses and social media posts, and the inferences that are legitimate given the data; and (c) practical and ethical considerations surrounding the use of the data. Estimates are likely to align to differing degrees depending on the research topic and the populations under consideration, the particular features of the surveys and social media sites involved, and the analytic techniques for extracting opinions and experiences from social media. Traditional population coverage may not be required for social media content to effectively predict social phenomena to the extent that social media content distills or summarizes broader conversations that are also measured by surveys.