Separating Facts from Fiction: Linguistic Models to Classify Suspicious and Trusted News Posts on Twitter

TLDR

Pew research shows that 62 % of U.S. adults obtain news from social media, 64 % report that fabricated stories cause confusion, and such misinformation contributes to global instability. The study aims to build predictive models that classify 130,000 Twitter news posts as suspicious or verified and distinguish four sub‑types of suspicious news: satire, hoaxes, clickbait, and propaganda. The authors trained neural‑network models on tweet content and social‑network interaction features, incorporating linguistic cues, to perform the classification.

Abstract

Pew research polls report 62 percent of U.S. adults get news on social media (Gottfried and Shearer, 2016). In a December poll, 64 percent of U.S. adults said that “made-up news” has caused a “great deal of confusion” about the facts of current events (Barthel et al., 2016). Fabricated stories in social media, ranging from deliberate propaganda to hoaxes and satire, contributes to this confusion in addition to having serious effects on global stability. In this work we build predictive models to classify 130 thousand news posts as suspicious or verified, and predict four sub-types of suspicious news – satire, hoaxes, clickbait and propaganda. We show that neural network models trained on tweet content and social network interactions outperform lexical models. Unlike previous work on deception detection, we find that adding syntax and grammar features to our models does not improve performance. Incorporating linguistic features improves classification results, however, social interaction features are most informative for finer-grained separation between four types of suspicious news posts.