Large Scale Crowdsourcing and Characterization of Twitter Abusive Behavior

TLDR

Online social networks have seen rising sexism, racism, and cyberbullying, with prior research focusing on abusive behavior on platforms such as Facebook and Twitter. The study aims to conduct an eight‑month, holistic analysis of abusive behavior on Twitter and to develop an incremental, crowdsourced annotation methodology. The authors use a wide range of labeling schemes and an incremental, crowdsourced annotation process to tag a large set of tweets for abuse‑related labels. The analysis yields a compact, robust set of abuse labels and a publicly available dataset of 80,000 annotated tweets.

Abstract

In recent years online social networks have suffered an increase in sexism, racism, and other types of aggressive and cyberbullying behavior, often manifesting itself through offensive, abusive, or hateful language. Past scientific work focused on studying these forms of abusive activity in popular online social networks, such as Facebook and Twitter. Building on such work, we present an eight month study of the various forms of abusive behavior on Twitter, in a holistic fashion. Departing from past work, we examine a wide variety of labeling schemes, which cover different forms of abusive behavior. We propose an incremental and iterative methodology that leverages the power of crowdsourcing to annotate a large collection of tweets with a set of abuse-related labels. By applying our methodology and performing statistical analysis for label merging or elimination, we identify a reduced but robust set of labels to characterize abuse-related tweets. Finally, we offer a characterization of our annotated dataset of 80 thousand tweets, which we make publicly available for further scientific exploration.