2018 · 172 citations · 15 references
Language StudiesLinguisticsComputational LinguisticsDiscourse AnalysisSocial Media AnalyticsHate SpeechAutomated Hate Speech DetectionHate Speech AnnotationHate Speech CorporaHate Speech DefinitionsSocial Media Hate Speech
The paper describes a recently-created Twitter corpus of about 6,000 tweets, annotated for hate speech against immigrants, and developed to be a reference dataset for an automatic system of hate speech monitoring.The annotation scheme was therefore specifically designed to account for the multiplicity of factors that can contribute to the definition of a hate speech notion, and to offer a broader tagset capable of better representing all those factors, which may increase, or rather mitigate, the impact of the message.This resulted in a scheme that includes, besides hate speech, the following categories: aggressiveness, offensiveness, irony, stereotype, and (on an experimental basis) intensity.The paper hereby presented namely focuses on how this annotation scheme was designed and applied to the corpus.In particular, also comparing the annotation produced by CrowdFlower contributors and by expert annotators, we make some remarks about the value of the novel resource as gold standard, which stems from a preliminary qualitative analysis of the annotated data and on future corpus development.
15
Assessing agreement on classification tasks: the kappa statistic
Jean Carletta · ArXiv.org · 1996 · 2.1K citations · Full text
Prejudice: Its Social Psychology
Catherine Campbell, Rupert Brown · British Journal of Sociology · 1997 · 792 citations