Developing Multi-Labelled Corpus of Twitter Short Texts: A Semi-Automatic Method

TLDR

The rapid growth of electronic documents in the digital media age has created a need to extract textual features, and sentiment classification—requiring annotated corpora—remains limited by costly manual labeling and the inadequacy of single‑label datasets for complex emotions. This study aims to develop a semi‑automatic method for multi‑label emotion tagging to build new semantic corpora. The approach selects a base corpus and emotion tags, preprocesses data, automatically annotates tweets via word matching and weight calculation, and applies manual correction, tagging each sentence with emotional tendency and polarity and each tweet with its two primary tendencies. Experiments on the Sentiment140 corpus confirm the method’s effectiveness and consistency with manual annotation, yielding a 6,500‑tweet multi‑label emotion corpus for algorithm training.

Abstract

Facing fast-increasing electronic documents in the Digital Media Age, the need to extract textual features of online texts for better communication is growing. Sentiment classification might be the key method to catch emotions of online communication, and developing corpora with annotation of emotions is the first step to achieving sentiment classification. However, the labour-intensive and costly manual annotation has resulted in the lack of corpora for emotional words. Furthermore, single-label semantic corpora could hardly meet the requirement of modern analysis of complicated user’s emotions, but tagging emotional words with multiple labels is even more difficult than usual. Improvement of the methods of automatic emotion tagging with multiple emotion labels to construct new semantic corpora is urgently needed. Taking Twitter short texts as the case, this study proposes a new semi-automatic method to annotate Internet short texts with multiple labels and form a multi-labelled corpus for further algorithm training. Each sentence is tagged with both the emotional tendency and polarity, and each tweet, which generally contains several sentences, is tagged with the first two major emotional tendencies. The semi-automatic multi-labelled annotation is achieved through the process of selecting the base corpus and emotional tags, data preprocessing, automatic annotation through word matching and weight calculation, and manual correction in case of multiple emotional tendencies are found. The experiments on the Sentiment140 published Twitter corpus demonstrate the effectiveness of the proposed approach and show consistency between the results of semi-automatic annotation and manual annotation. By applying this method, this study summarises the annotation specification and constructs a multi-labelled emotion corpus with 6500 tweets for further algorithm training.