Robust Emotion Navigation: Few-shot Visual Sentiment Analysis by Auxiliary Noisy Data

Abstract

Few-shot visual sentiment analysis on social media is an important affective computing task. However, features acquired from few-shot samples are difficult, becasue the visual sentiment is a high-level integration task based on content and style. To address this issue, inspired by human learning processing, only a small number of multi-category emotions are learned from courses or specific occasions. In this paper, we propose a robust emotion navigation framework using auxiliary noisy data to re-focus on few-shot precise emotion knowledge. Firstly, we pre-trained the network on a large noisy data with cross-entropy loss, and the noise matrix can be estimated by predicted probability. Secondly, few-shot precise samples are applied as the prototype center to guide noisy data clustering. Here, the noise matrix is embedded into the loss function for re-weighting, which improves the noise robustness of the network. Finally, we relabel the noisy dataset with above joint training predictions and then re-train the network coarse-to-fine. We conduct experiments on three public sentiment datasets, including Sentibank, Twitter and Emotion6. The results demonstrate the effectiveness of the proposed method.