Visual Sentiment Prediction with Deep Convolutional Neural Networks

TLDR

Images are a popular medium for expressing emotions online, yet sentiment analysis of images has received limited research compared to text. The authors propose a novel visual sentiment prediction framework that applies deep convolutional neural networks to image understanding. The framework leverages transfer learning from a large pre‑trained CNN trained on object recognition data to predict sentiment. Experiments on Twitter and Tumblr datasets demonstrate the effectiveness of the proposed visual sentiment analysis framework.

Abstract

Images have become one of the most popular types of media through which users convey their emotions within online social networks. Although vast amount of research is devoted to sentiment analysis of textual data, there has been very limited work that focuses on analyzing sentiment of image data. In this work, we propose a novel visual sentiment prediction framework that performs image understanding with Deep Convolutional Neural Networks (CNN). Specifically, the proposed sentiment prediction framework performs transfer learning from a CNN with millions of parameters, which is pre-trained on large-scale data for object recognition. Experiments conducted on two real-world datasets from Twitter and Tumblr demonstrate the effectiveness of the proposed visual sentiment analysis framework.