Multi-Tensor Fusion Network with Hybrid Attention for Multimodal Sentiment Analysis

Abstract

Multimodal sentiment analysis is a highly sought-after topic in natural language processing. In this paper, a multi-tensor fusion network with hybrid attention architecture for multimodal sentiment analysis is proposed. Firstly, Bi-LSTM is applied to encode contextual representation in different modalities. Following this, modalities features are extracted and interacted with by the hybrid attention mechanism. Finally, multi-tensor fusion approach is used to further enhance the effectiveness of fusing interaction features in different modalities. The proposed approach outperforms the existing advanced approaches on two benchmarks through a series of regression experiments for sentiment intensity prediction, as it improves F1-score by 3.4% and 2.1% points respectively. Our architecture would be open-sourced on Github <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> for researchers to use.