Mathematical Biosciences & Engineering · 2023 · 11 citations · 17 references
ComputingComputer ScienceFeature EngineeringMachine LearningNatural Language ProcessingDeep LearningEmotion RecognitionFeature-level FusionMultimodal InteractionOpinion MiningSentiment AnalysisAttentionAttention MechanismsFeature FusionGated Recurrent Unit NetworksMultimodal Data FusionMultimodal Feature FusionMultimodal FusionMultimodal Information Extraction
Multimodal sentiment analysis is an important area of artificial intelligence. It integrates multiple modalities such as text, audio, video and image into a compact multimodal representation and obtains sentiment information from them. In this paper, we improve two modules, i.e., feature extraction and feature fusion, to enhance multimodal sentiment analysis and finally propose an attention-based two-layer bidirectional GRU (AB-GRU, gated recurrent unit) multimodal sentiment analysis method. For the feature extraction module, we use a two-layer bidirectional GRU network and connect two layers of attention mechanisms to enhance the extraction of important information. The feature fusion part uses low-rank multimodal fusion, which can reduce the multimodal data dimensionality and improve the computational rate and accuracy. The experimental results demonstrate that the AB-GRU model can achieve 80.9% accuracy on the CMU-MOSI dataset, which exceeds the same model type by at least 2.5%. The AB-GRU model also possesses a strong generalization capability and solid robustness.
17
Hamed Jelodar, Yongli Wang, Rita Orji et al. · IEEE Journal of Biomedical and Health Informatics · 2020 · 379 citations
Multimodal sentiment analysis with word-level fusion and reinforcement learning
Minghai Chen, Sen Wang, Paul Pu Liang et al. · 2017 · 322 citations
Multimodal sentiment analysis based on fusion methods: A survey
Linan Zhu, Zhechao Zhu, Chenwei Zhang et al. · Information Fusion · 2023 · 307 citations
A BERT Framework to Sentiment Analysis of Tweets
Abayomi Bello, Sin-Chun Ng, Man-Fai Leung · Sensors · 2023 · 234 citations · Full text
Multimodal Language Analysis with Recurrent Multistage Fusion
Paul Pu Liang, Liu Ziyin, AmirAli Bagher Zadeh et al. · 2018 · 214 citations · Full text