2019 · 10 citations · 16 references
ComputingComputer ScienceComputational LinguisticsNatural Language ProcessingPsycholinguisticsEmotion RecognitionLanguage ModelingMultimodal AnalysisMultimodal InteractionOpinion MiningSentiment AnalysisAttentionEmpirical EvaluationHierarchical Interaction ModelingMulti-modal Representation LearningMultimodal Emotion RecognitionMultimodal Feature RepresentationSentiment ClassificationSentiment Regression
As an emerging research area in natural language processing, multi-modal human language analysis spans language, vision and audio modalities. Understanding multi-modal language requires not only the modeling of independent dynamics within each modality (intra-modal dynamics), but also more importantly interactive dynamics among different modalities (inter-modal dynamics). In this paper, we propose a hierarchical approach to multi-modal language analysis with two levels of attention mechanism, namely interaction-level, which captures the intra-modal and inter-modal dynamics across different modalities with multiple types of attention, and selection-level attention, which selects the effective representations for final prediction by calculating the importance of each vector obtained from interaction-level. Empirical evaluation demonstrates the effectiveness of our proposed approach to multi-modal sentiment classification, sentiment regression and emotion recognition.
16
Adam: A Method for Stochastic Optimization
Diederik P. Kingma, Jimmy Ba · UvA-DARE (University of Amsterdam) · 2014 · 84.8K citations · Full text
ADADELTA: An Adaptive Learning Rate Method
Matthew D. Zeiler · arXiv (Cornell University) · 2012 · 5.5K citations · Full text
Deep Sparse Rectifier Neural Networks
Xavier Glorot, Antoine Bordes, Yoshua Bengio · 2012 · 5.4K citations
Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering
Peter Anderson, Xiaodong He, Chris Buehler et al. · 2018 · 5.1K citations