Sensors · 2022 · 43 citations · 19 references
ComputingComputer ScienceComputational LinguisticsMachine LearningNatural Language ProcessingAffective ComputingEmotion RecognitionMultimodal InteractionOpinion MiningSentiment AnalysisSpeech RecognitionBinary ClassificationEmotion Recognition EvaluationEmotion Recognition From SpeechHuman Multimodal InteractionMultimodal Emotion RecognitionMultimodal FusionSelf-supervised Speech Representation LearningSentiment Classification
The study of understanding sentiment and emotion in speech is a challenging task in human multimodal language. However, in certain cases, such as telephone calls, only audio data can be obtained. In this study, we independently evaluated sentiment analysis and emotion recognition from speech using recent self-supervised learning models-specifically, universal speech representations with speaker-aware pre-training models. Three different sizes of universal models were evaluated for three sentiment tasks and an emotion task. The evaluation revealed that the best results were obtained with two classes of sentiment analysis, based on both weighted and unweighted accuracy scores (81% and 73%). This binary classification with unimodal acoustic analysis also performed competitively compared to previous methods which used multimodal fusion. The models failed to make accurate predictionsin an emotion recognition task and in sentiment analysis tasks with higher numbers of classes. The unbalanced property of the datasets may also have contributed to the performance degradations observed in the six-class emotion, three-class sentiment, and seven-class sentiment tasks.
19
SUPERB: Speech Processing Universal PERformance Benchmark
Shu-Wen Yang, Po-Han Chi, Yung-Sung Chuang et al. · 2021 · 637 citations
Convolutional MKL Based Multimodal Emotion Recognition and Sentiment Analysis
Soujanya Poria, Iti Chaturvedi, Erik Cambria et al. · 2016 · 594 citations