2024 · 18 citations · 20 references
ComputingComputer ScienceComputational LinguisticsMachine LearningNatural Language ProcessingAffective ComputingAnnotationEmotion RecognitionEmotional ExpressionLanguage ModelingSpoken CorporaAcoustic Feature ExtractionAffective State AnnotationAutomatic Emotion RecognitionData AugmentationEmotion AnnotationEmotion Recognition From SpeechLarge Language ModelsSpeech Data Annotation
The remarkable emergence of large language models (LLM) and their vast capabilities have opened a possibility for applications in various fields, including speech emotion recognition (SER). Despite the advancement of SER methods and the abundance of speech data, the requirement of having speech data labeled with emotions is a challenge to fulfill, considering the cost of human annotation. In this study, we propose utilizing LLM to annotate emotional speeches, investigating the use of conversation sequence transcription, and incorporating the textual acoustic feature descriptors into the prompt. Furthermore, we also examine the application of annotation results on emotional speeches as training and augmentation data. Our experiment using the IEMOCAP dataset shows that emotional speech annotation using LLMs can outperform human annotation with possibly lower annotation costs. The SER trained using the annotation result as a whole training data or augmentation data reaches a performance close to state-of-the-art SER methods.
20
IEMOCAP: interactive emotional dyadic motion capture database
Carlos Busso, Murtaza Bulut, Chi-Chun Lee et al. · Language Resources and Evaluation · 2008 · 3.6K citations
librosa: Audio and Music Signal Analysis in Python
Brian McFee, Colin Raffel, Dawen Liang et al. · Proceedings of the Python in Science Conferences · 2015 · 2.9K citations · Full text