2024 · 17 citations · 16 references
ComputingData ScienceFeature EngineeringNatural Language ProcessingPhoneticsAffective ComputingDeep LearningEmotion RecognitionEmotional ExpressionSpeech RecognitionSpeech SynthesisCross-domain Emotion RecognitionEmotion DetectionEmotion Recognition From SpeechMultimodal Emotion RecognitionProsody
Existing emotional speech synthesis methods often utilize an utterance-level style embedding extracted from reference audio, neglecting the inherent multi-scale property of speech prosody. We introduce ED-TTS, a multi-scale emotional speech synthesis model that leverages Speech Emotion Diarization (SED) and Speech Emotion Recognition (SER) to model emotions at different levels. Specifically, our proposed approach integrates the utterance-level emotion embedding extracted by SER with fine-grained frame-level emotion embedding obtained from SED. These embeddings are used to condition the reverse process of the denoising diffusion probabilistic model (DDPM). Additionally, we employ cross-domain SED to accurately predict soft labels, addressing the challenge of a scarcity of fine-grained emotion-annotated datasets for supervising emotional TTS training.
16
IEMOCAP: interactive emotional dyadic motion capture database
Carlos Busso, Murtaza Bulut, Chi-Chun Lee et al. · Language Resources and Evaluation · 2008 · 3.6K citations
Montreal Forced Aligner: Trainable Text-Speech Alignment Using Kaldi
Michael McAuliffe, Michaela Socolof, Sarah Mihuc et al. · 2017 · 1.1K citations
Generalized End-to-End Loss for Speaker Verification
Li Wan, Quan Wang, Alan Papir et al. · 2018 · 837 citations