Interspeech 2022 · 2022 · 21 citations · 21 references
Language StudiesLinguisticsAdaptation StudiesPhoneticsVocal PerformanceSpeech RecognitionSpeech SynthesisAttentionSpeaker AdaptationTemporal Relations
Zero-shot speaker adaptation aims to clone an unseen speaker's voice without any adaptation time and parameters.Previous researches usually use a speaker encoder to extract a global fixed speaker embedding from reference speech, and several attempts have tried variable-length speaker embedding.However, they neglect to transfer the personal pronunciation characteristics related to phoneme content, leading to poor speaker similarity in terms of detailed speaking styles and pronunciation habits.To improve the ability of the speaker encoder to model personal pronunciation characteristics, we propose content-dependent fine-grained speaker embedding for zero-shot speaker adaptation.The corresponding local content embeddings and speaker embeddings are extracted from a reference speech, respectively.Instead of modeling the temporal relations, a reference attention module is introduced to model the content relevance between the reference speech and the input text, and to generate the finegrained speaker embedding for each phoneme encoder output.The experimental results show that our proposed method can improve speaker similarity of synthesized speeches, especially for unseen speakers.
21
DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · 2023 · 76.3K citations · Full text
Laurens van der Maaten, Geoffrey E. Hinton · Journal of Machine Learning Research · 2008 · 35.7K citations
Natural TTS Synthesis by Conditioning Wavenet on MEL Spectrogram Predictions
Jonathan Shen, Ruoming Pang, Ron J. Weiss et al. · 2018 · 2.7K citations
Montreal Forced Aligner: Trainable Text-Speech Alignment Using Kaldi
Michael McAuliffe, Michaela Socolof, Sarah Mihuc et al. · 2017 · 1.1K citations