IEEE Transactions on Audio Speech and Language Processing · 2024 · 15 citations · 35 references
Health ProfessionsAudiologyMachine LearningPhoneticsPsychoacousticsGenerative ModelingSpeech Intelligibility EnhancementSpeech NaturalnessSpeech RecognitionSpeech SynthesisWaveform GeneratorsBandwidth ExtensionBandwidth Extension TechniquesConvolutional Neural NetworksGenerative Adversarial NetworksNeural Speech SynthesisSpectral EnhancementSpeech Quality AssessmentSpeech Waveform Generation
Speech bandwidth extension (BWE) refers to widening the frequency bandwidth range of speech signals, enhancing the speech quality towards brighter and fuller. This paper proposes a generative adversarial network (GAN) based BWE model with parallel prediction of Amplitude and Phase spectra, named AP-BWE, which achieves both high-quality and efficient wideband speech waveform generation. The proposed AP-BWE generator is entirely based on convolutional neural networks (CNNs). It features a dual-stream architecture with mutual interaction, where the amplitude stream and the phase stream communicate with each other and respectively extend the high-frequency components from the source narrowband amplitude and phase spectra. To improve the naturalness of the extended speech signals, we employ a multi-period discriminator at the waveform level and design a pair of multi-resolution amplitude and phase discriminators at the spectral level, respectively. Experimental results demonstrate that our proposed AP-BWE achieves state-of-the-art performance in terms of speech quality for BWE tasks targeting sampling rates of both 16 kHz and 48 kHz. In terms of generation efficiency, due to the all-convolutional architecture and all-frame-level operations, the proposed AP-BWE can generate 48 kHz waveform samples 292.3 times faster than real-time on a single RTX 4090 GPU and 18.1 times faster than real-time on a single CPU. Notably, to our knowledge, AP-BWE is the first to achieve the direct extension of the high-frequency phase spectrum, which is beneficial for improving the effectiveness of existing BWE methods.
35
DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · 2023 · 76.3K citations · Full text
Zhuang Liu, Hanzi Mao, Chao-Yuan Wu et al. · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · 2022 · 7.1K citations
LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speech
Heiga Zen, Viet Chau Dang, Rob Clark et al. · 2019 · 737 citations