arXiv (Cornell University) · 2019 · 26 citations · 20 references
ComputingComputer ScienceAcousticsMachine LearningPhase TransformationsSpeech AcousticsSpeech Intelligibility EnhancementSpeech RecognitionVoice ProcessingCorrelationLearningMaskingSpeech EnhancementSpeech Enhancement AlgorithmsSpeech Signal Enhancement
Time-frequency (T-F) domain masking is a mainstream approach for single-channel speech enhancement. Recently, focuses have been put to phase prediction in addition to amplitude prediction. In this paper, we propose a phase-and-harmonics-aware deep neural network (DNN), named PHASEN, for this task. Unlike previous methods that directly use a complex ideal ratio mask to supervise the DNN learning, we design a two-stream network, where amplitude stream and phase stream are dedicated to amplitude and phase prediction. We discover that the two streams should communicate with each other, and this is crucial to phase prediction. In addition, we propose frequency transformation blocks to catch long-range correlations along the frequency axis. The visualization shows that the learned transformation matrix spontaneously captures the harmonic correlation, which has been proven to be helpful for T-F spectrogram reconstruction. With these two innovations, PHASEN acquires the ability to handle detailed phase patterns and to utilize harmonic patterns, getting 1.76dB SDR improvement on AVSpeech + AudioSet dataset. It also achieves significant gains over Google's network on this dataset. On Voice Bank + DEMAND dataset, PHASEN outperforms previous methods by a large margin on four metrics.
20
Audio Set: An ontology and human-labeled dataset for audio events
Jort F. Gemmeke, Daniel P. W. Ellis, Dylan Freedman et al. · 2017 · 3K citations
Investigating RNN-based speech enhancement methods for noise-robust Text-to-Speech
Cassia Valentini-Botinhao, Xin Wang, Shinji Takaki et al. · 2016 · 466 citations