2018 · 688 citations · 24 references
EngineeringAudiologyAcousticsComputational AcousticsPsychoacousticsSource SeparationSpeech RecognitionTime-frequency AnalysisEncoder-decoder ArchitecturesFrequency DecompositionSound Source SeparationSpeech SegregationTime-domain Source Modeling
Speech separation in multi‑talker settings is essential for robust speech processing, yet real‑time, low‑latency applications remain difficult because conventional time‑frequency masking suffers from phase/magnitude decoupling and long‑window constraints. The authors introduce TasNet, a time‑domain audio separation network designed to address these real‑time challenges. TasNet employs an encoder‑decoder architecture that operates directly on raw audio, estimating nonnegative masks on encoder outputs and reconstructing separated signals with the decoder, thereby eliminating the need for time‑frequency decomposition. The system surpasses current causal and noncausal baselines, lowers computational cost and latency, making it well suited for low‑power, real‑time use in hearables and telecom devices.
Robust speech processing in multi-talker environments requires effective speech separation. Recent deep learning systems have made significant progress toward solving this problem, yet it remains challenging particularly in real-time, short latency applications. Most methods attempt to construct a mask for each source in time-frequency representation of the mixture signal which is not necessarily an optimal representation for speech separation. In addition, time-frequency decomposition results in inherent problems such as phase/magnitude decoupling and long time window which is required to achieve sufficient frequency resolution. We propose Time-domain Audio Separation Network (TasNet) to overcome these limitations. We directly model the signal in the time-domain using an encoder-decoder framework and perform the source separation on nonnegative encoder outputs. This method removes the frequency decomposition step and reduces the separation problem to estimation of source masks on encoder outputs which is then synthesized by the decoder. Our system outperforms the current state-of-the-art causal and noncausal speech separation algorithms, reduces the computational cost of speech separation, and significantly reduces the minimum required latency of the output. This makes TasNet suitable for applications where low-power, real-time implementation is desirable such as in hearable and telecommunication devices.
24
Adam: A Method for Stochastic Optimization
Diederik P. Kingma, Jimmy Ba · UvA-DARE (University of Amsterdam) · 2014 · 84.8K citations · Full text
Yoshua Bengio, Jérôme Louradour, Ronan Collobert et al. · 2009 · 5K citations · Full text
WaveNet: A Generative Model for Raw Audio
Aäron van den Oord, Sander Dieleman, Heiga Zen et al. · arXiv (Cornell University) · 2016 · 3.6K citations · Full text
Computing, Musicology, Acoustics +16