arXiv (Cornell University) · 2018 · 75 citations · 0 references
ComputingComputer ScienceAcousticsFeature EngineeringMachine LearningDeep LearningSpeech RecognitionVoice ProcessingAdditivity ModelingAudio Feature ExtractionDeep Neural Network Acoustic ModelsEnd-to-end Speech RecognitionFeature Engineering For Audio SignalsPhase InformationSound Source SeparationTime Scales
Models for audio source separation typically rely on magnitude spectra, ignoring phase and depending on hyper‑parameters, and the high sampling rates of audio make long temporal contexts difficult yet essential for high‑quality separation due to long‑range correlations. We investigate end‑to‑end time‑domain source separation to model phase and avoid fixed spectral transforms, proposing Wave‑U‑Net, a 1D U‑Net that repeatedly resamples feature maps to compute and combine multi‑scale features. Wave‑U‑Net adapts the U‑Net to the one‑dimensional time domain, repeatedly resampling feature maps to compute and combine multi‑scale features, and incorporates architectural improvements such as an additivity‑enforcing output layer, an upsampling technique, and a context‑aware prediction framework to reduce artifacts. Experiments on singing voice separation show that Wave‑U‑Net achieves performance comparable to a state‑of‑the‑art spectrogram‑based U‑Net with the same data, while also revealing outlier issues in SDR metrics and recommending rank‑based statistics.
Models for audio source separation usually operate on the magnitude spectrum, which ignores phase information and makes separation performance dependant on hyper-parameters for the spectral front-end. Therefore, we investigate end-to-end source separation in the time-domain, which allows modelling phase information and avoids fixed spectral transformations. Due to high sampling rates for audio, employing a long temporal input context on the sample level is difficult, but required for high quality separation results because of long-range temporal correlations. In this context, we propose the Wave-U-Net, an adaptation of the U-Net to the one-dimensional time domain, which repeatedly resamples feature maps to compute and combine features at different time scales. We introduce further architectural improvements, including an output layer that enforces source additivity, an upsampling technique and a context-aware prediction framework to reduce output artifacts. Experiments for singing voice separation indicate that our architecture yields a performance comparable to a state-of-the-art spectrogram-based U-Net architecture, given the same data. Finally, we reveal a problem with outliers in the currently used SDR evaluation metrics and suggest reporting rank-based statistics to alleviate this problem.