VoiceFixer: A Unified Framework for High-Fidelity Speech Restoration

Abstract

Speech restoration aims to remove distortions in speech signals. Prior\nmethods mainly focus on a single type of distortion, such as speech denoising\nor dereverberation. However, speech signals can be degraded by several\ndifferent distortions simultaneously in the real world. It is thus important to\nextend speech restoration models to deal with multiple distortions. In this\npaper, we introduce VoiceFixer, a unified framework for high-fidelity speech\nrestoration. VoiceFixer restores speech from multiple distortions (e.g., noise,\nreverberation, and clipping) and can expand degraded speech (e.g., noisy\nspeech) with a low bandwidth to 44.1 kHz full-bandwidth high-fidelity speech.\nWe design VoiceFixer based on (1) an analysis stage that predicts\nintermediate-level features from the degraded speech, and (2) a synthesis stage\nthat generates waveform using a neural vocoder. Both objective and subjective\nevaluations show that VoiceFixer is effective on severely degraded speech, such\nas real-world historical speech recordings. Samples of VoiceFixer are available\nat https://haoheliu.github.io/voicefixer.\n