Voice Conversion from Unaligned Corpora using Variational Autoencoding Wasserstein Generative Adversarial Networks

TLDR

Building a voice conversion system from non‑parallel speech corpora is challenging but valuable, especially when source and target speakers do not share the same texts or languages, so generative models that use latent variables can bypass the need for frame alignment. The authors aim to develop a non‑parallel VC framework that explicitly incorporates a voice conversion objective into a variational autoencoding Wasserstein GAN. The framework constructs a VAW‑GAN that learns latent speech representations without requiring aligned data. Experiments show that the proposed framework can build a VC system from unaligned data and yields improved conversion quality.

Abstract

Building a voice conversion (VC) system from non-parallel speech corpora is challenging but highly valuable in real application scenarios. In most situations, the source and the target speakers do not repeat the same texts or they may even speak different languages. In this case, one possible, although indirect, solution is to build a generative model for speech. Generative models focus on explaining the observations with latent variables instead of learning a pairwise transformation function, thereby bypassing the requirement of speech frame alignment. In this paper, we propose a non-parallel VC framework with a variational autoencoding Wasserstein generative adversarial network (VAW-GAN) that explicitly considers a VC objective when building the speech model. Experimental results corroborate the capability of our framework for building a VC system from unaligned data, and demonstrate improved conversion quality.