Spectral voice conversion for text-to-speech synthesis

TLDR

The paper proposes a voice conversion algorithm that transforms a source speaker’s speech to sound as if spoken by a target speaker. The method uses a residual‑excited LPC diphone synthesizer, mapping spectral parameters with a locally linear Gaussian mixture model and adjusting LPC residuals to match target pitch, with training data subsets selected by vector‑quantization. Objective and perceptual tests show the approach reliably outperforms a prior method on small training sets, achieves near‑optimal spectral conversion with limited data, and further improves quality as training size increases.

Abstract

A new voice conversion algorithm that modifies a source speaker's speech to sound as if produced by a target speaker is presented. It is applied to a residual-excited LPC text-to-speech diphone synthesizer. Spectral parameters are mapped using a locally linear transformation based on Gaussian mixture models whose parameters are trained by joint density estimation. The LPC residuals are adjusted to match the target speakers average pitch. To study effects of the amount of training on performance, data sets of varying sizes are created by automatically selecting subsets of all available diphones by a vector quantization method. In an objective evaluation, the proposed method is found to perform more reliably for small training sets than a previous approach. In perceptual tests, it was shown that nearly optimal spectral conversion performance was achieved, even with a small amount of training data. However, speech quality improved with increases in the training set size.