Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesis

TLDR

The authors introduce global style tokens (GSTs), a bank of embeddings jointly trained within Tacotron to capture expressive speech style without explicit labels. GSTs are learned unsupervised as soft, interpretable embeddings that can control synthesis parameters such as speed and speaking style, enable style transfer across long texts, and factor out noise and speaker identity. The trained GSTs successfully model diverse acoustic expressiveness, provide controllable synthesis, support style transfer, and separate noise and speaker traits, demonstrating scalable, robust speech synthesis.

Abstract

In this work, we propose global style tokens (GSTs), a bank of embeddings that are jointly trained within Tacotron, a state-of-the-art end-to-end speech synthesis system. The embeddings are trained with no explicit labels, yet learn to model a large range of acoustic expressiveness. GSTs lead to a rich set of significant results. The soft interpretable labels they generate can be used to control synthesis in novel ways, such as varying speed and speaking style - independently of the text content. They can also be used for style transfer, replicating the speaking style of a single audio clip across an entire long-form text corpus. When trained on noisy, unlabeled found data, GSTs learn to factorize noise and speaker identity, providing a path towards highly scalable but robust speech synthesis.