Speech Synthesis Evaluation — State-of-the-Art Assessment and Suggestion for a Novel Research Program

Abstract

Speech synthesis applications have become an ubiquity, in navigation systems, digital assistants or as screen or audio book readers.Despite their impact on the acceptability of the systems in which they are embedded, and despite the fact that different applications probably need different types of TTS voices, TTS evaluation is still largely treated as an isolated problem.Even though there is strong agreement among researchers that the mainstream approaches to Text-to-Speech (TTS) evaluation are often insufficient and may even be misleading, there exist few clear-cut suggestions as to (1) how TTS evaluations may be realistically improved on a large scale, and (2) how such improvements may lead to an informed feedback for system developers and, ultimately, better systems relying on TTS.This paper reviews the current state-of-the-art in TTS evaluation, and suggests a novel user-centered research program for this area.