Dialog trajectory analysis

Abstract

Spoken dialog systems are becoming increasingly common in deployed services. These systems are not perfect, and are often deployed "open-loop" - lacking in systematic procedures for diagnosing problems and for making improvements. In order to target improvements where they will have the biggest impact, two things are needed: first, methods and tools for detailed analysis of a data feed of call logs and customer audio; second, an interactive tool for presenting an intuitive view of the results to those responsible for the application. We discuss the paradigm and an implementation through which we are able to close the loop between system execution and system evolution by providing an empirical dialog trajectory analysis represented via a stochastic finite state machine. Novel graph analysis algorithms are introduced for change detection, compression and pruning for display, based on user-interface objectives.