Vulnerability in speaker verification - a study of technical impostor techniques

Abstract

The development of natural language spoken dialog systems requires collection and labeling of a large set of user-system interactions (knowledge). Ideally, it should be collected from live traffic in the field, since scripted scenarios in a lab typically result in unnatural phraseology. To achieve this, we introduce an extension of the of Oz, a hidden human agent who oversees the machine side of the interaction in collaboration with the automated dialog manager, unbeknownst to the user. The Wizard can provide a continuum of supervision ranging from explicitly controlling every step to a Wizardoverride mode where the machine operates semi-autonomously and the human overrides only when necessary. All interactions are instrumented and entered directly into a database using a standard interface and SQL. The collected data will typically be analyzed in the laboratory to quantify system performance, and to introduce algorithmic improvements prior to a new set of experiments. Over the course of system development iterations, the nature of the Wizard evolves over a continuum of functions from actual control (in the absence of any system knowledge) to passive observer. The user cannot tell the difference.