A Wizard of Oz Framework for Collecting Spoken Human Computer Dialogs: An Experiment Procedure for the Design and Testing of Natural Language In-Vehicle Technology Systems

Abstract

This paper discusses the tools and procedures used for collecting domain-specific natural language speech data for purposes of designing dialog system interfaces and improving dialog system performance. The tools used in our data collection process include the Wizard of Oz (WOZ) paradigm and a driving simulator. The WOZ paradigm utilizes a wizard (experimenter), behaving as an intelligent computer application, that interacts with a participant during simulated driving. Data collection takes place during simulated driving due to the impact that one's current location in an environment and the driving situation can have on the type of language used (i.e., utterances) and the manner in which the language is spoken (i.e., prosody). During simulated driving, participants perform domain-specific tasks with what we characterize as a task-constrained language-unconstrained approach. The goal of this approach is to strike a balance between tasks that do not sufficiently constrain participants speech and those that constrain speech too much.