Building a Conversational Agent Overnight with Dialogue Self-Play

TLDR

The authors introduce M2M, a framework that combines automation and crowdsourcing to rapidly bootstrap goal‑oriented dialogue agents and present a new 3,000‑dialogue corpus across two domains. M2M operates in two phases: first, simulated user and system bots produce template‑based dialogue outlines from a task schema and API client; second, crowd workers rewrite these outlines into natural utterances, enabling rapid, scalable data generation. Compared to Wizard‑of‑Oz data collection, M2M delivers more diverse and comprehensive dialogue flows with natural utterances, completes the entire process in a few hours, and yields a high‑quality 3,000‑dialogue corpus that outperforms popular datasets in surface‑form and flow diversity.

Abstract

We propose Machines Talking To Machines (M2M), a framework combining automation and crowdsourcing to rapidly bootstrap end-to-end dialogue agents for goal-oriented dialogues in arbitrary domains. M2M scales to new tasks with just a task schema and an API client from the dialogue system developer, but it is also customizable to cater to task-specific interactions. Compared to the Wizard-of-Oz approach for data collection, M2M achieves greater diversity and coverage of salient dialogue flows while maintaining the naturalness of individual utterances. In the first phase, a simulated user bot and a domain-agnostic system bot converse to exhaustively generate dialogue "outlines", i.e. sequences of template utterances and their semantic parses. In the second phase, crowd workers provide contextual rewrites of the dialogues to make the utterances more natural while preserving their meaning. The entire process can finish within a few hours. We propose a new corpus of 3,000 dialogues spanning 2 domains collected with M2M, and present comparisons with popular dialogue datasets on the quality and diversity of the surface forms and dialogue flows.