Learning to interpret natural language commands through human-robot dialog

TLDR

Intelligent robots must interpret natural language requests from naive users, yet existing methods struggle with language variation or require costly annotated corpora. The study introduces a dialog agent that semantically parses human instructions, resolves ambiguities via a dialog manager, and incrementally learns from user paraphrases. The agent was deployed on a web interface with hundreds of Mechanical Turk users and on a mobile robot over several days to handle navigation and delivery requests in an office setting. Both deployments showed significant increases in user satisfaction after the agent learned from conversations.

Abstract

Intelligent robots frequently need to understand requests from naive users through natural language. Previous approaches either cannot account for language variation, e.g., keyword search, or require gathering large annotated corpora, which can be expensive and cannot adapt to new variation. We introduce a dialog agent for mobile robots that understands human instructions through semantic parsing, actively resolves ambiguities using a dialog manager, and incrementally learns from human-robot conversations by inducing training data from user paraphrases. Our dialog agent is implemented and tested both on a web interface with hundreds of users via Mechanical Turk and on a mobile robot over several days, tasked with understanding navigation and delivery requests through natural language in an office environment. In both contexts, We observe significant improvements in user satisfaction after learning from conversations.