Towards End-to-End Learning for Dialog State Tracking and Management using Deep Reinforcement Learning

TLDR

The study proposes an end‑to‑end task‑oriented dialog framework that combines a Deep Recurrent Q‑Network with a hybrid reinforcement‑learning and supervised‑learning algorithm to accelerate learning. The framework interfaces with a relational database, jointly learns language understanding and dialog strategy, and was evaluated on a 20‑Question Game simulator. Results show the method surpasses a modular baseline and learns a distributed representation of latent dialog state.

Abstract

This paper presents an end-to-end framework for task-oriented dialog systems using a variant of Deep Recurrent Q-Networks (DRQN). The model is able to interface with a relational database and jointly learn policies for both language understanding and dialog strategy. Moreover, we propose a hybrid algorithm that combines the strength of reinforcement learning and supervised learning to achieve faster learning speed. We evaluated the proposed model on a 20 Question Game conversational game simulator. Results show that the proposed method outperforms the modular-based baseline and learns a distributed representation of the latent dialog state.