Structured Attention for Unsupervised Dialogue Structure Induction

Abstract

Inducing a meaningful structural representation from one or a set of\ndialogues is a crucial but challenging task in computational linguistics.\nAdvancement made in this area is critical for dialogue system design and\ndiscourse analysis. It can also be extended to solve grammatical inference. In\nthis work, we propose to incorporate structured attention layers into a\nVariational Recurrent Neural Network (VRNN) model with discrete latent states\nto learn dialogue structure in an unsupervised fashion. Compared to a vanilla\nVRNN, structured attention enables a model to focus on different parts of the\nsource sentence embeddings while enforcing a structural inductive bias.\nExperiments show that on two-party dialogue datasets, VRNN with structured\nattention learns semantic structures that are similar to templates used to\ngenerate this dialogue corpus. While on multi-party dialogue datasets, our\nmodel learns an interactive structure demonstrating its capability of\ndistinguishing speakers or addresses, automatically disentangling dialogues\nwithout explicit human annotation.\n