Learning Neural Templates for Text Generation

TLDR

Neural encoder‑decoder models have achieved empirical success in text generation but remain largely uninterpretable and difficult to control in phrasing or content. The study proposes a neural generation system that learns latent, discrete templates via a hidden semi‑markov model decoder. The system employs an HSMM decoder to jointly learn templates and generate text. The model learns useful templates that make generation more interpretable and controllable, and it scales to real datasets with performance near that of encoder‑decoder models.

Abstract

While neural, encoder-decoder models have had significant empirical success in text generation, there remain several unaddressed problems with this style of generation. Encoder-decoder models are largely (a) uninterpretable, and (b) difficult to control in terms of their phrasing or content. This work proposes a neural generation system using a hidden semi-markov model (HSMM) decoder, which learns latent, discrete templates jointly with learning to generate. We show that this model learns useful templates, and that these templates make generation both more interpretable and controllable. Furthermore, we show that this approach scales to real data sets and achieves strong performance nearing that of encoder-decoder text generation models.