Toward Controlled Generation of Text

TLDR

Generic generation and manipulation of text is challenging and has limited success compared to recent deep generative modeling in the visual domain. This paper aims to generate plausible natural language sentences whose attributes are dynamically controlled by learning disentangled latent representations with designated semantics. The model combines variational auto‑encoders with holistic attribute discriminators, uses differentiable approximations to discrete text, imposes explicit constraints on independent attribute controls, and collaborates generator and discriminators to learn highly interpretable representations from word annotations and produce realistic sentences with desired attributes. Quantitative evaluation validates the accuracy of sentence and attribute generation.

Abstract

Generic generation and manipulation of text is challenging and has limited success compared to recent deep generative modeling in visual domain. This paper aims at generating plausible natural language sentences, whose attributes are dynamically controlled by learning disentangled latent representations with designated semantics. We propose a new neural generative model which combines variational auto-encoders and holistic attribute discriminators for effective imposition of semantic structures. With differentiable approximation to discrete text samples, explicit constraints on independent attribute controls, and efficient collaborative learning of generator and discriminators, our model learns highly interpretable representations from even only word annotations, and produces realistic sentences with desired attributes. Quantitative evaluation validates the accuracy of sentence and attribute generation.