2017 · 224 citations · 37 references
ComputingComputer ScienceComputational LinguisticsMachine LearningNatural Language ProcessingDeep LearningGenerative ArchitectureGenerative ModelingLanguage ModelingNatural Language GenerationText GenerationAutomated Text GenerationNeural Language ModelsNeural Text RepresentationRecurrent Neural Language ModelsRecurrent Neural Network Language ModelsRun-time AdaptationSequence Generation
In this paper we explore the effect of architectural choices on learning a variational autoencoder (VAE) for text generation. In contrast to the previously introduced VAE model for text where both the encoder and decoder are RNNs, we propose a novel hybrid architecture that blends fully feed-forward convolutional and deconvolutional components with a recurrent language model. Our architecture exhibits several attractive properties such as faster run time and convergence, ability to better handle long sequences and, more importantly, it helps to avoid the issue of the VAE collapsing to a deterministic model.
37
Sepp Hochreiter, Jürgen Schmidhuber · Neural Computation · 1997 · 98.1K citations
Adam: A Method for Stochastic Optimization
Diederik P. Kingma, Jimmy Ba · UvA-DARE (University of Amsterdam) · 2014 · 84.8K citations · Full text
Auto-Encoding Variational Bayes
Diederik P. Kingma, Max Welling · UvA-DARE (University of Amsterdam) · 2013 · 15.6K citations · Full text