arXiv (Cornell University) · 2019 · 95 citations · 26 references
ComputingComputer ScienceArtificial IntelligenceMachine LearningNatural Language ProcessingDeep LearningGenerative ArchitectureGenerative ModelingNeural NetworksRepresentation LearningAutoencodersEncoder-decoder ArchitecturesEnergy Function OptimizationEnergy-based Generative ModelsGenerative Adversarial NetworksTuning
Although variational autoencoders (VAEs) represent a widely influential deep generative model, many aspects of the underlying energy function remain poorly understood. In particular, it is commonly believed that Gaussian encoder/decoder assumptions reduce the effectiveness of VAEs in generating realistic samples. In this regard, we rigorously analyze the VAE objective, differentiating situations where this belief is and is not actually true. We then leverage the corresponding insights to develop a simple VAE enhancement that requires no additional hyperparameters or sensitive tuning. Quantitatively, this proposal produces crisp samples and stable FID scores that are actually competitive with a variety of GAN models, all while retaining desirable attributes of the original VAE architecture. A shorter version of this work will appear in the ICLR 2019 conference proceedings (Dai and Wipf, 2019). The code for our model is available at https://github.com/daib13/ TwoStageVAE.
26
Gradient-based learning applied to document recognition
Yann LeCun, Léon Bottou, Yoshua Bengio et al. · Proceedings of the IEEE · 1998 · 58.2K citations · Full text
Auto-Encoding Variational Bayes
Diederik P. Kingma, Max Welling · UvA-DARE (University of Amsterdam) · 2013 · 15.6K citations · Full text
Independent component analysis: algorithms and applications
Aapo Hyvärinen, Erkki Oja · Neural Networks · 2000 · 8.8K citations
Deep Learning Face Attributes in the Wild
Ziwei Liu, Ping Luo, Xiaogang Wang et al. · 2015 · 7.7K citations