CaltechAUTHORS (California Institute of Technology) · 2016 · 261 citations · 22 references
Open access
ComputingComputer ScienceComputer GraphicsComputer VisionVisual PerceptionDeep LearningImage SynthesisMotion EstimationAnalogy GenerationFramesLearned Feature RepresentationsMotion Representation LearningVideo GenerationVideo Representation LearningVisual Analogy
We study the problem of synthesizing a number of likely future frames from a single input image. In contrast to traditional methods, which have tackled this problem in a deterministic or non-parametric way, we propose a novel approach which models future frames in a probabilistic manner. Our proposed method is therefore able to synthesize multiple possible next frames using the same model. Solving this challenging problem involves low- and high-level image and motion understanding for successful image synthesis. Here, we propose a novel network structure, namely a Cross Convolutional Network, that encodes images as feature maps and motion information as convolutional kernels to aid in synthesizing future frames. In experiments, our model performs well on both synthetic data, such as 2D shapes and animated game sprites, as well as on real-wold video data. We show that our model can also be applied to tasks such as visual analogy-making, and present analysis of the learned network representations.
22
Auto-Encoding Variational Bayes
Diederik P. Kingma, Max Welling · UvA-DARE (University of Amsterdam) · 2013 · 15.6K citations · Full text
Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks
Emily Denton, Soumith Chintala, Arthur Szlam et al. · arXiv (Cornell University) · 2015 · 1.7K citations · Full text