DeLiGAN: Generative Adversarial Networks for Diverse and Limited Data

TLDR

Generative Adversarial Networks can produce realistic images but typically require large training sets to capture modality diversity. This work introduces DeLiGAN, a GAN architecture designed for diverse generation from limited data. DeLiGAN reparameterizes the latent space as a learnable mixture model and jointly trains its parameters with the GAN, while a modified inception‑score quantifies intra‑class diversity. The simple modification yields models that generate diverse handwritten digits, objects, and sketches even with scarce data.

Abstract

A class of recent approaches for generating images, called Generative Adversarial Networks (GAN), have been used to generate impressively realistic images of objects, bedrooms, handwritten digits and a variety of other image modalities. However, typical GAN-based approaches require large amounts of training data to capture the diversity across the image modality. In this paper, we propose DeLiGAN - a novel GAN-based architecture for diverse and limited training data scenarios. In our approach, we reparameterize the latent generative space as a mixture model and learn the mixture models parameters along with those of GAN. This seemingly simple modification to the GAN framework is surprisingly effective and results in models which enable diversity in generated samples although trained with limited data. In our work, we show that DeLiGAN can generate images of handwritten digits, objects and hand-drawn sketches, all using limited amounts of data. To quantitatively characterize intra-class diversity of generated samples, we also introduce a modified version of inception-score, a measure which has been found to correlate well with human assessment of generated samples.