Generative Adversarial Text to Image Synthesis

TLDR

Automatic synthesis of realistic images from text is desirable but current AI systems lag behind; recent advances in recurrent neural networks for text representation and convolutional GANs for category‑specific image generation have paved the way. The authors aim to develop a novel deep architecture and GAN formulation that bridges text and image modeling to translate visual concepts from characters to pixels. They implement this architecture as a GAN that maps textual features to pixel‑level image generation. The model successfully generates plausible images of birds and flowers from detailed text descriptions.

Abstract

Automatic synthesis of realistic images from text would be interesting and useful, but current AI systems are still far from this goal. However, in recent years generic and powerful recurrent neural network architectures have been developed to learn discriminative text feature representations. Meanwhile, deep convolutional generative adversarial networks (GANs) have begun to generate highly compelling images of specific categories, such as faces, album covers, and room interiors. In this work, we develop a novel deep architecture and GAN formulation to effectively bridge these advances in text and image model- ing, translating visual concepts from characters to pixels. We demonstrate the capability of our model to generate plausible images of birds and flowers from detailed text descriptions.