Semantically Decomposing the Latent Spaces of Generative Adversarial\n Networks

Abstract

We propose a new algorithm for training generative adversarial networks that\njointly learns latent codes for both identities (e.g. individual humans) and\nobservations (e.g. specific photographs). By fixing the identity portion of the\nlatent codes, we can generate diverse images of the same subject, and by fixing\nthe observation portion, we can traverse the manifold of subjects while\nmaintaining contingent aspects such as lighting and pose. Our algorithm\nfeatures a pairwise training scheme in which each sample from the generator\nconsists of two images with a common identity code. Corresponding samples from\nthe real dataset consist of two distinct photographs of the same subject. In\norder to fool the discriminator, the generator must produce pairs that are\nphotorealistic, distinct, and appear to depict the same individual. We augment\nboth the DCGAN and BEGAN approaches with Siamese discriminators to facilitate\npairwise training. Experiments with human judges and an off-the-shelf face\nverification system demonstrate our algorithm's ability to generate convincing,\nidentity-matched photographs.\n