Disentangled Representation Learning GAN for Pose-Invariant Face Recognition

TLDR

Pose discrepancy between face images is a key challenge for face recognition. The authors propose jointly performing pose frontalization and representation learning to exploit mutual benefits. DR‑GAN uses an encoder‑decoder generator that learns a generative and discriminative representation, explicitly disentangles pose through a pose code and discriminator pose estimation, and can accept one or multiple images to produce a unified representation and synthetic faces. Experiments on controlled and in‑the‑wild datasets show DR‑GAN outperforms state‑of‑the‑art methods.

Abstract

The large pose discrepancy between two face images is one of the key challenges in face recognition. Conventional approaches for pose-invariant face recognition either perform face frontalization on, or learn a pose-invariant representation from, a non-frontal face image. We argue that it is more desirable to perform both tasks jointly to allow them to leverage each other. To this end, this paper proposes Disentangled Representation learning-Generative Adversarial Network (DR-GAN) with three distinct novelties. First, the encoder-decoder structure of the generator allows DR-GAN to learn a generative and discriminative representation, in addition to image synthesis. Second, this representation is explicitly disentangled from other face variations such as pose, through the pose code provided to the decoder and pose estimation in the discriminator. Third, DR-GAN can take one or multiple images as the input, and generate one unified representation along with an arbitrary number of synthetic images. Quantitative and qualitative evaluation on both controlled and in-the-wild databases demonstrate the superiority of DR-GAN over the state of the art.