CP-GAN: A Cross-Pose Profile Face Frontalization Boosting Pose-Invariant Face Recognition

Abstract

Pose variant or self-occlusion is one of the open issues which severely degrades the performance of pose-invariant face recognition (PIFR). Existing solutions to PIFR either have undesirable generalization based on challenging pose normalization or are complicated for implement on account of deep neural network. To relieve the impact of ill-pose on PIFR, we have proposed Cross-Pose Generative Adversarial Networks(CP-GAN) to frontalize the profile face with unaltered identity by learning the mapping between the profile and frontal faces in image space. The generator is an encoder-decoder U-net, and generate frontal face image by fusing multiple profile images to achieve a better performance in PIFR. The siamese discriminative network attends to extract the deep representations of the generated frontal face and the ground truth without introducing extra networks in verification and recognition. Besides the implementable architecture, this problem is well alleviated by introducing a combination of adversarial loss for both the generator and the discriminator, symmetry loss, patch-wise loss, and identity loss guiding an identity reserving property of the generated frontal view. Quantitative and qualitative evaluation on both controlled and in-the-wild datasets attest that the solution we proposed to PIFR presents satisfactory perceptual results and transcends state-of-the-art methods on ill-pose face recognition.