Face Image Illumination Processing Based on Generative Adversarial Nets

Abstract

It is a well-known fact that the variations in illumination could seriously affect the performance of 2D face analysis algorithms, such as face landmarking and face recognition. Unfortunately, the illumination condition is usually uncontrolled and unpredictable in most practical applications. Numerous methods have been developed to tackle this problem but the results is poor, especially for images with extreme lighting condition. Furthermore, most traditional illumination processing methods only demonstrate on grayscale images and require strict alignment of face images, resulting in limited applications in real world. In this paper, we proposed to reformulate the face image illumination processing problem as a style translation task with a Generative Adversarial Network (GAN). The key insight is to use the powerful mapping ability of GAN between two domains without knowing their true distributions. In this new sight, we developed a new multi-scale dual discriminate nets and employed multi-scale adversarial learning for visually realistic illumination processing. Advocating the use of the insights from traditional method, we also use reconstruction learning and add two new loss items of image quality assessment to enforce the preservation of all other illumination excluding details on the generated image. Experiments on CMU Multi-PIE and FRGC datasets show that our method can obtain promising illumination normalization results and preserve a superior visual quality.