Exchanging Faces in Images

TLDR

Face swapping with traditional tools requires matching viewpoint and illumination, which is rarely available for arbitrary image pairs. The authors introduce a system that can exchange faces across large differences in viewpoint and illumination. The system estimates 3D shape, texture, pose, and lighting from single images using an algorithm, requiring only about seven feature‑point clicks and a hairline mark. The method enables image processing, virtual hairstyle try‑on, and face recognition, and provides an abstract representation separating identity from imaging conditions. Categories and Subject Descriptors (according to ACM CCS): I.3.7 [Computer Graphics]: Animation.

Abstract

Abstract Pasting somebody's face into an existing image with traditional photo retouching and digital image processing tools has only been possible if both images show the face from the same viewpoint and with the same illumination. However, this is rarely the case for given pairs of images. We present a system that exchanges faces across large differences in viewpoint and illumination. It is based on an algorithm that estimates 3D shape and texture along with all relevant scene parameters, such as pose and lighting, from single images. Manual interaction is reduced to clicking on a set of about 7 feature points, and marking the hairline in the target image. The system can be used for image processing, virtual try‐on of hairstyles, and face recognition. By separating face identity from imaging conditions, our approach provides an abstract representation of images and a novel, high‐level tool for image manipulation. Categories and Subject Descriptors (according to ACM CCS): I.3.7 [Computer Graphics]: Animation