2023 · 77 citations · 33 references
ComputingStudio Art3D ModelingComputational ArtComputer VisionFace RecognitionImage EditingImage SynthesisEncoder DesignFace Representation LearningFacial Expression SynthesisLatent Space EmbeddingLatent Space ExplorationLatent Space InterpretationRegional Style VariationSwapping
We present a novel paradigm for high-fidelity face swapping that faithfully preserves the desired subtle geometry and texture details. We rethink face swapping from the perspective of fine-grained face editing, i.e., “editing for swapping” (E4S), and propose a framework that is based on the explicit disentanglement of the shape and texture of facial components. Following the E4S principle, our framework enables both global and local swapping of facial features, as well as controlling the amount of partial swapping specified by the user. Furthermore, the E4S paradigm is in-herently capable of handling facial occlusions by means of facial masks. At the core of our system lies a novel Regional GAN Inversion (RGI) method, which allows the explicit disentanglement of shape and texture. It also allows face swapping to be performed in the latent space of Style-GAN. Specifically, we design a multi-scale mask-guided encoder to project the texture of each facial component into regional style codes. We also design a mask-guided injection module to manipulate the feature maps with the style codes. Based on the disentanglement, face swapping is re-formulated as a simplified problem of style and mask swapping. Extensive experiments and comparisons with current state-of-the-art methods demonstrate the superiority of our approach in preserving texture and shape details, as well as working with high resolution images. The project page is https://e4s2022.github.io
33
Adam: A Method for Stochastic Optimization
Diederik P. Kingma, Jimmy Ba · UvA-DARE (University of Amsterdam) · 2014 · 84.8K citations · Full text
Image quality assessment: from error visibility to structural similarity
Zhou Wang, Alan C. Bovik, Hamid R. Sheikh et al. · IEEE Transactions on Image Processing · 2004 · 56K citations
A morphable model for the synthesis of 3D faces
Volker Blanz, Thomas Vetter · 1999 · 4.8K citations · Full text
Dlib-ml: A Machine Learning Toolkit
Davis E. King · 2009 · 2.9K citations
CosFace: Large Margin Cosine Loss for Deep Face Recognition
Hao Wang, Yitong Wang, Zheng Zhou et al. · 2018 · 2.9K citations