Advancing High Fidelity Identity Swapping for Forgery Detection

TLDR

The study evaluates deepfake detection benchmarks and introduces FaceShifter, a novel two‑stage face‑swapping algorithm. FaceShifter generates high‑fidelity swapped faces by fully exploiting target attributes and employs a second synthesis stage, HEAR‑Net, to recover occluded regions in a self‑supervised manner. Experiments show that current deepfake detectors perform poorly against FaceShifter, but the newly developed Face X‑Ray method reliably detects its forged images.

Abstract

In this work, we study various existing benchmarks for deepfake detection researches. In particular, we examine a novel two-stage face swapping algorithm, called FaceShifter, for high fidelity and occlusion aware face swapping. Unlike many existing face swapping works that leverage only limited information from the target image when synthesizing the swapped face, FaceShifter generates the swapped face with high-fidelity by exploiting and integrating the target attributes thoroughly and adaptively. FaceShifter can handle facial occlusions with a second synthesis stage consisting of a Heuristic Error Acknowledging Refinement Network (HEAR-Net), which is trained to recover anomaly regions in a self-supervised way without any manual annotations. Experiments show that existing deepfake detection algorithm performs poorly with FaceShifter, since it achieves advantageous quality over all existing benchmarks. However, our newly developed Face X-Ray method can reliably detect forged images created by FaceShifter.