Face recognition based on fitting a 3D morphable model

TLDR

The study proposes a face recognition method that works across pose and illumination variations, from frontal to profile views. The method fits a statistical 3D morphable face model to single images by simulating image formation, estimating 3D shape and texture from textured 3D scans, and representing faces with model parameters. Experiments on 4,488 CMU‑PIE and 1,940 FERET images demonstrate the approach’s effectiveness.

Abstract

This paper presents a method for face recognition across variations in pose, ranging from frontal to profile views, and across a wide range of illuminations, including cast shadows and specular reflections. To account for these variations, the algorithm simulates the process of image formation in 3D space, using computer graphics, and it estimates 3D shape and texture of faces from single images. The estimate is achieved by fitting a statistical, morphable model of 3D faces to images. The model is learned from a set of textured 3D scans of heads. We describe the construction of the morphable model, an algorithm to fit the model to images, and a framework for face identification. In this framework, faces are represented by model parameters for 3D shape and texture. We present results obtained with 4,488 images from the publicly available CMU-PIE database and 1,940 images from the FERET database.