3D Face Reconstruction from Video Using 3D Morphable Model and Silhouette

Abstract

Capitalizing on the advances of 3D face reconstruction from computer graphics and computer vision, this work presents a system to enable precise and robust 3D facial reconstruction from video in order to improve current approaches for doctor-patient communication tools. The reconstructed face can be used for emulation of aesthetic procedures in 3D. The proposed 3D face reconstruction algorithm uses a statistical shape model as well as information from facial landmarks and silhouette to iteratively model the face of a subject moving the face from front to left. The presented 3D active shape model approach enables the spatio-temporal tracking of facial landmarks and pose estimation based on a few initial facial landmarks defined in the first frame. Silhouette information extracted from key frames allows for better face reconstruction. The proposed methods were rigorously evaluated with experiments using real and artificial data. Results showed that the proposed method can detect facial landmarks with an approximate median error of 5% intraocular distance, can estimate pose with a median error below 5 degrees, and that silhouette information can improve reconstruction accuracy in all facial regions, specially on the cheeks.