Regressing Robust and Discriminative 3D Morphable Models with a Very Deep Neural Network

TLDR

The 3D shapes of faces are highly discriminative, yet they are rarely employed for face recognition and only under controlled viewing conditions. This work proposes a robust approach to regress discriminative 3D morphable face models from single images in unconstrained settings. A very deep convolutional neural network directly predicts 3DMM shape and texture parameters from a photo, and a data‑generation method supplies abundant labeled examples for training. The resulting 3D estimates surpass state‑of‑the‑art accuracy on the MICC dataset and enable competitive face recognition on LFW, YTF, and IJB‑A using 3D shape representations.

Abstract

The 3D shapes of faces are well known to be discriminative. Yet despite this, they are rarely used for face recognition and always under controlled viewing conditions. We claim that this is a symptom of a serious but often overlooked problem with existing methods for single view 3D face reconstruction: when applied in the wild, their 3D estimates are either unstable and change for different photos of the same subject or they are over-regularized and generic. In response, we describe a robust method for regressing discriminative 3D morphable face models (3DMM). We use a convolutional neural network (CNN) to regress 3DMM shape and texture parameters directly from an input photo. We overcome the shortage of training data required for this purpose by offering a method for generating huge numbers of labeled examples. The 3D estimates produced by our CNN surpass state of the art accuracy on the MICC data set. Coupled with a 3D-3D face matching pipeline, we show the first competitive face recognition results on the LFW, YTF and IJB-A benchmarks using 3D face shapes as representations, rather than the opaque deep feature vectors used by other modern systems.