From few to many: illumination cone models for face recognition under variable lighting and pose

TLDR

The authors propose a generative appearance-based method to recognize human faces under varying lighting and viewpoint. The method models each face as a convex cone of images under all illuminations, reconstructs shape and albedo from a few training images, approximates each pose’s illumination cone with a low-dimensional subspace, and assigns a test image to the nearest cone for recognition. Experiments demonstrate near-perfect recognition accuracy, with only occasional errors under the most extreme lighting directions.

Abstract

We present a generative appearance-based method for recognizing human faces under variation in lighting and viewpoint. Our method exploits the fact that the set of images of an object in fixed pose, but under all possible illumination conditions, is a convex cone in the space of images. Using a small number of training images of each face taken with different lighting directions, the shape and albedo of the face can be reconstructed. In turn, this reconstruction serves as a generative model that can be used to render (or synthesize) images of the face under novel poses and illumination conditions. The pose space is then sampled and, for each pose, the corresponding illumination cone is approximated by a low-dimensional linear subspace whose basis vectors are estimated using the generative model. Our recognition algorithm assigns to a test image the identity of the closest approximated illumination cone. Test results show that the method performs almost without error, except on the most extreme lighting directions.