Face detection, pose estimation, and landmark localization in the wild

TLDR

The paper proposes a unified model for face detection, pose estimation, and landmark localization in cluttered real‑world images. The model uses a mixture‑of‑trees framework with shared parts, modeling each landmark as a part and employing global mixtures to capture viewpoint‑induced topological changes. Tree‑structured models capture global elastic deformation efficiently, and extensive benchmark results—including a new in‑the‑wild dataset—demonstrate that the proposed system surpasses state‑of‑the‑art performance and competes favorably with commercial systems trained on billions of examples.

Abstract

We present a unified model for face detection, pose estimation, and landmark estimation in real-world, cluttered images. Our model is based on a mixtures of trees with a shared pool of parts; we model every facial landmark as a part and use global mixtures to capture topological changes due to viewpoint. We show that tree-structured models are surprisingly effective at capturing global elastic deformation, while being easy to optimize unlike dense graph structures. We present extensive results on standard face benchmarks, as well as a new "in the wild" annotated dataset, that suggests our system advances the state-of-the-art, sometimes considerably, for all three tasks. Though our model is modestly trained with hundreds of faces, it compares favorably to commercial systems trained with billions of examples (such as Google Picasa and face.com).