Poselets: Body part detectors trained using 3D human pose annotations

TLDR

The authors introduce poselets—novel body part detectors defined by 3D pose—to simplify detection, segmentation, and pose estimation of people in images, proposing that poselets should be easily discoverable and enable accurate 3D person localization. They constructed the H3D dataset of 2D images with inferred 3D joint annotations and used it to train linear SVM poselet detectors via a data‑driven search that clusters poselets in joint‑configuration and image‑appearance space, then applied the detectors in a multiscale scan to produce intermediate features for higher‑level tasks. The poselet detectors accurately identify body parts such as faces, heads, shoulders, and torsos, enabling precise keypoint localization, and achieve state‑of‑the‑art people detection on PASCAL VOC 2007 and other benchmarks, with the dataset and model parameters released publicly.

Abstract

We address the classic problems of detection, segmentation and pose estimation of people in images with a novel definition of a part, a poselet. We postulate two criteria (1) It should be easy to find a poselet given an input image (2) it should be easy to localize the 3D configuration of the person conditioned on the detection of a poselet. To permit this we have built a new dataset, H3D, of annotations of humans in 2D photographs with 3D joint information, inferred using anthropometric constraints. This enables us to implement a data-driven search procedure for finding poselets that are tightly clustered in both 3D joint configuration space as well as 2D image appearance. The algorithm discovers poselets that correspond to frontal and profile faces, pedestrians, head and shoulder views, among others. Each poselet provides examples for training a linear SVM classifier which can then be run over the image in a multiscale scanning mode. The outputs of these poselet detectors can be thought of as an intermediate layer of nodes, on top of which one can run a second layer of classification or regression. We show how this permits detection and localization of torsos or keypoints such as left shoulder, nose, etc. Experimental results show that we obtain state of the art performance on people detection in the PASCAL VOC 2007 challenge, among other datasets. We are making publicly available both the H3D dataset as well as the poselet parameters for use by other researchers.