Pedestrian detection: A benchmark

TLDR

Pedestrian detection is a key problem in computer vision with applications in robotics, surveillance, and automotive safety, and progress has been driven by challenging public datasets. The authors introduce the Caltech Pedestrian Dataset, two orders of magnitude larger than existing datasets, to accelerate innovation in pedestrian detection. The dataset comprises richly annotated video from a moving vehicle with low‑resolution, frequently occluded pedestrians, and the authors propose improved evaluation metrics and benchmark multiple detection systems. The study shows that common per‑window metrics are flawed, provides an unbiased benchmark of state‑of‑the‑art detectors, and identifies failure modes that suggest future research directions.

Abstract

Pedestrian detection is a key problem in computer vision, with several applications including robotics, surveillance and automotive safety. Much of the progress of the past few years has been driven by the availability of challenging public datasets. To continue the rapid rate of innovation, we introduce the Caltech Pedestrian Dataset, which is two orders of magnitude larger than existing datasets. The dataset contains richly annotated video, recorded from a moving vehicle, with challenging images of low resolution and frequently occluded people. We propose improved evaluation metrics, demonstrating that commonly used per-window measures are flawed and can fail to predict performance on full images. We also benchmark several promising detection systems, providing an overview of state-of-the-art performance and a direct, unbiased comparison of existing methods. Finally, by analyzing common failure cases, we help identify future research directions for the field.