Multispectral pedestrian detection: Benchmark dataset and baseline

TLDR

Pedestrian detection research has focused on color images, but thermal data can improve detection in low‑light conditions. The authors aim to create a multispectral pedestrian dataset and develop a multispectral ACF detector. They built a large, densely annotated color‑thermal dataset using beam‑splitter hardware and extended the ACF method to process both channels simultaneously. The multispectral ACF lowers ACF’s miss rate by 15%, marking a significant improvement in pedestrian detection.

Abstract

With the increasing interest in pedestrian detection, pedestrian datasets have also been the subject of research in the past decades. However, most existing datasets focus on a color channel, while a thermal channel is helpful for detection even in a dark environment. With this in mind, we propose a multispectral pedestrian dataset which provides well aligned color-thermal image pairs, captured by beam splitter-based special hardware. The color-thermal dataset is as large as previous color-based datasets and provides dense annotations including temporal correspondences. With this dataset, we introduce multispectral ACF, which is an extension of aggregated channel features (ACF) to simultaneously handle color-thermal image pairs. Multi-spectral ACF reduces the average miss rate of ACF by 15%, and achieves another breakthrough in the pedestrian detection task.