Looking at Vehicles on the Road: A Survey of Vision-Based Vehicle Detection, Tracking, and Behavior Analysis

TLDR

Vision‑based surround perception has matured over the past decade. The paper surveys recent literature on on‑road vision‑based vehicle detection, tracking, and behavior analysis, contextualizing it within sensor‑based surround perception. The review examines vehicle detection advances with monocular, stereo, and active sensor‑vision fusion, discusses vision‑based tracking in monocular and stereo domains using filtering, estimation, and dynamical models, and explores emerging intelligent vehicle research that leverages spatiotemporal measurements and trajectories to characterize on‑road behavior. The authors discuss the state of the art, outline common performance metrics and benchmarks, and suggest future research directions.

Abstract

This paper provides a review of the literature in on-road vision-based vehicle detection, tracking, and behavior understanding. Over the past decade, vision-based surround perception has progressed from its infancy into maturity. We provide a survey of recent works in the literature, placing vision-based vehicle detection in the context of sensor-based on-road surround analysis. We detail advances in vehicle detection, discussing monocular, stereo vision, and active sensor-vision fusion for on-road vehicle detection. We discuss vision-based vehicle tracking in the monocular and stereo-vision domains, analyzing filtering, estimation, and dynamical models. We discuss the nascent branch of intelligent vehicles research concerned with utilizing spatiotemporal measurements, trajectories, and various features to characterize on-road behavior. We provide a discussion on the state of the art, detail common performance metrics and benchmarks, and provide perspective on future research directions in the field.