A Survey on Visual Surveillance of Object Motion and Behaviors

TLDR

Visual surveillance of dynamic scenes, especially humans and vehicles, is a highly active research area with applications ranging from access control to anomaly detection. The paper reviews recent developments and general strategies across all stages of visual surveillance. The authors analyze future research directions, including occlusion handling, combined 2D/3D tracking, motion–biometric fusion, anomaly detection, behavior prediction, video retrieval, natural language description, multi‑sensor fusion, and remote surveillance.

Abstract

Visual surveillance in dynamic scenes, especially for humans and vehicles, is currently one of the most active research topics in computer vision. It has a wide spectrum of promising applications, including access control in special areas, human identification at a distance, crowd flux statistics and congestion analysis, detection of anomalous behaviors, and interactive surveillance using multiple cameras, etc. In general, the processing framework of visual surveillance in dynamic scenes includes the following stages: modeling of environments, detection of motion, classification of moving objects, tracking, understanding and description of behaviors, human identification, and fusion of data from multiple cameras. We review recent developments and general strategies of all these stages. Finally, we analyze possible research directions, e.g., occlusion handling, a combination of twoand three-dimensional tracking, a combination of motion analysis and biometrics, anomaly detection and behavior prediction, content-based retrieval of surveillance videos, behavior understanding and natural language description, fusion of information from multiple sensors, and remote surveillance.