You'll never walk alone: Modeling social behavior for multi-target tracking

TLDR

Object tracking relies on dynamic models, yet traditional approaches ignore social interactions and scene context, causing late collision resolution and suboptimal predictions in crowded scenes. The study proposes a dynamic social behavior model for multi‑target tracking inspired by crowd simulation. The model is trained on bird‑eye view videos from busy locations and used as a motion model for vehicle‑mounted multi‑person tracking. Experiments demonstrate that incorporating social interactions and scene knowledge improves tracking accuracy, particularly during occlusions.

Abstract

Object tracking typically relies on a dynamic model to predict the object's location from its past trajectory. In crowded scenarios a strong dynamic model is particularly important, because more accurate predictions allow for smaller search regions, which greatly simplifies data association. Traditional dynamic models predict the location for each target solely based on its own history, without taking into account the remaining scene objects. Collisions are resolved only when they happen. Such an approach ignores important aspects of human behavior: people are driven by their future destination, take into account their environment, anticipate collisions, and adjust their trajectories at an early stage in order to avoid them. In this work, we introduce a model of dynamic social behavior, inspired by models developed for crowd simulation. The model is trained with videos recorded from birds-eye view at busy locations, and applied as a motion model for multi-people tracking from a vehicle-mounted camera. Experiments on real sequences show that accounting for social interactions and scene knowledge improves tracking performance, especially during occlusions.