Tracking Without Bells and Whistles

TLDR

Tracking multiple objects in video sequences is challenging due to re‑identification, motion prediction, and occlusion issues. The authors aim to demonstrate that a simple detector‑based tracker can achieve state‑of‑the‑art multi‑object tracking without task‑specific training, by extending it with basic re‑identification and camera motion compensation. They convert a detector into a Tracktor by using its bounding‑box regression to predict next‑frame positions and analyze its performance against other trackers. Tracktor achieves superior performance on three benchmarks, surpassing dedicated trackers, yet all methods, including Tracktor, still struggle with small, occluded objects or missing detections, highlighting unresolved challenges.

Abstract

The problem of tracking multiple objects in a video sequence poses several challenging tasks. For tracking-by-detection, these include object re-identification, motion prediction and dealing with occlusions. We present a tracker (without bells and whistles) that accomplishes tracking without specifically targeting any of these tasks, in particular, we perform no training or optimization on tracking data. To this end, we exploit the bounding box regression of an object detector to predict the position of an object in the next frame, thereby converting a detector into a Tracktor. We demonstrate the potential of Tracktor and provide a new state-of-the-art on three multi-object tracking benchmarks by extending it with a straightforward re-identification and camera motion compensation. We then perform an analysis on the performance and failure cases of several state-of-the-art tracking methods in comparison to our Tracktor. Surprisingly, none of the dedicated tracking methods are considerably better in dealing with complex tracking scenarios, namely, small and occluded objects or missing detections. However, our approach tackles most of the easy tracking scenarios. Therefore, we motivate our approach as a new tracking paradigm and point out promising future research directions. Overall, Tracktor yields superior tracking performance than any current tracking method and our analysis exposes remaining and unsolved tracking challenges to inspire future research directions.