Forward-Backward Error: Automatic Detection of Tracking Failures

TLDR

The paper proposes a novel method for detecting tracking failures and introduces the Median Flow tracker. The method detects failures by computing a Forward‑Backward error from forward and backward tracking trajectories and measuring their discrepancies. The approach reliably detects failures, complements NCC, and achieves state‑of‑the‑art performance on challenging benchmarks with non‑rigid objects.

Abstract

This paper proposes a novel method for tracking failure detection. The detection is based on the Forward-Backward error, i.e. the tracking is performed forward and backward in time and the discrepancies between these two trajectories are measured. We demonstrate that the proposed error enables reliable detection of tracking failures and selection of reliable trajectories in video sequences. We demonstrate that the approach is complementary to commonly used normalized cross-correlation (NCC). Based on the error, we propose a novel object tracker called Median Flow. State-of-the-art performance is achieved on challenging benchmark video sequences which include non-rigid objects.