Evaluation of background subtraction techniques for video surveillance

TLDR

Background subtraction is a key technique for automatic video analysis in surveillance, yet recent evaluations of its methods suffer from significant shortcomings. The study aims to identify the main challenges of background subtraction in surveillance and to introduce a new evaluation dataset with accurate ground truth and shadow masks. Using this dataset, the authors compare the performance of nine background subtraction methods with post‑processing, assessing how well each meets the identified challenges. The comparison yields a precise in‑depth evaluation of the strengths and drawbacks of each method.

Abstract

Background subtraction is one of the key techniques for automatic video analysis, especially in the domain of video surveillance. Although its importance, evaluations of recent background subtraction methods with respect to the challenges of video surveillance suffer from various shortcomings. To address this issue, we first identify the main challenges of background subtraction in the field of video surveillance. We then compare the performance of nine background subtraction methods with post-processing according to their ability to meet those challenges. Therefore, we introduce a new evaluation data set with accurate ground truth annotations and shadow masks. This enables us to provide precise in-depth evaluation of the strengths and drawbacks of background subtraction methods.