Automatic soccer video analysis and summarization

TLDR

The paper proposes an automatic, computationally efficient framework for analyzing and summarizing soccer videos using cinematic and object‑based features. The framework combines low‑level algorithms for color region, shot boundary, and shot classification with higher‑level algorithms for goal, referee, and penalty‑box detection, producing three summary types—slow‑motion segments, goals, and object‑based classified slow‑motion segments—using cinematic features for speed and object features for accuracy. The framework is efficient, effective, and robust, achieving accurate event detection without unnecessary object‑based computations and validated on more than 13 hours of diverse soccer footage.

Abstract

We propose a fully automatic and computationally efficient framework for analysis and summarization of soccer videos using cinematic and object-based features. The proposed framework includes some novel low-level processing algorithms, such as dominant color region detection, robust shot boundary detection, and shot classification, as well as some higher-level algorithms for goal detection, referee detection, and penalty-box detection. The system can output three types of summaries: i) all slow-motion segments in a game; ii) all goals in a game; iii) slow-motion segments classified according to object-based features. The first two types of summaries are based on cinematic features only for speedy processing, while the summaries of the last type contain higher-level semantics. The proposed framework is efficient, effective, and robust. It is efficient in the sense that there is no need to compute object-based features when cinematic features are sufficient for the detection of certain events, e.g., goals in soccer. It is effective in the sense that the framework can also employ object-based features when needed to increase accuracy (at the expense of more computation). The efficiency, effectiveness, and robustness of the proposed framework are demonstrated over a large data set, consisting of more than 13 hours of soccer video, captured in different countries and under different conditions.