Event-based analysis of video

TLDR

Dynamic events are long‑term temporal objects defined by multi‑scale spatio‑temporal features. The study designs a simple statistical distance measure between video sequences of varying lengths based on behavioral content. The non‑parametric distance measure is applied to isolate and cluster events in long continuous videos without prior knowledge of event types, models, or temporal extent, and can be refined with multiple example clips to improve detection and clustering. Clustering produces temporal segmentation of long videos into event‑consistent sub‑sequences and groups them into event‑consistent clusters.

Abstract

Dynamic events can be regarded as long-term temporal objects, which are characterized by spatio-temporal features at multiple temporal scales. Based on this, we design a simple statistical distance measure between video sequences (possibly of different lengths) based on their behavioral content. This measure is non-parametric and can thus handle a wide range of dynamic events. We use this measure for isolating and clustering events within long continuous video sequences. This is done without prior knowledge of the types of events, their models, or their temporal extent. An outcome of such a clustering process is a temporal segmentation of long video sequences into event-consistent sub-sequences, and their grouping into event-consistent clusters. Our event representation and associated distance measure can also be used for event-based indexing into long video sequences, even when only one short example-clip is available. However, when multiple example-clips of the same event are available (either as a result of the clustering process, or given manually), these can be used to refine the event representation, the associated distance measure, and accordingly the quality of the detection and clustering process.