Detecting unusual activity in video

TLDR

The study proposes an unsupervised method for detecting unusual activity in large video collections by establishing a correspondence between simple feature prototypes and video segments, inspired by document‑keyword analysis. The method segments videos into equal‑length intervals, clusters extracted features into prototypes, and builds a prototype‑segment co‑occurrence matrix without using complex activity models or supervised feature selection. The authors demonstrate that the correspondence problem can be solved by co‑embedding prototypes and segments in Euclidean space, provide a globally optimal algorithm, and confirm its effectiveness through experiments on real‑world videos.

Abstract

We present an unsupervised technique for detecting unusual activity in a large video set using many simple features. No complex activity models and no supervised feature selections are used. We divide the video into equal length segments and classify the extracted features into prototypes, from which a prototype-segment co-occurrence matrix is computed. Motivated by a similar problem in document-keyword analysis, we seek a correspondence relationship between prototypes and video segments which satisfies the transitive closure constraint. We show that an important sub-family of correspondence functions can be reduced to co-embedding prototypes and segments to N-D Euclidean space. We prove that an efficient, globally optimal algorithm exists for the co-embedding problem. Experiments on various real-life videos have validated our approach.