Human activity prediction: Early recognition of ongoing activities from streaming videos

TLDR

Human activity prediction infers ongoing actions from videos showing only their onsets, a task especially important for surveillance systems that aim to prevent crimes and dangerous activities. The study aims to enable early recognition of unfinished activities rather than post‑hoc classification of completed ones. The authors formulate activity prediction probabilistically, model activities as integral histograms of spatio‑temporal features, and introduce a dynamic bag‑of‑words method that captures sequential patterns while handling noise. Experiments show that the approach reliably recognizes ongoing activities from streaming videos with high accuracy.

Abstract

In this paper, we present a novel approach of human activity prediction. Human activity prediction is a probabilistic process of inferring ongoing activities from videos only containing onsets (i.e. the beginning part) of the activities. The goal is to enable early recognition of unfinished activities as opposed to the after-the-fact classification of completed activities. Activity prediction methodologies are particularly necessary for surveillance systems which are required to prevent crimes and dangerous activities from occurring. We probabilistically formulate the activity prediction problem, and introduce new methodologies designed for the prediction. We represent an activity as an integral histogram of spatio-temporal features, efficiently modeling how feature distributions change over time. The new recognition methodology named dynamic bag-of-words is developed, which considers sequential nature of human activities while maintaining advantages of the bag-of-words to handle noisy observations. Our experiments confirm that our approach reliably recognizes ongoing activities from streaming videos with a high accuracy.