Behavior Recognition via Sparse Spatio-Temporal Features

TLDR

Sparse, informative feature points make object recognition more manageable and robust to noise and pose variation. This study extends sparse feature point methods to the spatio‑temporal domain, demonstrating that 3D counterparts of 2D detectors are inadequate and proposing an alternative. The authors anchor on the proposed interest points and devise a recognition algorithm using spatio‑temporally windowed data. Recognition results on multiple datasets, including human and rodent behavior, demonstrate the effectiveness of the approach.

Abstract

A common trend in object recognition is to detect and leverage the use of sparse, informative feature points. The use of such features makes the problem more manageable while providing increased robustness to noise and pose variation. In this work we develop an extension of these ideas to the spatio-temporal case. For this purpose, we show that the direct 3D counterparts to commonly used 2D interest point detectors are inadequate, and we propose an alternative. Anchoring off of these interest points, we devise a recognition algorithm based on spatio-temporally windowed data. We present recognition results on a variety of datasets including both human and rodent behavior.