Recognizing realistic actions from videos “in the wild”

TLDR

Unconstrained “in‑the‑wild” videos are plentiful, yet action recognition is difficult because of camera motion, clutter, and variable object appearance, making reliable feature extraction challenging. This paper proposes a systematic framework for recognizing realistic actions from such unconstrained videos. The framework extracts motion and static features, prunes noise using motion statistics and PageRank, groups related features with a divisive information‑theoretic algorithm, and fuses them with AdaBoost for robust recognition. Experiments on KTH and an 11‑category YouTube/personal dataset show impressive action recognition and localization performance.

Abstract

In this paper, we present a systematic framework for recognizing realistic actions from videos “in the wild.” Such unconstrained videos are abundant in personal collections as well as on the web. Recognizing action from such videos has not been addressed extensively, primarily due to the tremendous variations that result from camera motion, background clutter, changes in object appearance, and scale, etc. The main challenge is how to extract reliable and informative features from the unconstrained videos. We extract both motion and static features from the videos. Since the raw features of both types are dense yet noisy, we propose strategies to prune these features. We use motion statistics to acquire stable motion features and clean static features. Furthermore, PageRank is used to mine the most informative static features. In order to further construct compact yet discriminative visual vocabularies, a divisive information-theoretic algorithm is employed to group semantically related features. Finally, AdaBoost is chosen to integrate all the heterogeneous yet complementary features for recognition. We have tested the framework on the KTH dataset and our own dataset consisting of 11 categories of actions collected from YouTube and personal videos, and have obtained impressive results for action recognition and action localization.