Mining actionlet ensemble for action recognition with depth cameras

TLDR

Depth cameras enable action recognition but introduce noise and occlusion that increase intra‑class variation. The study proposes an actionlet ensemble model with novel depth‑suitable features to capture intra‑class variance. The model uses noise‑robust, translation‑ and time‑invariant features to characterize motion and interactions, and is evaluated on two depth‑camera datasets and a MoCap set. Experiments demonstrate that the approach outperforms state‑of‑the‑art algorithms.

Abstract

Human action recognition is an important yet challenging task. The recently developed commodity depth sensors open up new possibilities of dealing with this problem but also present some unique challenges. The depth maps captured by the depth cameras are very noisy and the 3D positions of the tracked joints may be completely wrong if serious occlusions occur, which increases the intra-class variations in the actions. In this paper, an actionlet ensemble model is learnt to represent each action and to capture the intra-class variance. In addition, novel features that are suitable for depth data are proposed. They are robust to noise, invariant to translational and temporal misalignments, and capable of characterizing both the human motion and the human-object interactions. The proposed approach is evaluated on two challenging action recognition datasets captured by commodity depth cameras, and another dataset captured by a MoCap system. The experimental evaluations show that the proposed approach achieves superior performance to the state of the art algorithms.