Fusing Spatiotemporal Features and Joints for 3D Action Recognition

Abstract

We present a novel approach to 3D human action recognition based on a feature-level fusion of spatiotemporal features and skeleton joints. First, 3D interest points detection and local feature description are performed to extract spatiotemporal motion information. Then the frame difference and pairwise distances of skeleton joint positions are computed to characterize the spatial information of the joints in 3D space. These two features are complementary to each other. A fusion scheme is then proposed to combine them effectively based on the random forests method. The proposed approach is validated on three challenging 3D action datasets for human action recognition. Experimental results show that the proposed approach outperforms the state-of-the-art methods on all three datasets.