Human Action Recognition Based on Selected Spatio-Temporal Features via Bidirectional LSTM

Abstract

Recently, many deep convolutional networks approaches have been proposed for human action recognition. The challenge is to capture complementary information from still and motion frames. In contrast to previous models, which use holistic clips to model spatial information and traditional temporal information for sequential processing, in this paper, we propose a novel framework that can select the discriminative part in the spatial dimension and enrich the modeling action of motion in the temporal dimension. We utilize part selection within clips and consider the bidirectional temporal information when modeling the temporal pattern using multiple layers of a long short-term memory framework, which can learn compositional representations in space and time. Our results are evaluated on the standard benchmarks UCF101 and HMDB51 and show that the proposed architecture achieves state-of-the-art results.