A compact pairwise trajectory representation for action recognition

Abstract

Dense trajectories are widely used in human action recognition. However, the relationships among trajectories are rarely exploited and a large mount of useful information is missing. In this paper, we propose a novel approach to employ the space-time relationships between different trajectories for action recognition. In our approach, each trajectory is paired up with several neighbors which are spatially and temporally close to it. A GMM (Gaussian Mixture Model) is then trained and Fisher Vector is employed to quantify the pairwise trajectories. In this way, the local spatial and temporal structure information around each trajectory is explored for feature representation, which improves the discriminative ability of the features. The experimental results on several benchmark datasets show that our pairwise trajectory representation outperforms the state-of-the-art approaches.