Trajectory Space Factorization for Deep Video-Based 3D Human Pose\n Estimation

Abstract

Existing deep learning approaches on 3d human pose estimation for videos are\neither based on Recurrent or Convolutional Neural Networks (RNNs or CNNs).\nHowever, RNN-based frameworks can only tackle sequences with limited frames\nbecause sequential models are sensitive to bad frames and tend to drift over\nlong sequences. Although existing CNN-based temporal frameworks attempt to\naddress the sensitivity and drift problems by concurrently processing all input\nframes in the sequence, the existing state-of-the-art CNN-based framework is\nlimited to 3d pose estimation of a single frame from a sequential input. In\nthis paper, we propose a deep learning-based framework that utilizes matrix\nfactorization for sequential 3d human poses estimation. Our approach processes\nall input frames concurrently to avoid the sensitivity and drift problems, and\nyet outputs the 3d pose estimates for every frame in the input sequence. More\nspecifically, the 3d poses in all frames are represented as a motion matrix\nfactorized into a trajectory bases matrix and a trajectory coefficient matrix.\nThe trajectory bases matrix is precomputed from matrix factorization approaches\nsuch as Singular Value Decomposition (SVD) or Discrete Cosine Transform (DCT),\nand the problem of sequential 3d pose estimation is reduced to training a deep\nnetwork to regress the trajectory coefficient matrix. We demonstrate the\neffectiveness of our framework on long sequences by achieving state-of-the-art\nperformances on multiple benchmark datasets. Our source code is available at:\nhttps://github.com/jiahaoLjh/trajectory-pose-3d.\n