Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition

TLDR

Human skeleton dynamics provide key information for action recognition, yet conventional methods rely on hand‑crafted parts or traversal rules, limiting expressiveness and generalization. The authors propose Spatial‑Temporal Graph Convolutional Networks (ST‑GCN) to automatically learn spatial and temporal patterns from skeleton data, surpassing prior hand‑crafted approaches. ST‑GCN applies graph convolutions across spatial joints and temporal frames to capture dynamic skeleton patterns. The model achieves greater expressiveness, stronger generalization, and substantial performance gains on Kinetics and NTU‑RGBD compared to mainstream methods.

Abstract

Dynamics of human body skeletons convey significant information for human action recognition. Conventional approaches for modeling skeletons usually rely on hand-crafted parts or traversal rules, thus resulting in limited expressive power and difficulties of generalization. In this work, we propose a novel model of dynamic skeletons called Spatial-Temporal Graph Convolutional Networks (ST-GCN), which moves beyond the limitations of previous methods by automatically learning both the spatial and temporal patterns from data. This formulation not only leads to greater expressive power but also stronger generalization capability. On two large datasets, Kinetics and NTU-RGBD, it achieves substantial improvements over mainstream methods.