An Investigation of Skeleton-Based Optical Flow-Guided Features for 3D Action Recognition Using a Multi-Stream CNN Model

Abstract

Deep learning-based techniques have recently been found significantly effective for handling skeleton-based action recognition tasks. It was observed that modeling the spatiotemporal variations is the key to effective skeleton-based action recognition approaches. This work proposes an easy and yet effective method for encoding different geometric relational features into static color texture images. Collectively, we refer to these features as skeletal optical flow-guided features. The temporal variations of different features are converted into the color variations of their corresponding images. Then, a multi-stream CNN model is employed to pick up the discriminating patterns that exist in the converted images for subsequent classification. Experimental results demonstrate that our proposed geometric relational features and framework can achieve competitive performances on both MSR Action 3D and NTU RGB+D datasets.