Real-Time Continuous Human Rehabilitation Action Recognition using OpenPose and FCN

Abstract

The action recognition of human rehabilitation movement in the home scene plays a positive role in promoting the rehabilitation process of patients, we present an efficient approach for real-time continuous human rehabilitation action recognition using OpenPose and FCN. The proposed method first fuses OpenPose with Kalman filter to track human targets and generate the 2D poses action sequences from the RGB videos stream. Then we extract the segmented action sequence by sliding the window and convert the rectangular coordinates to relative coordinates from each frame of the human skeleton. We design a 1D fully convolutional network to extract spatial-temporal characteristics and classify actions. The experimental results show that the method has strong adaptability to the interference of activity background, human body wearing and irrelevant personnel, and it can identify continuous rehabilitation actions online with an accuracy rate of 85.6%. At the same time, the model running speed on GTX1060 is up to 18.14 FPS which has certain application value in the home rehabilitation training scene.