Real-Time Human Movement Retrieval and Assessment With Kinect Sensor

TLDR

Vision‑based posture estimation is greatly facilitated by commercial depth cameras like Microsoft Kinect, yet linking estimated postures to meaningful human movements remains challenging, especially for movement assessment in healthcare exercise learning. The study proposes an action tutor system that lets users retrieve exemplar action videos and obtain instant motion instructions while learning with a Kinect. The system operates in two stages: a retrieval stage that uses nonlinear time‑warping to find video segments similar to the user’s query movement, and a learning stage that presents static and dynamic motion assessments based on the selected exemplar. Experiments on ten action types demonstrate that the proposed action descriptor is effective for video retrieval and that the tutor system significantly aids users in learning action movements.

Abstract

The difficulty of vision-based posture estimation is greatly decreased with the aid of commercial depth camera, such as Microsoft Kinect. However, there is still much to do to bridge the results of human posture estimation and the understanding of human movements. Human movement assessment is an important technique for exercise learning in the field of healthcare. In this paper, we propose an action tutor system which enables the user to interactively retrieve a learning exemplar of the target action movement and to immediately acquire motion instructions while learning it in front of the Kinect. The proposed system is composed of two stages. In the retrieval stage, nonlinear time warping algorithms are designed to retrieve video segments similar to the query movement roughly performed by the user. In the learning stage, the user learns according to the selected video exemplar, and the motion assessment including both static and dynamic differences is presented to the user in a more effective and organized way, helping him/her to perform the action movement correctly. The experiments are conducted on the videos of ten action types, and the results show that the proposed human action descriptor is representative for action video retrieval and the tutor system can effectively help the user while learning action movements.