Pose Tutor: An Explainable System for Pose Correction in the Wild

Abstract

Under the new norm of working from home, demand for fitness from home is on the rise. Different exercise forms solve different fitness needs for different people. Yoga gives flexibility and relieves stress. Pilates strengthens the muscles. Kung Fu brings balance. It is not feasible for everyone to hire a personal trainer. In this paper, we develop Pose Tutor, an AI-based explainable pose recognition and correction system. Pose Tutor combines vision and pose skeleton models in a novel coarse-to-fine framework to obtain pose class predictions. An angle-likelihood mechanism is used to explain which human joints maximally caused the pose class predictions and also correct any wrongly formed joints. Even without keypoint level training, Pose Tutor shows promising results on Yoga-82, Pilates-32, and Kungfu-7 datasets. Additionally, user studies conducted with multiple domain experts validate the explanations provided by our framework.