Multisensor-Based 3D Gesture Recognition for a Decision-Making Training System

Abstract

This article demonstrates a gesture recognition method using multiple Inertial Measurement Unit (IMU) sensors, which can record acceleration and rotation information for hand joints. The proposed gesture recognition method comprises frequency ConvNet and TemporalNet to extract the representative features within a sliding window of IMU signals for recognizing various types of hand gestures. To validate the proposed gesture recognition method, basketball official referee signals (ORSs), which comprise sixty-five types of gestures including both large motion hand movement and subtle motion hand movement, are utilized as the main recognition task to evaluate the proposed method. The evaluation results reveal the proposed recognition model can achieve convinced performance, which outperforms other existing works. In addition, the satisfied performance of the proposed recognition model encourages us to develop a decision-making training (DMT) system for cultivating basketball referees. The results of subjective evaluations by the recruited 20 participants indicate the training system based on the proposed gestures recognition method can efficiently strengthen the decision-making skills of users.