Cooperative Sensing and Wearable Computing for Sequential Hand Gesture Recognition

TLDR

Hand gesture recognition is a key area of human‑computer interaction, yet vision‑based approaches suffer from fixed lab settings, lighting sensitivity, and background clutter. The study develops a novel hand‑gesture recognition system that fuses a wearable armband and a smart glove with customizable pressure sensor arrays to detect sequential hand gestures. The system employs an LSTM model trained on inertial, electromyographic, and finger‑palm pressure data collected from a database of ten sequential gestures recorded from ten participants. The LSTM achieved outstanding classification accuracy, indicating promising potential for sequential hand‑gesture recognition in HCI.

Abstract

Hand gestures recognition (HGR) has been considered as one of the crucial research fields of human-computer interaction (HCI). Computer vision is a very active research field in the HGR, traditional vision-based methods, which used camera and ultrasonic/optical sensor to collect the videos or images of the hand gestures shown by participants, have some limitations, such as fixed in-lab location, complex lighting conditions, and cluttered backgrounds. In order to provide new approaches, we described the development of a novel hand gesture recognition system that combined wearable armband and smart glove made by customizable pressure sensor arrays to detect sequential hand gestures. A deep learning technique long short-term memory (LSTM) algorithm had been computed to build an effective model to classify hand gestures by training and testing the collected inertial measurement unit (IMU), electromyographic (EMG), and finger and palm's pressure data. Furthermore, we built a relatively large database of ten sequential hand gestures consisted by five dynamic gestures and five air gestures collected from ten participants. Our experimental results showed an outstanding classification performance of the proposed LSTM algorithm. These findings have promising implications for sequential hand gesture recognition and the HCI research status.