DCS-CTN: Subtle Gesture Recognition Based on TD-CNN-Transformer via Millimeter-Wave Radar

Abstract

Gesture recognition has been a hot research topic in human–computer interaction, since contactless gesture recognition will provide increasing applications in many fields. Millimeter-wave (mmWave) radar well serves this technology because of its high accuracy, easy integration, and strong anti-jamming ability in moving object detection. However, it is still challenging to meet the requirement of high precision in subtle gesture recognition based on traditional methods via point cloud or Range-Doppler heat map of mmWave radar. Considering the raw data from mmWave radar with more information, such as phase, we propose a system that uses the constructed mmWave radar data cube sequence and timedistributed-CNN-transformer network (CTN), called DCS-CTN system, to get higher hand gesture recognition accuracy. In this system, we introduce a time-distributed wrapper (TD) and convolutional neural network (CNN) to extract local features of the data cube sequence, a position encoder to retain time information of the sequence, and a transformer network to get global features of the sequence. The experiments results show that this system can achieve hand gesture recognition accuracy of 99.75%, which is significantly higher than the other traditional approaches.