Temporal Segment Neural Networks-Enabled Dynamic Hand-Gesture Recognition for Industrial Cyber-Physical Authentication Systems

Abstract

In recent years, the data security and privacyprotection of human cyber-physical systems (CPSs) in the industrial Internet of Things (IoT) have attracted much research interest. In the meantime, applying human–machine identity authentication mechanism to the CPSs has been considered as a promising solution for secure and privacy-enhancing authentication in industrial IoT. In view of the application limitation of traditional authentication methods, especially during the current COVID-19 virus outbreak, the mainstream face recognition and fingerprint recognition methods are constrained by wearing masks and gloves. Recently, with the flourishing studies of artificial intelligence (AI), many AI-based dynamic gesture recognition methods have been developed for the application of human–computer identity recognition. A temporal segment network (TSN) has been designed with a dual-stream convolutional neural network inside to match the flexibility of gesture transformations. Although it has achieved a success in accuracy, the TSN still has the main shortcomings of the insufficient temporal information fusion and the high cost of optical flow feature extraction. In this article, we propose an enhanced temporal segment network (dubbed as <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$e$</tex-math></inline-formula> -TSN). First, hand skeleton features are used instead of optical flow features. Second, a short-term networks (long short-term memory) is utilized as segmental consensus function to improve the accuracy and reduce the feature extraction cost. Experiments demonstrate that the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$e$</tex-math></inline-formula> -TSN can achieve an accuracy of 91.3% on the Jester dataset. Finally, a human–machine identity verification system is developed based on the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$e$</tex-math></inline-formula> -TSN, which can effectually accomplish the human–machine identity in real time and has a high promotion value.