Video Understanding-Based Random Hand Gesture Authentication

Abstract

Existing hand gesture authentication methods require the probe gesture types to be consistent with the registered ones, which reduces the user-friendliness and efficiency of authentication. In this paper, a video understanding based random hand gesture authentication method is introduced to eliminate this limitation, in which users only need to improvise a random hand gesture in front of an RGB camera without memory and hesitation in both the enrollment and verification stage. The random hand gesture is a promising biometric trait containing both physiological and behavioral characteristics. To fully unleash the potential of random hand gesture authentication, we design a simple but effective behavior representation (modality), temporal difference map, for better behavioral characteristic understanding, and present an efficient model called 3D Temporal Difference Symbiotic Neural Network (3DTDS-Net) that can separately extract physiological and behavioral features as well as automatically assign fusion weights for the two features to complement each other’s strengths based on the magnitude of behavioral features in an end-to-end fashion. We also adapt and reimplement 17 SOTA neural networks for authentication from other tasks, such as action classification and gait recognition, to make convincing comparisons. Extensive experiments on the SCUT-DHGA dataset demonstrate the effectiveness of temporal difference maps and the superiority of 3DTDS-Net. The code is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/SCUT-BIP-Lab/3DTDS-Net</uri> .