Finger Multimodal Features Fusion and Recognition Based on CNN

Abstract

Finger-based multimodal fusion recognition has attracted increasing attention due to its high stability and security in practical application. Traditional fusion methods exist some challenges in scale inconsistency and universality. In additon, multimodal dimension standardization has not been well realized. In this paper, we propose a novel convolutional neural network(CNN) framework for finger multimodal fusion and recognition, which obtains the fusion features by network learning automatically. The fusion network makes full use of the complementary information among three modals of finger to make the fusion features more stable and effective. Firstly, the three parallel CNNs have been designed to extract unimodal features. Then, a size standardization method based on principal component analysis(PCA) is utilized for different unimodal features. Finally, the high-level unimodal features are integrated to learn fusion features with better representation capability. Extensive experiments on the finger multimodal dataset show that the proposed multimodal fusion network performs better than other state-of-the-arts.