A Novel Deep Learning Model for Understanding Two-Person Interactions Using Depth Sensors

Abstract

Despite the ever-increasing efforts made in the field of data science and artificial intelligence, the task of automatic human interaction recognition remains challenging. Advanced computer vision sensors like depth sensors have made it easier to achieve the goal of accurate recognition of human interactions in complex situations. The reason for their success is that they are robust against lighting and illumination variation and are insensitive to color and texture changes. Therefore, the proposed system combines both depth and RGB images to train a Convolutional Neural Network (CNN). The robust features extracted from CNN have been classified using a Softmax classifier. Two publically available large RGB-D datasets have been used for training and evaluating the proposed method. During the experiments, the proposed method achieved an accuracy of 87.03% with the NTU RGB+D dataset and 86.21% with the UoL3D Social Interaction dataset.