Turkish Sign Language Recognition Using Kinect Skeleton and Convolutional Neural Network

Abstract

In this paper, a system that converts Turkish signs to words using convolutional neural networks is presented. Skeleton data obtained by Microsoft's Kinect device is used in the proposed system. User located in front of Kinect sensor makes the signs of corresponding word for a limited time. Afterwards, skeleton joints are extracted. Finally, skeleton points on consecutive frames are merged, and Word classification is performed by convulational neural network. In experimental studies, evaluations are carried out with our own dataset and performance of the proposed method is compared with various classification methods. Moreover, the effect of being closer or further to the camera and movements in different speeds are also investigated. In the lights of the experimental results, it is seen that proposed convolutional neural network gives better performances than other classifiers such as Gaussian SVM, Linear SVM, weighted KNN and decision trees.