Sign Language Recognition using Machine Learning Algorithm

Abstract

Sign Language Recognition System aims to rephrase sign language into text or speech to ease communication between deaf and hearing individuals. This issue has far-reaching implications but remains exceptionally hard due to the complexity and large variation of hand movements. Existing SLR styles use palm-drafted characteristics to determine the shift of sign language and make classification models based on these features. Still, it’s delicate to design dependable features that accommodate wide variations in gestures. KNN (k-nearest neighbor) algorithm has shown major challenges like curse of dimensionalities, large dataset sizes, class imbalance, noise, and outliers. To address this conclusion, this study suggests a new convolutional neural network (CNN) that can automatically extract discriminative spatial-temporal elements extracted without information known from unprocessed video streams and avoid designing features. To improve performance, multi-channel video streams (such as color data, depth cues, and body joint orientations) are sent into CNN as input to combine color, depth, and movement data. Using a real-world dataset acquired using Microsoft Kinect, this study evaluated the proposed model and showed its effectiveness over traditional methods based on manual labor feature.