An Efficient Sign Language Translator Device Using Convolutional Neural Network and Customized ROI Segmentation

Abstract

Sign language is widely used by hearing impaired people all over the world. With the advancement of cutting-edge deep learning techniques, there has been immense attention given by the researchers for sign language conversion. But only a few works have been executed on the Bangla Sign Language conversion for hearing impaired people. This paper aims to demonstrate a user-friendly approach towards Bangla Sign language to text conversion through customized Region of Interest (ROI) segmentation and Convolutional Neural Network (CNN). 5 sign gestures are trained using custom image dataset and implemented in Raspberry Pi for portability. Using the ROI selection approach, the process shows better outcomes than conventional approaches in terms of accuracy level and real time detection from video streaming through webcam. Furthermore, this method serves to offer an efficient model which ultimately results in easy addition of more signs to the final prototype made using Raspberry Pi.