Image-based Bengali Sign Language Alphabet Recognition for Deaf and Dumb Community

Abstract

For the deaf and dumb (D&D) people, sign language is one of the primary and most used methods for communication. All over the world, every day the D&D community faces difficulties while communicating with the general mass. Most of the times, they need an interpreter to communicate with others and the interpreter may not always be available. The issue is also faced by the people using Bengali Sign Language (BdSL) due to the lack of BdSL interpreters. Recently, computer vision-based systems have been introduced for automatic recognition of sign languages to mitigate this problem. But so far the number of reliable works done for the recognition of BdSL is not adequate. In this paper, we propose a method for automatic detection of BdSL alphabets. Our system solely relies on the images of bare hands, which allows the users to interact with the system in a natural way. We have collected in total 12581 different hand signs for the 38 BdSL alphabets in collaboration with the National Federation of the Deaf. We propose a VGG19 based convolutional neural network for the recognition of 38 classes and achieve an overall test accuracy of 89.6%.