Real-Time Malaysian Sign Language Translation using Colour Segmentation and Neural Network

Abstract

In this paper we present an automatic visual-based sign language translation system. Our proposed automatic sign-language translator provides a real-time English translation of the Malaysia SL. To date, there have been studies on sign language recognition based on visual approach (video camera). However, the emphasis on these works is limited to a small lexicon of sign language or solely focuses on fingerspelling, which takes different approaches respectively. In practical sense, fingerspelling is used if a word cannot be expressed via sign language. Our sign language translator can recognise both fingerspelling and sign gestures that involve static and motion signs. Trained neural networks are used to identify the signs to translate into English.