Enabling Two-Way Communication of Deaf Using Saudi Sign Language

Abstract

Disabled people are facing many difficulties communicating with others and involving in society. Modern societies have dedicated significant efforts to promote the integration of disabled individuals into their societies and services. Currently, smart healthcare systems are used to facilitate disabled people. The objective of this paper is to enable two-way communication of deaf individuals with the rest of society, thus enabling their migration from marginal elements of society to mainstream contributing elements. In the proposed system, we developed three modules; the sign recognition module (SRM) that recognizes the signs of a deaf individual, the speech recognition and synthesis module (SRSM) that processes the speech of a non-deaf individual and converts it to text, and an Avatar module (AM) to generate and perform the corresponding sign of the non-deaf speech, which were integrated into the sign translation companion system called Saudi deaf companion system (SDCS) to facilitate the communication from the deaf to the hearing and vice versa. This paper also contributes to the literature by utilizing our self-developed database, the largest Saudi Sign Language (SSL) database—the King Saud University Saudi-SSL (KSU-SSL). The proposed SDCS system performs 293 Saudi signs that are recommended by the Saudi Association for Hearing Impairment (SAHI) from 10 domains (healthcare, common, alphabets, verbs, pronouns and adverbs, numbers, days, kings, family, and regions).