Increased accessibility to nonverbal communication through facial and expression recognition technologies for blind/visually impaired subjects

Abstract

Conversation between two individuals requires verbal dialogue; the majority of human communication however consists of non-verbal cues such as gestures and facial expressions. Blind individuals are thus hindered in their interaction capabilities. To address this, we are building a computer vision system with facial recognition and expression algorithms to relay nonverbal messages to a blind user. The device will communicate the identities and facial expressions of communication partners in realtime. In order to ensure that this device will be useful to the blind community, we conducted surveys and interviews and we are working with subjects to test prototypes of the device. This paper describes the algorithms and design concepts incorporated in this device, and it provides a commentary on early survey and interview results. A corresponding poster with demonstration stills is exhibited at this conference.