Real-time 3D pointing gesture recognition for natural HCI

Abstract

Hand gestures play a natural and intuitive communication mode for all human dialogs. The use of human movements, especially hand gestures, has become an important part of Human-Computer Interaction (HCI) in recent years. Among natural human gestures occurring during non-verbal communication, pointing gesture can be easily recognized and included in more natural new human computer interfaces. Researchers have proposed many systems based on pointing gesture for HCI. Theses systems, however, share the same elements such as a video projector, a camera and a planar surface. Unlike these existing approaches, a novel approach to pointing gestures recognition based on binocular stereo vision is developed. The pointing direction is based on the line of sight connecting the point of pseudo-eyes. The mentioned model does not constrain the flat surface pointed to by user to be visible by the cameras. The unique feature of our system is that cameras do not need to capture entire bodies and faces. Experimental results show that the system works in real time on a low cost hardware platform. It is fairly accurate and independent of user characteristics and environmental changes.