Human-computer interaction using gesture recognition and 3D hand tracking

Abstract

This paper describes a real time system for human-computer interaction through gesture recognition and three dimensional hand-tracking. Using two cameras that are focused at the user's hand the system recognizes three gestures and tracks the hand in three dimensions. The system can simultaneously track two fingers (thumb and pointing finger) and output their poses. The pose for each finger consists of three positional coordinates and two angles (azimuth and elevation). By moving the thumb and the pointing finger in 3D, the user can control 10 degrees of freedom in a smooth and natural fashion. We have used this system as a multi-dimensional input-interface to computer games, terrain navigation software and graphical editors. In addition to providing 10 degrees of control, our system is much more natural and intuitive to use compared to traditional input devices. The system is user independent, and operates at the rate of 60 Hz.