Hand tracking and pose recognition via depth and color information

Abstract

As one of the most natural and intuitive way of communication between people and machines, hand gesture is widely used in HCI (Human-Computer-interaction). In this paper, we proposed a novel method for hand tracking and pose recognition based on Kinect. For hand tracking, skin information is used for initialization of hand segmentation, and then a region growing algorithm is applied in the depth image to separate hand from other skin colored objects. Finally, a Kalman filter is used for tracking hand in 3D space. For hand recognition, we decompose the problem of recognizing hand pose into recognizing different finger states. Both contour information of the whole hand and depth information inside the contour are considered for finger states recognition. It is shown in the experiments that our system can track the hand robustly and recognize more than 90% of the hand poses we define for our depth image database.