A robust method of detecting hand gestures using depth sensors

Abstract

Depth sensors, including Kinect and Xtion, open up a new possibility for future human-computer interaction (HCI). Even though there already are some mature methods of detecting human skeleton and poses using depth sensors, it is still an unsolved problem to detect hands and recognize delicate gestures effectively, because hands are too small a part in the images generated from depth sensor, so the details of hands are hard to extract. In this paper, we present a gesture detecting method that is able to: firstly segment hands through skin color segmentation and K-means clustering; secondly find the convex hull and the contour that form the hand shape; thirdly detect positions of each fingertip; and finally represent gestures using the sets of detected hand data. Having been tested with a series of applications, our method is proved to be robust and effective.