Towards robust cross-user hand tracking and shape recognition

Abstract

Low-cost depth cameras create new opportunities for robust and ubiquitous vision-based interfaces. While much research has focused on full-body pose estimation and the interpretation of gross body movement, this work investigates skeleton-free hand detection, tracking, and shape classification. Our goal is to build a rich and reliable gestural interface by developing methods that recognize a broad set of hand shapes and which maintain high accuracy rates across a wide range of users. In this paper, we describe our approach to real-time hand detection and tracking using depth data from the Microsoft Kinect. We present quantitative shape recognition results for eight hand shapes collected from 16 users and then discuss physical configuration and interface design issues that help boost reliability and overall user experience.