Real-time hand-tracking with a color glove

TLDR

Articulated hand‑tracking systems are common in virtual reality but are rarely adopted in consumer products because of their cost and complexity. This work proposes an inexpensive, user‑friendly system that enables 3‑D articulated hand input. The system uses a single camera to track a cloth glove printed with a custom pattern that simplifies pose estimation, allowing a nearest‑neighbor algorithm to achieve interactive tracking rates. Proof‑of‑concept applications in modeling, animation control, and augmented reality demonstrate the system’s potential as a foundation for new interaction techniques.

Abstract

Articulated hand-tracking systems have been widely used in virtual reality but are rarely deployed in consumer applications due to their price and complexity. In this paper, we propose an easy-to-use and inexpensive system that facilitates 3-D articulated user-input using the hands. Our approach uses a single camera to track a hand wearing an ordinary cloth glove that is imprinted with a custom pattern. The pattern is designed to simplify the pose estimation problem, allowing us to employ a nearest-neighbor approach to track hands at interactive rates. We describe several proof-of-concept applications enabled by our system that we hope will provide a foundation for new interactions in modeling, animation control and augmented reality.