Efficient model-based 3D tracking of hand articulations using Kinect

TLDR

The study proposes a markerless method to recover and track the 3D position, orientation, and full articulation of a human hand using a Kinect sensor. The method formulates hand tracking as an optimization problem solved with a variant of Particle Swarm Optimization, requiring no markers or complex acquisition. Experiments show the approach yields accurate, robust, continuous 3D hand tracking at 15 Hz in near real‑time.

Abstract

We present a novel solution to the problem of recovering and tracking the 3D position, orientation and full articulation of a human hand from markerless visual observations obtained by a Kinect sensor. We treat this as an optimization problem, seeking for the hand model parameters that minimize the discrepancy between the appearance and 3D structure of hypothesized instances of a hand model and actual hand observations. This optimization problem is effectively solved using a variant of Particle Swarm Optimization (PSO). The proposed method does not require special markers and/or a complex image acquisition setup. Being model based, it provides continuous solutions to the problem of tracking hand articulations. Extensive experiments with a prototype GPU-based implementation of the proposed method demonstrate that accurate and robust 3D tracking of hand articulations can be achieved in near real-time (15Hz).