Markerless 4 gestures 6 DOF real‐time visual tracking of the human hand with automatic initialization

Abstract

Abstract In this paper we present a novel computer vision based hand‐tracking technique, which is capable of robustly tracking 6+4DOF of the human hand in real‐time (at least 25 frames per second) with the help of 3 (or more) off‐the‐shelf consumer cameras. ‘6+4DOF’ means that the system can track the global pose (6 continuous parameters for translation and rotation) of 4 different gestures. A key feature of our system is its fully automatic real‐time initialization procedure, which, along with a sound tracking‐lost detector, makes the system fit for real‐world applications. Because of this, our method acts as an enabling technology for uncumbersome hand‐based 3D Human‐Computer‐Interaction (HCI). Previously, using the hand as an at least 6DOF input device involved the use of either datagloves or markers. Using our tracking we evaluated the use of the hand as an input device for two prevalent Virtual Reality applications: fly‐through exploration of a virtual world and a simple digital assembly simulation.