Hand Tracking with Flocks of Features

Abstract

Tracking hands in live video is a challenging task: the hand appearance can change too rapidly for appearance-based trackers to work, and color-based trackers (that do not rely on geometry) have to make limiting assumptions about the background color. This article shows the results of hand tracking with "Flocks of Features", a tracking method that combines motion cues and a learned foreground color distribution to achieve fast and robust 2D tracking of highly articulated objects. Many independent image artifacts are tracked from one frame to the next, adhering only to local constraints. This concept is borrowed from nature since these tracks mimic the flight of flocking birds - exhibiting local individualism and variability while maintaining a clustered entirety. Hand tracking has important applications for interaction with wearable computers, for intuitive manipulation of virtual objects, for detection of activity signatures, and much more. Tracking with Flocks of Features is not limited to hands - any articulated or appearance-changing object can benefit from this multi-cue tracking method.