Using human body gestures as inputs for gaming via depth analysis

Abstract

Natural ways of input greatly enhance the entertainment experience for emerging gaming systems represented by the Sony Playstation 2 EyeToy and the Nintendo Wii Console. In this paper we present a novel method of using human body gestures depth image as gaming application input. Depth images have natural advantages over grayscale or color images in terms of robustness against illumination change, texture complexity, and background interference. Our proposed method consists of three major components: depth image acquisition, mean shift based preprocessing, and HMM-based gesture recognition. We validate our method by applying it to a boxing game scenario to distinguish boxing gestures such as dodge, jab, hook, and uppercut. The experiment results indicate that our method can efficiently distinguish the subtle differences among these gestures and yield excellent accuracy (up to about 98%). The potential usage of the proposed method on gaming applications and generic human computer interaction is very promising.