Using marking menus to develop command sets for computer vision based hand gesture interfaces

Abstract

This paper presents the first stages of a project that studies the use of hand gestures for interaction, in an approach based on computer vision. A first prototype for exploring the use of marking menus for interaction has been built. The purpose is not menu-based interaction per se, but to study if marking menus, with practice, could support the development of autonomous command sets for gestural interaction. Some early observations are reported, mainly concerning problems with user fatigue and precision of gestures. Future work is discussed, such as introducing flow menus for reducing fatigue, and control menus for continuous control functions. The computer vision algorithms will also have to be developed further.