Interactive online learning

Abstract

This paper describes a system for visual object recognition based on mobile augmented reality gear. The user can train the system to the recognition of objects online using advanced methods of interaction with mobile systems: Hand gestures and speech input control “virtual menus,” which are displayed as overlays within the camera image. Here we focus on the underlying neural recognition system, which implements the key requirement of an online trainable system—fast adaptation to novel object data. The neural three-stage architecture can be adapted in two modes: In a fast training mode (FT), only the last stage is adapted, whereas complete training (CT) rebuilds the system from scratch. Using FT, online acquired views can be added at once to the classifier, the system being operational after a delay of less than a second, though still with reduced classification performance. In parallel, a new classifier is trained (CT) and loaded to the system when ready.