Hard Zero Shot Learning for Gesture Recognition

Abstract

Gesture based systems allow humans to interact with devices and robots in a natural way. Yet, current gesture recognition systems can not recognize the gestures outside a limited lexicon. This opposes the idea of lifelong learning which require systems to adapt to unseen object classes. These issues can be best addressed using Zero Shot Learning (ZSL), a paradigm in machine learning that leverages the semantic information to recognize new classes. ZSL systems developed in the past used hundreds of training examples to detect new classes and assumed that test examples come from unseen classes. This work introduces two complex and more realistic learning problems referred as Hard Zero Shot Learning (HZSL) and Generalized HZSL (G-HZSL) necessary to achieve Life Long Learning. The main objective of these problems is to recognize unseen classes with limited training information and relax the assumption that test instances come from unseen classes. We propose to leverage one shot learning (OSL) techniques coupled with ZSL approaches to address and solve the problem of HZSL for gesture recognition. Further, supervised clustering techniques are used to discriminate seen classes from unseen classes. We assessed and compared the performance of various existing algorithms on HZSL for gestures using two standard datasets: MSRC-12 and CGD2011. For four unseen classes, results show that the marginal accuracy of HZSL-15.2% and G-HZSL-14.39% are comparable to the performance of conventional ZSL. Given that we used only one instance and do not assume that test classes are unseen, the performance of HZSL and G-HZSL models were remarkable.