Synthesized Classifiers for Zero-Shot Learning

TLDR

Zero‑shot learning uses semantic descriptions of unseen classes to recognize them by relating them to seen classes with labeled examples. The study proposes to address zero‑shot learning via manifold learning by aligning the external semantic space with the visual model space. The authors introduce phantom object classes that exist in both semantic and model spaces and serve as dictionary bases optimized from labeled data, enabling synthesis of discriminative real object classifiers. The approach achieves higher accuracy than state‑of‑the‑art methods on four benchmark datasets, including ImageNet Fall 2011 with over 20,000 unseen classes.

Abstract

Given semantic descriptions of object classes, zero-shot learning aims to accurately recognize objects of the unseen classes, from which no examples are available at the training stage, by associating them to the seen classes, from which labeled examples are provided. We propose to tackle this problem from the perspective of manifold learning. Our main idea is to align the semantic space that is derived from external information to the model space that concerns itself with recognizing visual features. To this end, we introduce a set of "phantom" object classes whose coordinates live in both the semantic space and the model space. Serving as bases in a dictionary, they can be optimized from labeled data such that the synthesized real object classifiers achieve optimal discriminative performance. We demonstrate superior accuracy of our approach over the state of the art on four benchmark datasets for zero-shot learning, including the full ImageNet Fall 2011 dataset with more than 20,000 unseen classes.