Multimodal concept-dependent active learning for image retrieval

TLDR

Active learning is effective for learning complex, subjective query concepts in image retrieval, yet it has traditionally been applied in a concept‑independent manner with identical kernel parameters and sampling strategies across concepts. The study aims to characterize a concept’s complexity using hit‑rate, isolation, and diversity metrics. A multimodal learning approach is proposed that uses images’ semantic labels to guide a concept‑dependent active‑learning process, adjusting the sampling strategy and pool based on the measured complexity to improve learnability. Experiments on a 300‑K‑image dataset demonstrate that concept‑dependent learning substantially improves image‑retrieval accuracy.

Abstract

It has been established that active learning is effective for learning complex, subjective query concepts for image retrieval. However, active learning has been applied in a concept independent way, (i.e., the kernel-parameters and the sampling strategy are identically chosen) for learning query concepts of differing <i>complexity</i>. In this work, we first characterize a concept's complexity using three measures: <i>hit-rate</i>, <i>isolation</i> and <i>diversity</i>. We then propose a multimodal learning approach that uses images' semantic labels to guide a <i>concept-dependent</i>, <i>active-learning</i> process. Based on the complexity of a concept, we make intelligent adjustments to the sampling strategy and the sampling pool from which images are to be selected and labeled, to improve concept learnability. Our empirical study on a $300$K-image dataset shows that concept-dependent learning is highly effective for image-retrieval accuracy.