Theory of Disagreement-Based Active Learning

TLDR

Active learning is a supervised learning protocol that sequentially queries labels from a large unlabeled pool, contrasting with passive learning where labels are randomly sampled. The article aims to illustrate the theoretical advantages of active learning by presenting key theorems and proofs that demonstrate how it can achieve high accuracy with fewer labels than passive learning. It focuses on disagreement‑based active learning, a mature technique, and briefly surveys other approaches in the literature. The main findings are rigorous theorems that establish the performance guarantees of several general disagreement‑based algorithms.

Abstract

Active learning is a protocol for supervised machine learning, in which a learning algorithm sequentially requests the labels of selected data points from a large pool of unlabeled data. This contrasts with passive learning, where the labeled data are taken at random. The objective in active learning is to produce a highly-accurate classifier, ideally using fewer labels than the number of random labeled data sufficient for passive learning to achieve the same. This article describes recent advances in our understanding of the theoretical benefits of active learning, and implications for the design of effective active learning algorithms. Much of the article focuses on a particular technique, namely disagreement-based active learning, which by now has amassed a mature and coherent literature. It also briefly surveys several alternative approaches from the literature. The emphasis is on theorems regarding the performance of a few general algorithms, including rigorous proofs where appropriate. However, the presentation is intended to be pedagogical, focusing on results that illustrate fundamental ideas, rather than obtaining the strongest or most general known theorems. The intended audience includes researchers and advanced graduate students in machine learning and statistics, interested in gaining a deeper understanding of the recent and ongoing developments in the theory of active learning.