Toward Optimal Active Learning through Sampling Estimation of Error Reduction

Abstract

This paper presents an active learning method that di-rectly optimizes expected future error. This is in con-trast to many other popular techniques that instead aim to reduce version space size. These other meth-ods are popular because for many learning models, closed form calculation of the expected future error is intractable. Our approach is made feasible by taking a sampling approach to estimating the expected reduc-tion in error due to the labeling of a query. In exper-imental results on two real-world data sets we reach high accuracy very quickly, sometimes with four times fewer labeled examples than competing methods. 1.