An Evaluation of Landmarking Variants

Abstract

. Landmarking is a novel technique for data characterization in metalearning. While conventional approaches typically describe a database with its statistical measurements and properties, landmarking proposes to enrich such a description with quick and easy-to-obtain performance measures of simple learning algorithms. In this paper, we will discuss two novel aspects of landmarking. First, we investigate relative landmarking, which tries to exploit the relative order of the landmark measures instead of their absolute value. Second, we propose to use subsampling estimates as a different way for efficiently obtaining landmarks. In general, our results are mostly negative. The most interesting result is a surprisingly simple rule that predicts quite accurately when it is worth to boost decision trees. 1