Hybrid-MST: A Hybrid Active Sampling Strategy for Pairwise Preference\n Aggregation

Abstract

In this paper we present a hybrid active sampling strategy for pairwise\npreference aggregation, which aims at recovering the underlying rating of the\ntest candidates from sparse and noisy pairwise labelling. Our method employs\nBayesian optimization framework and Bradley-Terry model to construct the\nutility function, then to obtain the Expected Information Gain (EIG) of each\npair. For computational efficiency, Gaussian-Hermite quadrature is used for\nestimation of EIG. In this work, a hybrid active sampling strategy is proposed,\neither using Global Maximum (GM) EIG sampling or Minimum Spanning Tree (MST)\nsampling in each trial, which is determined by the test budget. The proposed\nmethod has been validated on both simulated and real-world datasets, where it\nshows higher preference aggregation ability than the state-of-the-art methods.\n