Learning to Rank Using an Ensemble of Lambda-Gradient Models

Abstract

Abstract We describe the system that won Track 1 of the Yahoo! Learning to Rank Challenge.Keywords: Learning to Rank, Gradient Boosted Trees, Lambda Gradients, Web Search 1. Introduction and Summary The Yahoo! Learning to Rank Challenge, Track 1, was a public competition on a Ma-chine Learning for Information Retrieval task: given a set of queries, and given a set ofretrieved documents for each query, train a system to maximize the Expected ReciprocalRank (Chapelle et al.,2009) on a blind test set, where the training data takes the formof a feature vector x 2R d with label y2Y; Yf0;1;2;3;4g(a more positive numberdenoting higher relevance) for each query/document pair (the original, textual data wasnot made available). The Challenge setup, background information, and results have beenextensively covered elsewhere and we refer toChapelle and Chang(2011) for details. Inthis paper we summarize the work that resulted in the winning system. 1 We limit the workdescribed in this paper to the work done speci cally for the Challenge; the work was doneover a four week period prior to the end of the Challenge.Our approach used a linear combination of twelve ranking models, eight of which werebagged LambdaMART boosted tree models, two of which were LambdaRank neural nets,and two of which were MART models using a logistic regression cost. LambdaRank isa method for learning arbitrary information retrieval measures; it can be applied to anyalgorithm that learns through gradient descent (Burges et al.,2006). LambdaRank is alistwise method, in that the cost depends on the sorted order of the documents. We brieysummarize the ideas here, where the x