BEER: BEtter Evaluation as Ranking

Abstract

We present the UvA-ILLC submission of the BEER metric to WMT 14 metrics task. BEER is a sentence level metric that can incorporate a large number of features combined in a linear model. Novel con-tributions are (1) efficient tuning of a large number of features for maximizing corre-lation with human system ranking, and (2) novel features that give smoother sentence level scores. 1