Two Easy Improvements to Lexical Weighting

Abstract

We introduce two simple improvements to the lexical weighting features of Koehn, Och, and Marcu (2003) for machine translation: one which smooths the probability of translating word f to word e by simplifying English morphology, and one which conditions it on the kind of training data that f and e co-occurred in. These new variations lead to improvements of up to +0.8 BLEU, with an average improvement of +0.6 BLEU across two language pairs, two genres, and two translation systems. each i, let c ( f j, ei) ← c ( f j, ei) + 1 |ai| for j ∈ ai (1) c(NULL, ei) ← c(NULL, ei) + 1 if |ai | = 0 (2) Then t(e | f) = c ( f, e) e c ( f, e)