arXiv (Cornell University) · 2018 · 10 citations · 11 references
ComputingComputer ScienceComputational LinguisticsCorpus-based Translation StudiesNatural Language ProcessingLanguage ModelingMachine TranslationAttentionNeural Language ModelsNeural Machine TranslationPhrase-based Machine TranslationStatistical Machine TranslationTransformer Language ModelsTransformer Models
Most state-of-the-art neural machine translation systems, despite being different in architectural skeletons (e.g. recurrence, convolutional), share an indispensable feature: the Attention. However, most existing attention methods are token-based and ignore the importance of phrasal alignments, the key ingredient for the success of phrase-based statistical machine translation. In this paper, we propose novel phrase-based attention methods to model n-grams of tokens as attention entities. We incorporate our phrase-based attentions into the recently proposed Transformer network, and demonstrate that our approach yields improvements of 1.3 BLEU for English-to-German and 0.5 BLEU for German-to-English translation tasks on WMT newstest2014 using WMT'16 training data.
11
Adam: A Method for Stochastic Optimization
Diederik P. Kingma, Jimmy Ba · UvA-DARE (University of Amsterdam) · 2014 · 84.8K citations · Full text
Kishore Papineni, Salim Roukos, Todd J. Ward et al. · 2001 · 21.5K citations · Full text
Ashish Vaswani, Noam Shazeer, Niki Parmar et al. · 2025 · 6.6K citations · Full text
Statistical phrase-based translation
Philipp Koehn, Franz Josef Och, Daniel Marcu · 2003 · 3.3K citations · Full text