arXiv (Cornell University) · 2018 · 53 citations · 18 references
HumanitiesTranslation StudiesCorpus-based Translation StudiesTranslation TheoryTranslator TrainingLanguage ModelingMachine TranslationMultilingual Natural Language ProcessingMultilingual ProcessingParallel CorporaTranslation Quality AssessmentAttentionLow-resource Machine TranslationMultilingual Machine TranslationNeural Machine TranslationTranslation Quality EstimationZero-shot Machine Translation
Multilingual machine translation addresses the task of translating between multiple source and target languages. We propose task-specific attention models, a simple but effective technique for improving the quality of sequence-to-sequence neural multilingual translation. Our approach seeks to retain as much of the parameter sharing generalization of NMT models as possible, while still allowing for language-specific specialization of the attention model to a particular language-pair or task. Our experiments on four languages of the Europarl corpus show that using a target-specific model of attention provides consistent gains in translation quality for all possible translation directions, compared to a model in which all parameters are shared. We observe improved translation quality even in the (extreme) low-resource zero-shot translation directions for which the model never saw explicitly paired parallel data.
18
Adam: A Method for Stochastic Optimization
Diederik P. Kingma, Jimmy Ba · UvA-DARE (University of Amsterdam) · 2014 · 84.8K citations · Full text
Kishore Papineni, Salim Roukos, Todd J. Ward et al. · 2001 · 21.5K citations · Full text
Rich Caruana · Machine Learning · 1997 · 6.3K citations · Full text
Philipp Koehn, Richard Zens, Chris Dyer et al. · 2007 · 4.9K citations · Full text