DSpace (IIT Bombay) · 2018 · 26 citations · 11 references
ComputingComputer ScienceComputational LinguisticsMachine LearningNatural Language ProcessingAttention NetworksDeep LearningLanguage ModelsSequence LabelingSequence LearningSpeech RecognitionAttentionProbabilistic Sequence LearningSequence Representation LearningSequence-to-sequence RecognitionSequential Attention
Modern neural architectures critically rely on attention for mapping structured inputs to sequences. In this paper we show that prevalent attention architectures do not adequately model the dependence among the attention and output tokens across a predicted sequence. We present an alternative architecture called Posterior Attention Models that after a principled factorization of the full joint distribution of the attention and output variables, proposes two major changes. First, the position where attention is marginalized is changed from the input to the output. Second, the attention propagated to the next decoding stage is a posterior attention distribution conditioned on the output. Empirically on five translation and two morphological inflection tasks the proposed posterior attention models yield better BLEU score and alignment accuracy than existing attention models. © 7th International Conference on Learning Representations, ICLR 2019. All Rights Reserved.
11
Statistical machine translation
Choice Reviews Online · 2010 · 1.4K citations
Supervised Learning of Complete Morphological Paradigms
Greg Durrett, John DeNero · 2013 · 136 citations
Improved Neural Machine Translation with a Syntax-Aware Encoder and Decoder
Huadong Chen, Shujian Huang, David Chiang et al. · 2017 · 129 citations · Full text