arXiv (Cornell University) · 2015 · 274 citations · 23 references
ComputingComputer ScienceArtificial IntelligenceMachine LearningMemoryAttention NetworksDeep LearningLong-term MemoryNeural NetworksPattern RecognitionSequence LearningAttentionAttention MechanismsLong-term Memory ModelsRecurrent Neural NetworksSequence Representation LearningSimplified Models
We propose a simplified model of attention which is applicable to feed-forward neural networks and demonstrate that the resulting model can solve the synthetic "addition" and "multiplication" long-term memory problems for sequence lengths which are both longer and more widely varying than the best published results for these tasks.
23
Sepp Hochreiter, Jürgen Schmidhuber · Neural Computation · 1997 · 98.1K citations
Adam: A Method for Stochastic Optimization
Diederik P. Kingma, Jimmy Ba · UvA-DARE (University of Amsterdam) · 2014 · 84.8K citations · Full text
Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
Kelvin Xu, Jimmy Ba, Ryan Kiros et al. · arXiv (Cornell University) · 2015 · 7.5K citations · Full text