2014 · 158 citations · 19 references
ComputingComputer ScienceHigh-performance ComputingComputer VisionMachine LearningNatural Language ProcessingDeep LearningLong-term MemoryNeural NetworksPattern RecognitionShort-term MemoryWord RecognitionGated Recurrent Unit NetworksHandwriting RecognitionLong Short-term Memory NetworksMini-batch LearningMini-batch TrainingRecurrent Neural NetworksSequence ChunkingSequence-to-sequence Recognition
In this paper we demonstrate a modified topology for long short-term memory recurrent neural networks that controls the shape of the squashing functions in gating units. We further propose an efficient training framework based on a mini-batch training on sequence level combined with a sequence chunking approach. The framework is evaluated on publicly available data sets containing English and French handwriting by utilizing a GPU based implementation. Speedups of more than 3x are achieved in training recurrent neural network models which outperform state of the art recognition results.
19
Sepp Hochreiter, Jürgen Schmidhuber · Neural Computation · 1997 · 98.1K citations
Jeffrey L. Elman · Cognitive Science · 1990 · 10.8K citations
Jeffrey L. Elman · Cognitive Science · 1990 · 2.9K citations · Full text