Spiral (Imperial College London) · 2015 · 163 citations · 7 references
Open access
ComputingComputer ScienceHigh-performance ComputingMachine LearningNatural Language ProcessingParallel ComputingData ParallelismDeep LearningHardware AccelerationNeural NetworksPattern RecognitionSpeech RecognitionCompute Unified Device ArchitectureDeep Learning AccelerationDeep Learning AcceleratorsDeep Learning On GpusLong Short-term Memory NetworksParallel Deep LearningRecurrent Neural NetworksVanishing Gradient Problem
In this article, we introduce CURRENNT, an open-source parallel implementation of deep recurrent neural networks (RNNs) supporting graphics processing units (GPUs) through NVIDIA's Computed Unified Device Architecture (CUDA).CURRENNT supports uni-and bidirectional RNNs with Long Short-Term Memory (LSTM) memory cells which overcome the vanishing gradient problem.To our knowledge, CURRENNT is the first publicly available parallel implementation of deep LSTM-RNNs.Benchmarks are given on a noisy speech recognition task from the 2013 2nd CHiME Speech Separation and Recognition Challenge, where LSTM-RNNs have been shown to deliver best performance.In the result, double digit speedups in bidirectional LSTM training are achieved with respect to a reference single-threaded CPU implementation.CURRENNT
7
Speech recognition with deep recurrent neural networks
Alex Graves, Abdelrahman Mohamed, Geoffrey E. Hinton · 2013 · 8.8K citations
Learning to Forget: Continual Prediction with LSTM
Felix A. Gers, Jürgen Schmidhuber, Fred Cummins · Neural Computation · 2000 · 5.4K citations
LSTM neural networks for language modeling
Martin Sundermeyer, Ralf Schlüter, Hermann Ney · 2012 · 2K citations
Computing, Computer Science, French +14
Deep Neural Networks for Acoustic Modeling in Speech Recognition
Geoffrey E. Hinton, Li Deng, Dong Yu et al. · 2012 · 1.9K citations