2015 · 109 citations · 11 references
ComputingComputer ScienceMachine LearningNatural Language ProcessingPhoneticsDeep LearningLanguage ModelingPattern RecognitionShort-term MemorySpeech RecognitionConnectionist Temporal ClassificationConvolutional Neural NetworksDeep Neural Network Acoustic ModelsLong Short-term Memory NetworksRecurrent Neural NetworksSpeech Pattern RecognitionStreaming Speech RecognitionTemporal Classification
This paper describes a series of experiments to extend the application of Context-Dependent (CD) long short-term memory (LSTM) recurrent neural networks (RNNs) trained with Connectionist Temporal Classification (CTC) and sMBR loss. Our experiments, on a noisy, reverberant voice search task, include training with alternative pronunciations and the application to child speech recognition; combination of multiple models, and convolutional input layers. We also investigate the latency of CTC models and show that constraining forward-backward alignment in training can reduce the delay for a real-time streaming speech recognition system. Finally we investigate transferring knowledge from one network to another through alignments.
11
Sepp Hochreiter, Jürgen Schmidhuber · Neural Computation · 1997 · 98.1K citations
Distilling the Knowledge in a Neural Network
Geoffrey E. Hinton, Oriol Vinyals · arXiv (Cornell University) · 2015 · 14K citations · Full text
Connectionist temporal classification
Alex Graves, Santiago Fernández, Faustino Gomez et al. · 2006 · 5.4K citations
Convolutional, Long Short-Term Memory, fully connected Deep Neural Networks
Tara N. Sainath, Oriol Vinyals, Andrew Senior et al. · 2015 · 1.6K citations