2019 · 40 citations · 33 references
ComputingComputer ScienceKnowledge TransferLanguage AcquisitionNatural Language ProcessingCross-linguistic TransferLanguage AdaptationLanguage ModelingLanguage TransferSpeech RecognitionTransfer LearningCross-lingual Language ModelingCross-lingual TransferCross-lingual Transfer LearningCross-linguistic AdaptationLanguage Model IntegrationSequence-to-sequence Recognition
This work explores better adaptation methods to low-resource languages using an external language model (LM) under the framework of transfer learning. We first build a language-independent ASR system in a unified sequence-to-sequence (S2S) architecture with a shared vocabulary among all languages. During adaptation, we perform LM fusion transfer, where an external LM is integrated into the decoder network of the attention-based S2S model in the whole adaptation stage, to effectively incorporate linguistic context of the target language. We also investigate various seed models for transfer learning. Experimental evaluations using the IARPA BABEL data set show that LM fusion transfer improves performances on all target five languages compared with simple transfer learning when the external text data is available. Our final system drastically reduces the performance gap from the hybrid systems.
33
Sepp Hochreiter, Jürgen Schmidhuber · Neural Computation · 1997 · 98.1K citations
Automatic differentiation in PyTorch
Adam Paszke, Sam Gross, Soumith Chintala et al. · 2017 · 11.1K citations
Librispeech: An ASR corpus based on public domain audio books
Vassil Panayotov, Guoguo Chen, Daniel Povey et al. · 2015 · 6K citations
Computing, Computer Science, English +17