2013 · 288 citations · 31 references
ComputingComputer ScienceLinguisticsComputational LinguisticsMachine LearningNatural Language ProcessingDeep LearningLanguage ModelingMultilingual Natural Language ProcessingSpeech RecognitionConvolutional Neural NetworksCross-lingual TransferCross-lingual Transfer LearningCross-linguistic Speech RecognitionDeep Neural Network Acoustic ModelsMultilingual Speech RecognitionTraining Time
Today's speech recognition technology is mature enough to be useful for many practical applications. In this context, it is of paramount importance to train accurate acoustic models for many languages within given resource constraints such as data, processing power, and time. Multilingual training has the potential to solve the data issue and close the performance gap between resource-rich and resource-scarce languages. Neural networks lend themselves naturally to parameter sharing across languages, and distributed implementations have made it feasible to train large networks. In this paper, we present experimental results for cross- and multi-lingual network training of eleven Romance languages on 10k hours of data in total. The average relative gains over the monolingual baselines are 4%/2% (data-scarce/data-rich languages) for cross- and 7%/2% for multi-lingual training. However, the additional gain from jointly training the languages on all data comes at an increased training time of roughly four weeks, compared to two weeks (monolingual) and one week (crosslingual).
31
A Fast Learning Algorithm for Deep Belief Nets
Geoffrey E. Hinton, Simon Osindero, Yee‐Whye Teh · Neural Computation · 2006 · 16.4K citations
Rich Caruana · Machine Learning · 1997 · 6.3K citations · Full text