2018 · 11 citations · 17 references
ComputingEnglishComputational LinguisticsFeature EngineeringNatural Language ProcessingDeep LearningLanguage ModelingLanguage ModelsNatural Language UnderstandingText Feature EngineeringWord EmbeddingsCharacter N-gramsLanguage Model PretrainingN-gram RepresentationNeural Language ModelsRecurrent Neural Language ModelsRecurrent Neural Network Language ModelsTrainingWord N-grams
© LREC 2018 - 11th International Conference on Language Resources and Evaluation. All rights reserved. Recently, an abundance of deep learning toolkits has been made freely available. These toolkits typically offer the building blocks and sometimes simple example scripts, but designing and training a model still takes a considerable amount of time and knowledge. We present language modeling scripts based on TensorFlow that allow one to train and test competitive models directly, by using a pre-defined configuration or changing it to their needs. There are several options for input features (words, characters, words combined with characters, character n-grams) and for batching (sentence- or discourse-level). The models can be used to test the perplexity, predict the next word(s), re-score hypotheses or generate debugging files for interpolation with n-gram models. Additionally, we make available LSTM language models trained on a variety of Dutch texts and English benchmarks, that can be used immediately, thereby avoiding the time and computationally expensive training process. The toolkit is open source and can be found at https://github.com/lverwimp/tf-lm.
17
Yangqing Jia, Evan Shelhamer, Jeff Donahue et al. · 2014 · 11.2K citations
Recurrent neural network based language model
Tomáš Mikolov, Martin Karafiát, Lukáš Burget et al. · 2010 · 5.4K citations
SRILM - an extensible language modeling toolkit
Andreas Stolcke · 2002 · 4.4K citations
LSTM neural networks for language modeling
Martin Sundermeyer, Ralf Schlüter, Hermann Ney · 2012 · 2K citations
Computing, Computer Science, French +14