2022 · 19 citations · 32 references
ComputingComputer ScienceComputational LinguisticsMachine LearningNatural Language ProcessingLanguage ModelingLanguage ModelsLexical AnalysisMorphological ParsingMorphological SegmentationWord EmbeddingsLanguage Model PretrainingLanguage Processing ModelsMorphological Parsing ModelsMorphological StructureNeural Language ModelsText ClassificationTokenizationTokenization For Morphological Analysis
We introduce FLOTA (Few Longest Token Approximation), a simple yet effective method to improve the tokenization of pretrained language models (PLMs). FLOTA uses the vocabulary of a standard tokenizer but tries to preserve the morphological structure of words during tokenization. We evaluate FLOTA on morphological gold segmentations as well as a text classification task, using BERT, GPT-2, and XLNet as example PLMs. FLOTA leads to performance gains, makes inference more efficient, and enhances the robustness of PLMs with respect to whitespace noise.
32
Adam: A Method for Stochastic Optimization
Diederik P. Kingma, Jimmy Ba · UvA-DARE (University of Amsterdam) · 2014 · 84.8K citations · Full text
The CELEX Lexical Database (CD-ROM)
R. Harald Baayen, Richard Piepenbrock, Leon Gulikers · Max Planck Digital Library · 1996 · 2.2K citations
Japanese and Korean voice search
Mike Schuster, Kaisuke Nakajima · 2012 · 859 citations