Northern European Journal of Language Technology · 2009 · 11 citations · 33 references
Language StudiesComputer ScienceLinguisticsComputational LinguisticsMorphologyNatural Language ProcessingCorpus-based TerminologyMorphological AnalysisDictionary Entry GenerationEntryFinite-state TransducersInflectional Categories
Language software applications encounter new words, e.g., acronyms, technical terminology, loan words, names or compounds of such words. To add new words to a lexicon, we need to indicate their base form and inflectional paradigm. In this article, we evaluate a combination of corpus-based and lexicon-based methods for assigning the base form and inflectional paradigm to new words in Finnish, Swedish and English finite-state transducer lexicons. The methods have been implemented with the open-source Helsinki Finite-State Technology (Lindén & al., 2009). As an entry generator often produces numerous suggestions, it is important that the best suggestions be among the first few, otherwise it may become more efficient to create the entries by hand. By combining the probabilities calculated from corpus data and from lexical data, we get a more precise combined model. The combined method has 77-81 % precision and 89-97 % recall, i.e. the first correctly generated entry is on the average found as the first or second candidate for the test languages. A further study demonstrated that a native speaker could revise suggestions from the entry generator at a speed of 300-400 entries per hour.
33
Language identification in the limit
Eric Gold · Information and Control · 1967 · 3.6K citations
Class-based n -gram models of natural language
Peter F. Brown, P.V. deSouza, Robert L. Mercer et al. · 1992 · 2.9K citations
Language Studies, Computer Science, Computational Linguistics +15
Automatic rule induction for unknown-word guessing
Andrei Mikheev · 1997 · 260 citations
Language Studies, Computer Science, Computational Linguistics +16
A study of grammatical inference
James J. Horning · Defense Technical Information Center (DTIC) · 1969 · 220 citations