LDV-Forum/Journal for language technology and computational linguistics · 2010 · 33 citations · 44 references
Language StudiesComputer ScienceLinguisticsComputational LinguisticsInformation RetrievalNatural Language ProcessingAnnotationHistorical MorphologyMorphological DisambiguationPart-of-speech TaggingCanonical FormsCanonicalizationLexical ResourcesMorphosyntactic FeaturesMorphosyntactic ParsingPart-of-speech DisambiguationText CanonicalizationTokenization For Morphological Analysis
Historical text presents numerous challenges for \ncontemporary natural language processing techniques. In particular, the absence of consistent orthographic conventions in historical text presents difficulties for any system requiring reference to a static lexicon indexed by orthographic form. Canonicalization approaches seek to address these issues by associating one or more extant ``canonical cognates'' with each word of the input text and deferring application analysis to these canonical forms. Type-wise conflation techniques treating each input word in isolation often suffer from a pronounced precision -- recall trade-off pattern: high-precision techniques such as conservative transliteration have comparatively poor recall, whereas high-recall techniques such as phonetic conflation tend to be disappointingly imprecise. In this paper, I present a technique for disambiguation of type conflation sets at the token level using a Hidden Markov Model whose lexical probability matrix is dynamically computed from the candidate conflations, and evaluate its performance on a manually annotated corpus of historical German. \n
44
A Mathematical Theory of Communication
Claude E. Shannon · Bell System Technical Journal · 1948 · 80.7K citations
A note on two problems in connexion with graphs
E. Dijkstra · Numerische Mathematik · 1959 · 23.8K citations
Binary codes capable of correcting deletions, insertions and reversals
V.I. Levenshtein · Soviet physics. Doklady · 1965 · 10.4K citations
An algorithm for suffix stripping
Martin Porter · Program electronic library and information systems · 1980 · 8.1K citations