Journal of Management Information Systems · 1999 · 76 citations · 19 references
ComputingComputer ScienceComputational LinguisticsNatural Language ProcessingText AnalyticsClassificationLanguage ClassificationLanguage ModelingMultilingual Natural Language ProcessingMultilingual ProcessingMultilingualismMultilingual Document AnalysisMultilingual Document ProcessingMultilingual DocumentsMultilingual Information ExtractionMultilingual Text ClassificationOverhead Reduction
We present in this paper Linguini, a vector-space based categorize tailored for high-precision language identlj?cation. We show how the accuracy depends on the size of the input document, the set of languages under consideration, and the features used. We found that Linguini could identifi the language of documents as short as 5-10 % of the size of average Web documents with 100% accuracy. We also present how to determine z ~ a document is in two or more languages, and in what proportions, without incurring any appreciable computational overhead beyond the monolingual analysis. This approach can be applied to subject-categorization systems to distinguish between cases where, when the system recommends two or more categories, the document belongs strongly to all or really to none.
19
An algorithm for suffix stripping
Martin Porter · Program electronic library and information systems · 1980 · 8.1K citations
A Comparative Study on Feature Selection in Text Categorization
Yiming Yang, Jan Pedersen · 1997 · 4.8K citations
Pattern Classification and Scene Analysis
Michael Thompson, Richard O. Duda, Peter E. Hart · Leonardo · 1974 · 4.5K citations