2004 · 39 citations · 8 references
ComputingComputer ScienceMachine LearningNatural Language ProcessingText AnalyticsClassificationFeature SelectionLanguage ClassificationText Feature EngineeringText MiningText Similarity AnalysisDocument ClassificationFeature Selection For ClassificationNaive BayesNaive Bayes ClassificationText CategorizationText ClassificationText Classification FeaturesText Mining Features
We define a new feature selection score for text classification based on the KL-divergence between the distribution of words in training documents and their classes. The score favors words that have a similar distribution in documents of the same class but different distributions in documents of different classes. Experiments on two standard data sets indicate that the new method outperforms mutual information, especially for smaller categories.
8
On Information and Sufficiency
S. Kullback, R. A. Leibler · The Annals of Mathematical Statistics · 1951 · 20K citations · Full text
A Comparative Study on Feature Selection in Text Categorization
Yiming Yang, Jan Pedersen · 1997 · 4.8K citations
Elements of Information Theory
Simon J. D. Phoenix · Journal of Modern Optics · 1992 · 1.4K citations