2003 · 25 citations · 10 references
ComputingComputer ScienceFeature EngineeringMachine LearningStatistical LearningClassificationGenerative ModelingPattern RecognitionProbabilistic Machine LearningRepresentation LearningStatistical InferenceData DistributionData Distribution AnalysisGraphical ModelsHidden Variable ModelsProbabilistic Classification
This paper addresses the problem of classification in situations where the data distribution is not homoge-neous: Data instances might come from different lo-cations or times, and therefore are sampled from re-lated but different distributions. In particular, features may appear in some parts of the data that are rarely or never seen in others. In most situations with non-homogeneous data, the training data is not representa-tive of the distribution under which the classifier must operate. We propose a method, based on probabilistic graphical models, for utilizing unseen features during classification. Our method introduces, for each such unseen feature, a continuous hidden variable describ-ing its influence on the class — whether it tends to be associated with some label. We then use probabilis-tic inference over the test data to infer a distribution over the value of this hidden variable. Intuitively, we “learn ” the role of this unseen feature from the test set, generalizing from those instances whose label we are fairly sure about. Our overall probabilistic model is learned from the training data. In particular, we also learn models for characterizing the role of unseen fea-tures; these models use “meta-features ” of those fea-tures, such as words in the neighborhood of an un-seen feature, to infer its role. We present results for this framework on the task of classifying news arti-cles and web pages, showing significant improvements over models that do not use unseen features. 1.
10
William H. Press, Saul A. Teukolsky, William T. Vetterling et al. · 1994 · 15.4K citations
An Introduction to Variational Methods for Graphical Models
Michael I. Jordan, Zoubin Ghahramani, Tommi Jaakkola et al. · Machine Learning · 1999 · 3.8K citations · Full text
Text Classification from Labeled and Unlabeled Documents using EM
Kamal Nigam, Andrew Kachites McCallum, Sebastian Thrun et al. · Machine Learning · 2000 · 2.7K citations · Full text
Bursty and hierarchical structure in streams
Jon Kleinberg · 2002 · 1.1K citations