Proceedings · 2006 · 11 citations · 14 references
ComputingComputer ScienceData MiningMachine LearningStatistical LearningDimension ReductionDimensionality ReductionFeature SelectionPrincipal Component AnalysisScalable Machine LearningSupervised LearningBenchmark DatasetsInformation GainInformation-gain MaximizationLinear Discriminant AnalysisOnline Feature SelectionPrincipal ComponentsSingularity ProblemSubspace Learning
Subspace learning approaches aim to discover important statistical distribution on lower dimensions for high dimensional data. Methods such as principal component analysis (PCA) do not make use of the class information, and linear discriminant analysis (LDA) could not be performed efficiently in a scalable way. In this paper, we propose a novel highly scalable supervised subspace learning algorithm called as supervised Kampong measure (SKM). It assigns data points as close as possible to their corresponding class mean, simultaneously assigns data points to be as far as possible from the other class means in the transformed lower dimensional subspace. Theoretical derivation shows that our algorithm is not limited by the number of classes or the singularity problem faced by LDA. Furthermore, our algorithm can be executed in an incremental manner in which learning is done in an online fashion as data streams are received. Experimental results on several datasets, including a very large text data set RCV1, show the outstanding performance of our proposed algorithm on classification problems as compared to PCA, LDA and a popular feature selection approach, information gain (IG).
14
UCI Repository of machine learning databases
Catherine Blake · Medical Entomology and Zoology · 1998 · 10.5K citations
A Comparative Study on Feature Selection in Text Categorization
Yiming Yang, Jan Pedersen · 1997 · 4.8K citations
Aleix M. Martı́nez, A.C. Kak · IEEE Transactions on Pattern Analysis and Machine Intelligence · 2001 · 3.2K citations
RCV1: A New Benchmark Collection for Text Categorization Research
David Lewis, Yiming Yang, Tony Rose et al. · Goldsmiths (University of London) · 2004 · 2.6K citations · Full text
Constrained K-means Clustering with Background Knowledge
Kiri L. Wagstaff, Claire Cardie, Seth Rogers et al. · 2001 · 2.4K citations