IEEE Transactions on Knowledge and Data Engineering · 2006 · 20 citations · 28 references
ComputingComputer ScienceMathematicsData MiningMachine LearningStatistical LearningData PreprocessingDimension ReductionDimensionality ReductionKernel MethodsOptimization For Machine LearningPattern RecognitionKernel Discriminant AnalysisLinear Discriminant AnalysisSingular Value Decomposition
Kernelized nonlinear extensions of Fisher's discriminant analysis, discriminant analysis based on generalized singular value decomposition (LDA/GSVD), and discriminant analysis based on the minimum squared error formulation (MSE) have recently been widely utilized for handling undersampled high-dimensional problems and nonlinearly separable data sets. As the data sets are modified from incorporating new data points and deleting obsolete data points, there is a need to develop efficient updating and downdating algorithms for these methods to avoid expensive recomputation of the solution from scratch. In this paper, an efficient algorithm for adaptive linear and nonlinear kernel discriminant analysis based on regularized MSE, called adaptive KDA/RMSE, is proposed. In adaptive KDA/RMSE, updating and downdating of the computationally expensive eigenvalue decomposition (EVD) or singular value decomposition (SVD) is approximated by updating and downdating of the QR decomposition achieving an order of magnitude speed up. This fast algorithm for adaptive kernelized discriminant analysis is designed by utilizing regularization techniques and the relationship between linear and nonlinear discriminant analysis and the MSE. In addition, an efficient algorithm to compute leave-one-out cross validation is also introduced by utilizing downdating of KDA/RMSE.
28
Yuhai Wu, Vladimir Vapnik · Technometrics · 1999 · 27K citations
Numerical methods for least squares problems
Choice Reviews Online · 1996 · 3.1K citations
Fisher discriminant analysis with kernels
Gunnar Rätsch, Jason Weston, Bernhard Schölkopf et al. · 2003 · 2.7K citations
Lawrence Barker · Technometrics · 2002 · 2.5K citations
Yuhai Wu · Technometrics · 1999 · 1.8K citations