Sparse Discriminant Analysis

TLDR

High‑dimensional classification, where many features outnumber observations, is a well‑studied problem in chemometrics and increasingly common in biology and medicine. The authors propose sparse discriminant analysis, a linear discriminant method that simultaneously performs classification and feature selection by imposing a sparsity constraint. The method derives from the optimal‑scoring formulation of LDA and can be extended to sparse discrimination using Gaussian mixtures to handle nonlinear class boundaries or within‑class subgroups. The approach yields low‑dimensional representations of discriminative directions. © 2011 American Statistical Association and the American Society for Quality.

Abstract

We consider the problem of performing interpretable classification in the high-dimensional setting, in which the number of features is very large and the number of observations is limited. This setting has been studied extensively in the chemometrics literature, and more recently has become commonplace in biological and medical applications. In this setting, a traditional approach involves performing feature selection before classification. We propose sparse discriminant analysis, a method for performing linear discriminant analysis with a sparseness criterion imposed such that classification and feature selection are performed simultaneously. Sparse discriminant analysis is based on the optimal scoring interpretation of linear discriminant analysis, and can be extended to perform sparse discrimination via mixtures of Gaussians if boundaries between classes are nonlinear or if subgroups are present within each class. Our proposal also provides low-dimensional views of the discriminative directions. © 2011 American Statistical Association and the American Society for Qualitys.