Regularized Discriminant Analysis

TLDR

Linear and quadratic discriminant analysis are studied in small‑sample, high‑dimensional settings. The study proposes alternatives to the standard maximum‑likelihood covariance estimates. The method uses two‑parameter covariance alternatives tuned by minimizing a sample‑based misclassification risk estimate, with fast implementations. Simulation and real‑data studies show dramatic gains in classification accuracy.

Abstract

Abstract Linear and quadratic discriminant analysis are considered in the small-sample, high-dimensional setting. Alternatives to the usual maximum likelihood (plug-in) estimates for the covariance matrices are proposed. These alternatives are characterized by two parameters, the values of which are customized to individual situations by jointly minimizing a sample-based estimate of future misclassification risk. Computationally fast implementations are presented, and the efficacy of the approach is examined through simulation studies and application to data. These studies indicate that in many circumstances dramatic gains in classification accuracy can be achieved.