A Note On the Connection and Equivalence of Three Sparse Linear Discriminant Analysis Methods

Abstract

In this article, we reveal the connection between and equivalence of three sparse linear discriminant analysis methods: the ℓ1-Fisher’s discriminant analysis proposed by Wu et al. in 2008 Wu, M., Zhang, L., Wang, Z., Christiani, D. and Lin, X. 2008. “Sparse Linear Discriminant Analysis for Simultaneous Testing for the Significance of a Gene Set/Pathway and Gene Selection. Bioinformatics, 25: 1145–1151. [Crossref] , [Google Scholar], the sparse optimal scoring proposed by Clemmensen et al. in 2011 Clemmensen, L., Hastie, T., Witten, D. and Ersbøll, B. 2011. “Sparse Discriminant Analysis. Technometrics, 53: 406–413. [Taylor & Francis Online], [Web of Science ®] , [Google Scholar], and the direct sparse discriminant analysis (DSDA) proposed by Mai et al. in 2012 Mai, Q., Zou, H. and Yuan, M. 2012. “A Direct Approach to Sparse Discriminant Analysis in Ultra-High Dimensions. Biometrika, 99: 29–42. [Crossref], [Web of Science ®] , [Google Scholar]. It is shown that, for any sequence of penalization parameters, the normalized solutions of DSDA equal the normalized solutions of the other two methods at different penalization parameters. A prostate cancer dataset is used to demonstrate the theory.