The mRMR variable selection method: a comparative study for functional data

Abstract

The use of variable selection methods is particularly appealing in statistical problems with functional data. The obvious general criterion for variable selection is to choose the ‘most representative’ or ‘most relevant’ variables. However, it is also clear that a purely relevance-oriented criterion could lead to select many redundant variables. The minimum Redundance Maximum Relevance (mRMR) procedure, proposed by Ding and Peng [Minimum redundancy feature selection from microarray gene expression data. J Bioinform Comput Biol. 2005;3:185–205] and Peng et al. [Feature selection based on mutual information: criteria of max-dependency, max-relevance, and min-redundancy. IEEE Trans Pattern Anal Mach Intell. 2005;27:1226–1238] is an algorithm to systematically perform variable selection, achieving a reasonable trade-off between relevance and redundancy. In its original form, this procedure is based on the use of the so-called <i>mutual information criterion</i> to assess relevance and redundancy. Keeping the focus on functional data problems, we propose here a modified version of the mRMR method, obtained by replacing the mutual information by the new association measure (called <i>distance correlation</i>) suggested by Székely et al. [Measuring and testing dependence by correlation of distances. Ann Statist. 2007;35:2769–2794]. We have also performed an extensive simulation study, including 1600 functional experiments (100 functional models ×4 sample sizes ×4 classifiers) and three real-data examples aimed at comparing the different versions of the mRMR methodology. The results are quite conclusive in favour of the new proposed alternative.