A Filter Approach to Feature Selection Based on Mutual Information

Abstract

In pattern recognition, feature selection aims to choose the smallest subset of features that is necessary and sufficient to describe the target concept. In this paper, a mutual information-based constructive criterion under arbitrary information distributions of input features is presented for feature selection. This criterion can capture both the relevance to the output classes and the redundancy with respect to the already-selected features without any parameters like ß in MIFS or MIFS-U methods to be preset. Furthermore, a modified greedy feature selection algorithm called MICC is proposed, and experimental results demonstrate the good performance of MICC on both synthetic and benchmark data sets.