Normalized Mutual Information Feature Selection

TLDR

The paper proposes NMIFS, a filter feature selection method based on normalized mutual information. NMIFS uses the average normalized mutual information to quantify feature redundancy and is extended into GAMIFS by integrating it into a genetic algorithm with NMIFS‑based initialization and mutation to accelerate convergence. Experiments show NMIFS surpasses MIFS, MIFS‑U, and mRMR on multiple datasets, and GAMIFS further improves feature selection by overcoming incremental search limitations.

Abstract

A filter method of feature selection based on mutual information, called normalized mutual information feature selection (NMIFS), is presented. NMIFS is an enhancement over Battiti's MIFS, MIFS-U, and mRMR methods. The average normalized mutual information is proposed as a measure of redundancy among features. NMIFS outperformed MIFS, MIFS-U, and mRMR on several artificial and benchmark data sets without requiring a user-defined parameter. In addition, NMIFS is combined with a genetic algorithm to form a hybrid filter/wrapper method called GAMIFS. This includes an initialization procedure and a mutation operator based on NMIFS to speed up the convergence of the genetic algorithm. GAMIFS overcomes the limitations of incremental search algorithms that are unable to find dependencies between groups of features.