Unsupervised feature selection using feature similarity

TLDR

The algorithm is generic and supports multiscale representation of data sets. The paper proposes an unsupervised feature selection algorithm for large, high‑dimensional data sets. The method measures feature similarity using a new maximum information compression index to remove redundancy. The algorithm is fast, outperforms existing methods in speed and performance, and quantifies redundancy and information loss using an entropy measure.

Abstract

In this article, we describe an unsupervised feature selection algorithm suitable for data sets, large in both dimension and size. The method is based on measuring similarity between features whereby redundancy therein is removed. This does not need any search and, therefore, is fast. A new feature similarity measure, called maximum information compression index, is introduced. The algorithm is generic in nature and has the capability of multiscale representation of data sets. The superiority of the algorithm, in terms of speed and performance, is established extensively over various real-life data sets of different sizes and dimensions. It is also demonstrated how redundancy and information loss in feature selection can be quantified with an entropy measure.