Unsupervised feature selection for multi-cluster data

TLDR

High‑dimensional data analysis relies on feature selection to reduce dimensionality and improve clustering, classification, and retrieval, yet traditional unsupervised methods ignore feature correlations and treat the problem as a costly combinatorial optimization. This study addresses unsupervised feature selection, aiming to identify relevant feature subsets without class labels. The authors introduce Multi‑Cluster Feature Selection (MCFS), which selects features that best preserve the data’s multi‑cluster structure by solving a sparse eigen‑problem coupled with an L1‑regularized least‑squares optimization. Experiments on real‑life datasets show MCFS outperforms existing unsupervised feature selection methods.

Abstract

In many data analysis tasks, one is often confronted with very high dimensional data. Feature selection techniques are designed to find the relevant feature subset of the original features which can facilitate clustering, classification and retrieval. In this paper, we consider the feature selection problem in unsupervised learning scenario, which is particularly difficult due to the absence of class labels that would guide the search for relevant information. The feature selection problem is essentially a combinatorial optimization problem which is computationally expensive. Traditional unsupervised feature selection methods address this issue by selecting the top ranked features based on certain scores computed independently for each feature. These approaches neglect the possible correlation between different features and thus can not produce an optimal feature subset. Inspired from the recent developments on manifold learning and L1-regularized models for subset selection, we propose in this paper a new approach, called Multi-Cluster Feature Selection (MCFS), for unsupervised feature selection. Specifically, we select those features such that the multi-cluster structure of the data can be best preserved. The corresponding optimization problem can be efficiently solved since it only involves a sparse eigen-problem and a L1-regularized least squares problem. Extensive experimental results over various real-life data sets have demonstrated the superiority of the proposed algorithm.