Laplacian Score for Feature Selection

TLDR

Feature selection is well studied in supervised learning, but in unsupervised settings it is more challenging because class labels are absent, and most existing unsupervised methods rely on wrapper techniques that require a learning algorithm. This paper introduces a filter-based feature selection method that does not depend on any learning algorithm. The method evaluates each feature’s locality‑preserving power via the Laplacian Score and can be applied in both supervised and unsupervised contexts, with comparisons to data variance and Fisher score on two datasets. Experiments show that the proposed algorithm is both effective and efficient.

Abstract

In supervised learning scenarios, feature selection has been studied widely in the literature. Selecting features in unsupervised learning scenarios is a much harder problem, due to the absence of class labels that would guide the search for relevant information. And, almost all of previous unsupervised feature selection methods are wrapper techniques that require a learning algorithm to evaluate the candidate feature subsets. In this paper, we propose a filter method for feature selection which is independent of any learning algorithm. Our method can be performed in either supervised or unsupervised fashion. The proposed method is based on the observation that, in many real world classification problems, data from the same class are often close to each other. The importance of a feature is evaluated by its power of locality preserving, or, Laplacian Score. We compare our method with data variance (unsupervised) and Fisher score (supervised) on two data sets. Experimental results demonstrate the effectiveness and efficiency of our algorithm.