A Branch and Bound Algorithm for Feature Subset Selection

TLDR

Existing feature‑subset selection methods such as sequential selection and dynamic programming do not guarantee optimality, and exhaustive search is computationally infeasible, requiring evaluation of over 2.7 million subsets. The paper develops a branch‑and‑bound algorithm to optimally select m features from an n‑feature set. The algorithm efficiently explores the search space using branch‑and‑bound, avoiding exhaustive enumeration. Experiments show the algorithm achieves substantial computational savings, selecting a 12‑feature subset from 24 features by evaluating only 6,000 subsets instead of over 2.7 million.

Abstract

A feature subset selection algorithm based on branch and bound techniques is developed to select the best subset of m features from an n-feature set. Existing procedures for feature subset selection, such as sequential selection and dynamic programming, do not guarantee optimality of the selected feature subset. Exhaustive search, on the other hand, is generally computationally unfeasible. The present algorithm is very efficient and it selects the best subset without exhaustive search. Computational aspects of the algorithm are discussed. Results of several experiments demonstrate the very substantial computational savings realized. For example, the best 12-feature set from a 24-feature set was selected with the computational effort of evaluating only 6000 subsets. Exhaustive search would require the evaluation of 2 704 156 subsets.