Combination Feature Selection Based on Relief

Abstract

Relief is a feature evaluation method which performs well, while Relief cannot discriminate redundant features. It proposes two combination feature selection algorithm based on Relief: ReCorre and ReSBSW. The two algorithms both first use Relief to filter irrelevant features, then use correlation analysis and sequential backward search (SBS) in Wrapper form to remove redundant features,respectively. It makes experiments on real and artificial datasets, analyze and make comparison between Relief,ReCorre and ReSBSW. It gets the following conclusions: Relief can reduce dimension well on datasets with many irrelevant features, but can remove relatively few irrelevant features and may remove relevant features for real datasets with complex relationship among features. ReCorre can remove most of redundant features based on ReliefF, while ReSBSW can get better generalization performance with high computing, and is not fit to large-scale datasets.