Extended Relief Algorithms in Instance-Based Feature Filtering

Abstract

This paper presents extended Relief algorithms and their use in instance-based feature filtering for document feature selection. The Relief algorithms are general and successful feature estimators that detect conditional dependencies of features between instances, and are applied in the preprocessing step for document classification and regression. Since the introduction the Relief algorithm, many kinds of extended Relief algorithms have been suggested as solutions to problems of redundancy, irrelevant and noisy features as well as Relief algorithm's limitations in two-class and multi-class datasets. In this paper, we introduce additional problems including the negative influence of computation similarities and weights caused by the small number of features in an instance, the absence of nearest Hits or nearest Misses for some instances using Relief algorithms, and other of problems. We suggest new extended Relief algorithms to solve those problems, having in the course of our research, and experimented on the estimation of the quality of features from instances, and classified datasets, and having compared the results of the new extended Relief algorithms. Indeed in the experimental results, the new extended Relief algorithms showed better performances for all of the datasets than did the Relief algorithms