Dimensionality reduction using genetic algorithms

TLDR

Pattern recognition relies on measurable features, and feature extraction aims to reduce measurement cost and improve classifier efficiency, but linear transformations often do not reduce the number of measured features because new features are combinations of all originals. The study introduces a simultaneous feature selection, extraction, and classifier training method using a genetic algorithm. A genetic algorithm optimizes feature weights and a masking vector to select a subset, and the resulting features are used with k‑nearest‑neighbor classification, benchmarked against sequential floating forward selection and linear discriminant analysis. The approach successfully identifies favorable water‑binding sites on protein surfaces.

Abstract

Pattern recognition generally requires that objects be described in terms of a set of measurable features. The selection and quality of the features representing each pattern affect the success of subsequent classification. Feature extraction is the process of deriving new features from original features to reduce the cost of feature measurement, increase classifier efficiency, and allow higher accuracy. Many feature extraction techniques involve linear transformations of the original pattern vectors to new vectors of lower dimensionality. While this is useful for data visualization and classification efficiency, it does not necessarily reduce the number of features to be measured since each new feature may be a linear combination of all of the features in the original pattern vector. Here, we present a new approach to feature extraction in which feature selection and extraction and classifier training are performed simultaneously using a genetic algorithm. The genetic algorithm optimizes a feature weight vector used to scale the individual features in the original pattern vectors. A masking vector is also employed for simultaneous selection of a feature subset. We employ this technique in combination with the k nearest neighbor classification rule, and compare the results with classical feature selection and extraction techniques, including sequential floating forward feature selection, and linear discriminant analysis. We also present results for the identification of favorable water-binding sites on protein surfaces.