Effective pruning of neural network classifier ensembles

Abstract

Neural network ensemble techniques have been shown to be very accurate classification techniques. However, in some real-life applications a number of classifiers required to achieve a reasonable accuracy is enormously large and hence very space consuming. The paper proposes several methods for pruning neural network ensembles. The clustering based approach applies k-means clustering to entire set of classifiers in order to identify the groups of similar classifiers and then eliminates redundant classifiers inside each cluster. Another proposed approach contains the sequence of the depth-first building the tree of classifiers according to their diversity followed by the process of tree pruning. The proposed methods applied to several data sets have shown that by selecting an optimal subset of neural network classifiers, it is possible to obtain significantly smaller ensemble of classifiers while achieving the same or even slightly better generalizability as when using the entire ensemble.