A Learning Algorithm For Neural Network Ensembles

Abstract

The performance of a single regressor/classifier can be improved by combining the outputs of many predictors. This is true provided the combined predictors are accurate and diverse enough, which posses the problem of generating suitable aggregate members in order to have optimal generalization capabilities. We propose here a new method for selecting members of regression/classification ensembles. In particular, using artificial neural networks as learners in a regression context, we show that this method leads to small aggregates with few but very diverse individual networks. The algorithm is favorably tested against other methods recently proposed in the literature, producing a remarkable performance improvement on the standard statistical databases used as benchmarks.