Effective Neural Network Ensemble Approach for Improving Generalization Performance

Abstract

This paper, with an aim at improving neural networks' generalization performance, proposes an effective neural network ensemble approach with two novel ideas. One is to apply neural networks' output sensitivity as a measure to evaluate neural networks' output diversity at the inputs near training samples so as to be able to select diverse individuals from a pool of well-trained neural networks; the other is to employ a learning mechanism to assign complementary weights for the combination of the selected individuals. Experimental results show that the proposed approach could construct a neural network ensemble with better generalization performance than that of each individual in the ensemble combining with all the other individuals, and than that of the ensembles with simply averaged weights.