A cooperative ensemble learning system

Abstract

This paper presents a new cooperative ensemble learning system (CELS) for designing neural network ensembles. The idea behind CELS is to encourage different individual networks in an ensemble to learn different parts or aspects of the training data so that the ensemble can learn the whole training data better. Rather than producing unbiased individual networks whose errors are uncorrelated, CELS tends to create negatively correlated networks with a novel correlation penalty term in the error function to encourage such specialisation. In CELS, individual networks are trained simultaneously rather than sequentially. This provides an opportunity for different networks to cooperate with each other and to specialise. This paper analyses CELS in terms of bias-variance-covariance trade-off. Experiments on a real-world problem demonstrate that CELS can produce neural network ensembles with good generalisation ability.