Optimising weights for heterogeneous ensemble of classifiers with differential evolution

Abstract

The classification performance of a weighted voting ensemble of classifiers largely depends on the proper weight chosen for each base classifier's vote. In this paper, we propose the use of Differential Evolution algorithm for adjustment of voting-weights of base classifiers used in a heterogeneous ensemble of classifiers (HEoC). We used the average Matthews Correlation Coefficient (MCC), calculated over 10-fold cross-validation, as the quality measure of an ensemble. We applied the vanilla DE algorithm to maximise the average MCC score over the training dataset. The algorithm optimises the base classifiers' voting weights in order to attain better generalisation performance of the ensemble on testing datasets. Experiments were performed using 10 binary-class datasets taken from UCI-Machine Learning Repository. The results show consistent and superior generalisation performance of the constructed ensembles when compared with the base classifiers and other well-known ensemble of classifiers.