A comparison of evolved linear and non-linear ensemble vote aggregators

Abstract

Ensemble classifiers have become a widely researched area in machine learning because they are able to generalise well to unseen data, making them suitable for real world applications. Many approaches implement simple voting techniques, such as majority voting or averaging, to form the overall output. Alternatively, Genetic Algorithms (GAs) can be used to train the ensemble by optimising either binary or floating weights in an average voting scheme to produce evolved linear voting aggregators. Non-linear voting schemes can also be formed by inputting the base classifiers' outputs into a secondary classifier in the form of an expression tree or an Artificial Neural Network; this being subsequently evolved by an Evolutionary Algorithm to optimise the ensemble prediction. This paper aims to firstly, establish the impact of evolving linear aggregators on traditional voting techniques, and secondly whether these linear functions produce more accurate predictions than more complex nonlinear evolved aggregators. The results indicate that optimising the ensemble combination method with GAs offers significant advantages over standard approaches and produces comparable ensembles to those with nonlinear aggregators despite their additional complexity.