The Optimized Selection of Base-Classifiers for Ensemble Classification using a Multi-Objective Genetic Algorithm

Abstract

We propose an ensemble classification algorithm that uses multi-objective optimization instead of relying on heuristics or fragile user-defined parameters. Only two user-defined parameters are included, with both being found to have large windows of values that produce statistically indistinguishable results, indicating the low level of expertise required from the user to achieve good results. Additionally, when given a large initial set of trained base-classifiers, we demonstrate that a multi-objective genetic algorithm aiming to optimize prediction accuracy and diversity will prefer particular types of classifiers over others. The total number of chosen classifiers is also surprisingly small - only 5.86 classifiers on average, out of a starting pool of 900. This occurs without any explicit preference for small ensembles of classifiers. Even with these small ensembles, significantly lower empirical classification error is achieved compared to the current state-of-the-art.