A Divide-and-Conquer-Based Ensemble Classifier Learning by Means of Many-Objective Optimization

Abstract

Divide-and-conquer-based methods are quite successful across various problems from different disciplines. These methods divide a complex task into multiple simple tasks and solve them collectively. This paper presents a divide-and-conquer-based hierarchical optimization framework for ensemble classifier learning (ECL). The optimization framework includes a search space creation process [called data training environments (DTE)] that divides the data into multiple clusters, and then trains a set of heterogeneous base classifiers with the DTEs. The classifiers are then combined to form an optimal ensemble, by finding the fittest ones using many-objective optimization. The many-objective optimization algorithm considers each class accuracy as a separate objective and maximizes the class accuracies. An additional objective is also taken into account by maximizing the ensemble size. Since the partitioning of data creates diversity within the pool of classifiers, class accuracy tradeoff among the classifiers is observed. As a result, increasing the number of classifiers also increases the diversity within the ensemble. In order to tackle the optimization, a specialized many-objective optimization algorithm based on decomposition is proposed. Since ECL can be regarded as an NP-hard problem, the proposed optimization algorithm, instead identifies the optimal ensemble using a divide-and-conquer rule-based chromosome encoding. Moreover, with the involvement of individual class accuracy in the objectives, the performance does not get biased toward any majority class. The proposed framework is experimented with 24 benchmark datasets obtained from the UCI machine learning repository and compared with the existing approaches. The experimental results show better classification accuracy with the proposed framework in comparison with the recent ensemble classifiers.