An iterative method for multi-class cost-sensitive learning

TLDR

Cost‑sensitive learning addresses the issue of classification in the presence of varying costs associated with different types of misclassification. The paper proposes a method to solve multi‑class cost‑sensitive learning problems using any binary classification algorithm. The method is built on three key ideas: iterative weighting, expanding the data space, and gradient boosting with stochastic ensembles. The authors provide theoretical guarantees, show a variant has a boosting property under a weak‑learning assumption, and empirically demonstrate that the method outperforms representative cost‑sensitive learning approaches in predictive performance and often in computational efficiency.

Abstract

Cost-sensitive learning addresses the issue of classification in the presence of varying costs associated with different types of misclassification. In this paper, we present a method for solving multi-class cost-sensitive learning problems using any binary classification algorithm. This algorithm is derived using hree key ideas: 1) iterative weighting; 2) expanding data space; and 3) gradient boosting with stochastic ensembles. We establish some theoretical guarantees concerning the performance of this method. In particular, we show that a certain variant possesses the boosting property, given a form of weak learning assumption on the component binary classifier. We also empirically evaluate the performance of the proposed method using benchmark data sets and verify that our method generally achieves better results than representative methods for cost-sensitive learning, in terms of predictive performance (cost minimization) and, in many cases, computational efficiency.