RUSBoost: A Hybrid Approach to Alleviating Class Imbalance

TLDR

Class imbalance, where one class dominates the training data, causes traditional algorithms to produce suboptimal models and has been addressed by sampling and boosting techniques. This paper introduces RUSBoost, a hybrid sampling/boosting algorithm designed to learn from skewed data. RUSBoost combines random undersampling with AdaBoost, offering a simpler, faster alternative to SMOTEBoost, and its performance is evaluated across 15 datasets, four base learners, and four metrics. Experiments show RUSBoost matches or surpasses SMOTEBoost while being simpler and faster, making it a recommended choice for improving classification on imbalanced data.

Abstract

Class imbalance is a problem that is common to many application domains. When examples of one class in a training data set vastly outnumber examples of the other class(es), traditional data mining algorithms tend to create suboptimal classification models. Several techniques have been used to alleviate the problem of class imbalance, including data sampling and boosting. In this paper, we present a new hybrid sampling/boosting algorithm, called RUSBoost, for learning from skewed training data. This algorithm provides a simpler and faster alternative to SMOTEBoost, which is another algorithm that combines boosting and data sampling. This paper evaluates the performances of RUSBoost and SMOTEBoost, as well as their individual components (random undersampling, synthetic minority oversampling technique, and AdaBoost). We conduct experiments using 15 data sets from various application domains, four base learners, and four evaluation metrics. RUSBoost and SMOTEBoost both outperform the other procedures, and RUSBoost performs comparably to (and often better than) SMOTEBoost while being a simpler and faster technique. Given these experimental results, we highly recommend RUSBoost as an attractive alternative for improving the classification performance of learners built using imbalanced data.