Comparing Oversampling Techniques to Handle the Class Imbalance Problem: A Customer Churn Prediction Case Study

TLDR

Customer retention is critical for service‑based firms, especially telecoms, yet predictive models perform poorly when training data are highly imbalanced—a situation where one class dominates the other and is typically addressed by over/under sampling. This study surveys six sampling techniques and compares their performance, while also evaluating four rule‑generation algorithms on publicly available datasets. The authors benchmarked MTDF, SMOTE, ADASYN, TOP‑N reverse k‑NN, MWMOS, and I‑COS, and assessed LEM2, covering, exhaustive, and genetic algorithms. Results show that MTDF combined with genetic‑algorithm rule generation achieved the best overall predictive performance among the evaluated methods.

Abstract

Customer retention is a major issue for various service-based organizations particularly telecom industry, wherein predictive models for observing the behavior of customers are one of the great instruments in customer retention process and inferring the future behavior of the customers. However, the performances of predictive models are greatly affected when the real-world data set is highly imbalanced. A data set is called imbalanced if the samples size from one class is very much smaller or larger than the other classes. The most commonly used technique is over/under sampling for handling the class-imbalance problem (CIP) in various domains. In this paper, we survey six well-known sampling techniques and compare the performances of these key techniques, i.e., mega-trend diffusion function (MTDF), synthetic minority oversampling technique, adaptive synthetic sampling approach, couples top-N reverse k-nearest neighbor, majority weighted minority oversampling technique, and immune centroids oversampling technique. Moreover, this paper also reveals the evaluation of four rules-generation algorithms (the learning from example module, version 2 (LEM2), covering, exhaustive, and genetic algorithms) using publicly available data sets. The empirical results demonstrate that the overall predictive performance of MTDF and rules-generation based on genetic algorithms performed the best as compared with the rest of the evaluated oversampling methods and rule-generation algorithms.