Training deep neural networks on imbalanced data sets

TLDR

Deep learning has gained widespread use across pattern recognition, computer vision, and natural language processing, yet most research assumes balanced class labels, leaving performance on the common real‑world imbalanced datasets largely unexplored. This study addresses classification with deep networks on imbalanced datasets. We propose a novel loss function, mean false error, and its improved variant mean squared false error, to train deep networks that treat majority and minority class errors equally. The new loss functions effectively balance classification errors and outperform conventional methods on imbalanced datasets in deep neural network experiments.

Abstract

Deep learning has become increasingly popular in both academic and industrial areas in the past years. Various domains including pattern recognition, computer vision, and natural language processing have witnessed the great power of deep networks. However, current studies on deep learning mainly focus on data sets with balanced class labels, while its performance on imbalanced data is not well examined. Imbalanced data sets exist widely in real world and they have been providing great challenges for classification tasks. In this paper, we focus on the problem of classification using deep network on imbalanced data sets. Specifically, a novel loss function called mean false error together with its improved version mean squared false error are proposed for the training of deep networks on imbalanced data sets. The proposed method can effectively capture classification errors from both majority class and minority class equally. Experiments and comparisons demonstrate the superiority of the proposed approach compared with conventional methods in classifying imbalanced data sets on deep neural networks.