Beyond softmax loss: Intra-concentration and inter-separability loss for classification

Abstract

In the past years, most works have focused on designing an indigenous network architecture to advance progress in classification, but another potential opportunity for improvement, that is, research on classification losses, is underdeveloped. Although some new losses have been proposed, most of them either are variants of softmax loss or should combine with softmax loss. Hence, the inherent deficiencies of softmax loss, such as sensitiveness, class-balanced restriction, closed-set limitation, non-scale-invariance and incoordination between the intraclass distance and interclass distance, cannot be completely overcome. In light of this, we pave a new way to design a loss that has no relation to softmax loss and can avoid its weaknesses. We also propose an efficient algorithm to optimize the new loss that can circumvent computing the complicated gradients of a fraction, and the convergence is theoretically ensured. Extensive experimental results on benchmark datasets demonstrate that the new loss is competitive with state-of-the-art losses for classification. Additionally, other specially designed experiments show that the new loss is also effective at handling class-imbalanced problems, is robust in addressing outliers and can discover samples of unseen classes in open-set cases.