Large-Margin Softmax Loss for Convolutional Neural Networks

TLDR

Cross‑entropy loss with softmax is widely used in CNNs but does not explicitly promote discriminative feature learning. This paper introduces a generalized large‑margin softmax (L‑Softmax) loss to encourage intra‑class compactness and inter‑class separability. L‑Softmax adjusts the margin, mitigates overfitting, and can be optimized with standard stochastic gradient descent. Experiments on four benchmarks show that L‑Softmax yields more discriminative features and significantly improves performance on visual classification and verification tasks.

Abstract

Cross-entropy loss together with softmax is arguably one of the most common used supervision components in convolutional neural networks (CNNs). Despite its simplicity, popularity and excellent performance, the component does not explicitly encourage discriminative learning of features. In this paper, we propose a generalized large-margin softmax (L-Softmax) loss which explicitly encourages intra-class compactness and inter-class separability between learned features. Moreover, L-Softmax not only can adjust the desired margin but also can avoid overfitting. We also show that the L-Softmax loss can be optimized by typical stochastic gradient descent. Extensive experiments on four benchmark datasets demonstrate that the deeply-learned features with L-softmax loss become more discriminative, hence significantly boosting the performance on a variety of visual classification and verification tasks.