The Lov\\'asz-Softmax loss: A tractable surrogate for the optimization of\n the intersection-over-union measure in neural networks

Abstract

The Jaccard index, also referred to as the intersection-over-union score, is\ncommonly employed in the evaluation of image segmentation results given its\nperceptual qualities, scale invariance - which lends appropriate relevance to\nsmall objects, and appropriate counting of false negatives, in comparison to\nper-pixel losses. We present a method for direct optimization of the mean\nintersection-over-union loss in neural networks, in the context of semantic\nimage segmentation, based on the convex Lov\\'asz extension of submodular\nlosses. The loss is shown to perform better with respect to the Jaccard index\nmeasure than the traditionally used cross-entropy loss. We show quantitative\nand qualitative differences between optimizing the Jaccard index per image\nversus optimizing the Jaccard index taken over an entire dataset. We evaluate\nthe impact of our method in a semantic segmentation pipeline and show\nsubstantially improved intersection-over-union segmentation scores on the\nPascal VOC and Cityscapes datasets using state-of-the-art deep learning\nsegmentation architectures.\n