Object Detection Made Simpler by Eliminating Heuristic NMS

Abstract

It is valuable and promising to remove post-processing non-maximum suppression (NMS) for object detectors, making detectors simpler and purely end-to-end. Removing NMS is possible if the object detector can identify only one positive sample for prediction for each ground-truth object instance in an image. In this work, we propose a compact and plug-in head, named PSS head, which can be attached to any one-stage detectors to make them NMS-free. Specifically, the PSS head works by automatically selecting a positive sample for each instance to be detected, so that the detectors with our PSS head can directly remove NMS. The success of our PSS head lies in three aspects, namely one-to-one label assignment, stop-gradient operation for eliminating optimization conflicts, and the pss loss and ranking loss specifically designed for the PSS head. Experiments on the COCO dataset demonstrate the effectiveness of our method. In particular, when compared with stage-of-the-art NMS-free methods, our (attaching PSS head to VFNET) achieves 44.0% mAP, which exceeds the 41.5% mAP of DeFCN with a large margin. When taking Res2Net-101-DCN as backbone network, our achieves 50.3% mAP on the COCO test set, which is a promising performance even among NMS-based methods.