Is Sampling Heuristics Necessary in Training Deep Object Detectors

Abstract

In training deep object detectors, the problem of foreground-background imbalance has been addressed by several heuristic methods, such as online hard example mining (OHEM), Focal Loss, and gradient harmonizing mechanism (GHM). These methods either re-sample the training examples (i.e. hard sampling), or re-weight them discriminatively (i.e. soft sampling). In this paper, we challenge the necessity of such hard/soft sampling heuristics in training deep object detectors. First, without hard/soft sampling, we reveal that the scale and the stability of the classification loss greatly influence the final accuracy. Thus, we propose a guided loss scaling technique to control the classification loss during training, without using any hyper-parameter. We also propose to optimally initialize the model to ensure the stability of the classification loss. Moreover, we propose an adaptive thresholding technique to refine predictions during inference. These three ingredients constitute our Sampling-Free mechanism, which is fully data diagnostic and avoids the laborious hyper-parameter search for hard/soft sampling. We verify the effectiveness of our Sampling-Free mechanism in training one-stage, two-stage, multi-stage, and anchor-free object detectors, where our method always achieves higher accuracy on COCO and PASCAL VOC datasets. We also use the Sampling-Free mechanism for instance segmentation to demonstrate its generalization ability. Code is released at: this https URL