Certifiably Adversarially Robust Detection of Out-of-Distribution Data

TLDR

Deep neural networks are overconfident on out‑of‑distribution inputs, a flaw that jeopardizes safety‑critical systems by preventing reliable uncertainty assessment. The study seeks certifiable worst‑case guarantees for OOD detection by requiring low confidence not only at the OOD point but also throughout its surrounding l∞‑ball. To achieve this, the authors employ interval bound propagation to upper‑bound the maximal confidence within the l∞‑ball and train the network to minimize that bound. They show that such bounds generalize beyond the OOD data seen during training and can be obtained without a substantial drop in classification accuracy.

Abstract

Deep neural networks are known to be overconfident when applied to out-of-distribution (OOD) inputs which clearly do not belong to any class. This is a problem in safety-critical applications since a reliable assessment of the uncertainty of a classifier is a key property, allowing the system to trigger human intervention or to transfer into a safe state. In this paper, we aim for certifiable worst case guarantees for OOD detection by enforcing not only low confidence at the OOD point but also in an $l_\infty$-ball around it. For this purpose, we use interval bound propagation (IBP) to upper bound the maximal confidence in the $l_\infty$-ball and minimize this upper bound during training time. We show that non-trivial bounds on the confidence for OOD data generalizing beyond the OOD dataset seen at training time are possible. Moreover, in contrast to certified adversarial robustness which typically comes with significant loss in prediction performance, certified guarantees for worst case OOD detection are possible without much loss in accuracy.