Benchmarking Neural Network Robustness to Common Corruptions and\n Perturbations

Abstract

In this paper we establish rigorous benchmarks for image classifier\nrobustness. Our first benchmark, ImageNet-C, standardizes and expands the\ncorruption robustness topic, while showing which classifiers are preferable in\nsafety-critical applications. Then we propose a new dataset called ImageNet-P\nwhich enables researchers to benchmark a classifier's robustness to common\nperturbations. Unlike recent robustness research, this benchmark evaluates\nperformance on common corruptions and perturbations not worst-case adversarial\nperturbations. We find that there are negligible changes in relative corruption\nrobustness from AlexNet classifiers to ResNet classifiers. Afterward we\ndiscover ways to enhance corruption and perturbation robustness. We even find\nthat a bypassed adversarial defense provides substantial common perturbation\nrobustness. Together our benchmarks may aid future work toward networks that\nrobustly generalize.\n