Adversarial Examples for Semantic Segmentation and Object Detection

TLDR

Adversarial examples—images with imperceptible perturbations—have been shown to cause deep networks to fail in image classification, and segmentation and detection rely on classifying many targets such as pixels or object proposals. This work extends adversarial attacks to semantic segmentation and object detection by optimizing a loss over all targets to generate perturbations. The authors introduce Dense Adversary Generation (DAG), an algorithm that applies this loss‑based optimization to state‑of‑the‑art segmentation and detection networks. They demonstrate that the resulting perturbations transfer across networks with different training data, architectures, and tasks—especially between networks sharing the same architecture—and that combining heterogeneous perturbations improves transfer, enabling effective black‑box attacks.

Abstract

It has been well demonstrated that adversarial examples, i.e., natural images with visually imperceptible perturbations added, cause deep networks to fail on image classification. In this paper, we extend adversarial examples to semantic segmentation and object detection which are much more difficult. Our observation is that both segmentation and detection are based on classifying multiple targets on an image (e.g., the target is a pixel or a receptive field in segmentation, and an object proposal in detection). This inspires us to optimize a loss function over a set of targets for generating adversarial perturbations. Based on this, we propose a novel algorithm named Dense Adversary Generation (DAG), which applies to the state-of-the-art networks for segmentation and detection. We find that the adversarial perturbations can be transferred across networks with different training data, based on different architectures, and even for different recognition tasks. In particular, the transfer ability across networks with the same architecture is more significant than in other cases. Besides, we show that summing up heterogeneous perturbations often leads to better transfer performance, which provides an effective method of black-box adversarial attack.