Task-Specific Importance-Awareness Matters: On Targeted Attacks Against Object Detection

Abstract

Targeted Attacks on Object Detection (TAOD) aim to deceive the victim detector into recognizing a specific instance as the predefined target category while minimizing the changes to the predicted bounding box of that instance. Yet, this kind of flexible attack paradigm, which is capable of manipulating the decision outcome of the victim detector, received limited attention, especially in the context of attacking object detection in optical remote sensing images, where relevant research remains a blank. To fill this gap, this paper concentrates on TAOD in optical remote sensing images, and pays attention to a fundamental question, how to deploy TAOD via the raw predictions (the predictions before non-maximum suppression) of a victim detector. In this regard, we depart from widely adopted task-independent importance measurements and hard-weighted ensemble optimization schemes present in existing methods. Instead, we first define the task-specific importance score, which considers both the qualities and the attack costs of predictions. Further, we propose the Task-Specific Importance-Aware Candidate Predictions Selection Scheme (TSIA-CPSS) alongside the Soft-Weighted Ensemble Optimization Scheme (SW-EOS). A total of eleven detectors on DIOR and DOTA, two commonly employed benchmarks, are included to comprehensively evaluate our approach. Furthermore, we indicate that the effectiveness of our approach is not only substantial for vanilla TAOD, but also can be better generalized to extended scenarios, which encompasses random TAOD, TAOD on oriented object detection, and targeted patch attacks, highlighting the noteworthy potential of our approach. Our codes will be released on Github.