Complementarity-Aware Feature Fusion for Aircraft Detection via Unpaired Opt2SAR Image Translation

Abstract

Detecting aircraft in complex remote sensing scene has significant value for military and civilian applications. To overcome the interference of complex environmental factors and achieve high-accuracy detection performance, the comprehensive utilization of optical and SAR images for object detection has become a promising research direction. However, currently there are problems with optical and SAR fusion detection, such as difficulty in obtaining paired registration training data and incomplete consideration of feature elements in fusion model. To tackle these challenges, we present an aircraft detection method based on optical-SAR complementarity-aware feature fusion. Firstly, an unpaired image translation model based on scattering feature enhancement GAN (SFEG) is designed to generate SAR images that are pixel-level registered with the input optical image. On this basis, a complementarity-aware feature fusion detection network (CFFDNet) combining differential feature spatial-aware complementary (DFSC) units and gate-generated weighted fusion (GWF) units is proposed to enhance the effective features of single source image while improving the complementary fusion effect of multimodal features. Experiments on CORS-ADD and MAR20 datasets demonstrate that our method outperforms the compared classical single-modal and multimodal detection models. The latest code is available soon at: https://github.com/JimmyRSlab/Complementarity-aware-Feature-Fusion-for-Aircraft-Detection-via-Unpaired-Opt2SAR-Image-Translation.