Remote Sensing Fine-Grained Ship Data Augmentation Pipeline With Local-Aware Progressive Image-to-Image Translation

Abstract

Remote sensing image ship fine-grained classification is a challenging vision problem due to factors such as inter-class similarity, real-world image scarcity, and class imbalance. Data augmentation aims to solve these problems from the data perspective. Most of these methods cannot work effectively in complex scenes, which restricts their practical application. Here, we propose a novel data augmentation pipeline based on local-aware image translation to achieve the representation mapping between cross-domain corresponding instances. Our pipeline contains three modules: Imaging Simulation System (ISS), Local-aware Progressive Image-to-Image Translation (LoPIT), and Image Harmonization (IH) modules. The ISS module generates simulated images with correct appearance and diverse features based on the input requirement information. To tackle the domain gap between simulated image and real-world image, we propose the Local-aware CycleGAN in the LoPIT module to achieve mapping based on local-aware learning and apply two sub-modules to progressively complete the remote sensing image global cross-domain translation. The IH module uses image harmonization technology to coordinate the visual appearance between the foreground and background to generate sufficient remote sensing ship images with precise representation, photorealistic style, and harmonious features. Moreover, we present a mixed dataset including real-world images and our synthetic images for remote sensing image fine-grained ship classification. Our dataset named RSSA-12 contains 12 categories of ship targets in 3831 images with high-quality annotated category labels, effectively alleviating the long-tail problem of existing datasets. Experimental results demonstrate that our progressive pipeline outperforms the state-of-the-art data augmentation method on the remote sensing fine-grained ship classification task.