SCT-CR: A synergistic convolution-transformer modeling method using SAR-optical data fusion for cloud removal

Abstract

Traditional CNNs struggle with SAR and optical image fusion cloud removal due to SAR image noise, feature space differences and random cloud distribution. This often leads to blurred results with less texture information. This paper proposes a synergistic convolution-transformer cloud removal method (SCT-CR), which is based on a specially designed synergistic convolution module that enables the synergistic fusion of SAR and optical imagery. The proposed network employs a transformer module in the high-dimensional section to better perceive the contextual information of the image and achieve intelligent extraction of global image features. The proposed SCT-CR network successfully addresses the problem of image blur in generated images and makes full use of the texture information present in SAR images. The SCT-CR model is tested on the spectral properties and recovery of visual effects. The experimental results on public datasets SEN12MS-CR and LuojiaSET-OSFCR show that the proposed model has stable and optimal performance. On the SEN12MS-CR dataset, the proposed model improves the SSIM metrics by 15.7 %, 10.2 %, 4.9 %, and 0.5 % compared to the SAR2OPT, SarOptcAGN, DSen2-CR, and GLF-CR models, respectively. On the LuojiaSET-OSFCR dataset, it was improved by 20.0 %, 10.0 %, 6.6 %, and 1.9 %, respectively.