SAR–Optical Image Matching Using Self-Supervised Detection and a Transformer–CNN-Based Network

Abstract

The SAR-optical image matching is a research hotspot in the field of remote sensing. In this letter, we propose an end-to-end learning based SAR-optical image matching algorithm. Initially, the algorithm applies local normalization filter on multi-model image pairs, and then detects keypoints with high repeatability and ease of matching via a self-supervised keypoint detection network. The detected keypoints are fed into our proposed Transformer-CNN dual branch feature description siamese network, extracting global and local contextual information of images to gain robust feature descriptors. In the training phase, we adopt a two-stage training strategy in the form of description then detection, which enables the keypoint detection network to learn in conjunction with the output of the feature description network, so as to obtain more robust keypoints. Experimental results show that our method achieves the average Root Mean Square Error (aRMSE) of 3.37 and 2.98 pixels on the test data, outperforming the three compared existing advanced algorithms.