Cascadic Multireceptive Learning for Multispectral Pansharpening

Abstract

Pansharpening refers to the fusion of a panchromatic (PAN) image with high spatial resolution and a multispectral (LRMS) image with low spatial resolution to obtain a high spatial resolution multispectral (HRMS) image, which is beneficial to visual display and geographic research. Recently, many deep learning (DL) methods have been proposed to address the pansharpening problem, but still a few examples of DL-based techniques are designed from the perspective of a better receptive field while the scale of features greatly varies among different ground objects. In this paper, we mainly focus on designing a cascadic multi-receptive learning module (CML-resblock) relying on the ResNet block, which can efficiently extract multi-scale features from both the PAN and LRMS images. Moreover, we propose a novel multiplication network preserving a physical significance, which uses deep neural networks (DNNs) to learn the coefficients of the pixel-wise restoration mapping and multiplies the up-sampled LRMS image with the learned coefficients to get the HRMS image. The two parts mentioned above constitute our cascadic multi-receptive learning network (CMLNet). Extensive experiments on both reduced-resolution and full-resolution images acquired by the WorldView-3, GaoFen-2, and QuickBird satellites show that the proposed approach outperforms state-of-the-art methods. Furthermore, additional experiments have been conducted to prove the generality of the CML-resblock and multiplication network. The code is available at: https://github.com/wajuda/CML.