PMACNet: Parallel Multiscale Attention Constraint Network for Pan-Sharpening

Abstract

Pan-sharpening, a task involving information fusion, entails merging panchromatic (PAN) images with high spatial resolution and low-resolution multispectral (LRMS) images in order to obtain high-resolution multispectral (HRMS) images. Due to deep learning’s excellent regression capabilities, it has recently become the dominating technique for this assignment. Meanwhile, the development of the transformer, a novel deep learning architecture for natural language processing, has provided researchers with new insights. In this letter, we seek to extend transformer’s excellent mechanisms to pixel-level fusion challenges. We designed a parallel convolutional neural network structure for learning both the regions of interest from the LRMS images and the residuals required for regression to HRMS images. Then, in our proposed pixelwise attention constraint (PAC) module, the residuals will be changed utilizing the learned region of interest. In addition, we presented a novel multireceptive-field attention block (MRFAB) to frame our network. Experiments on two datasets also show that our work is better than the mainstream algorithms at both indicators and visualization.