Spectral Reconstruction From Satellite Multispectral Imagery Using Convolution and Transformer Joint Network

Abstract

Spectral reconstruction based on satellite multispectral (MS) images can produce high spatial resolution hyperspectral (HS) images at a reasonable cost, significantly expanding the application of satellite-based HS remote sensing. As a challenging ill-posed problem, existing methods have difficulty making full use of local and global information of space and spectra to guide the reconstruction, resulting in limited accuracy in large-scale scenes with complex ground features and severe spectral mixing. In this article, we propose a novel convolution and Transformer joint network (CTJN) to address the challenge of high-accuracy spectral reconstruction in complex scenes. The CTJN is cascaded with shallow feature extraction modules (SFEMs) and deep feature extraction modules (DFEMs), which can explore local spatial features and global spectral features. Besides, a high-frequency Transformer block (HF-TB) is designed to highlight the detailed features of the images to prevent significant high-frequency information loss, which could improve the reconstruction results in regions with drastic feature changes. Moreover, a spatial–spectral recalibration block (SSRB) is proposed to perform explicit constraints on the reconstructed points by exploiting the correlation among neighboring pixels and adjacent spectra. Extensive experimental results on four HS–MS datasets and one MS dataset demonstrate that the proposed CTJN outperforms the state-of-the-art methods in large-scale and small-scale scenes.