Model-Guided Coarse-to-Fine Fusion Network for Unsupervised Hyperspectral Image Super-Resolution

Abstract

Fusing a low-resolution hyperspectral image (LrHSI) with an auxiliary high-resolution multispectral image (HrMSI) is a burgeoning technique to realize hyperspectral image super-resolution, in which learning-based methods have dominated the mainstream direction. However, the underutilization of degradation models and strong dependence on large-scale training triplets severely impedes their applicability and performance. Considering these issues, we reformulate the fusion task as a spectral mapping problem and hence propose an unsupervised model-guided coarse-to-fine fusion network. Specifically, degradation knowledge learning is first performed to fully excavate latent model information, which will serve as guidance for better mapping learning. Following that, a coarse-to-fine fusion network is constructed with a multi-scale attentional fusion module in the head and a coarse-to-fine structure in the tail. The former is deployed to achieve a more informative compression, and the latter is adopted to capture the spectral relationship, including a spectral degradation-guided subnetwork for group-by-group coarse reconstruction and a refinement subnetwork for inter-group correlation and dependencies. Finally, high-resolution HSI can be recovered via established spectral mapping. Extensive experiments on simulated and real datasets verify the superiority of our proposed method. The code is available at https://github.com/JiaxinLiCAS/UMC2FF_GRSL.