ADMM-HFNet: A Matrix Decomposition-Based Deep Approach for Hyperspectral Image Fusion

Abstract

Hyperspectral image (HSI) fusion refers to the reconstruction of a high-resolution HSI by fusing a low-resolution HSI (LR-HSI) and a high-resolution multispectral image (HR-MSI) over the same scene. Recently, researchers have proposed many approaches to handle this issue. However, most of them assume that both the spatial and spectral degradation functions are known, which are often limited or unavailable in reality. This article presents a novel model-driven deep network based on matrix decomposition, which considers spectral correlations and reasonably embeds the well-known observation models. Specifically, the proposed method decomposes the desired HSI into spectral basis and coefficients. The spectral basis can be estimated from the LR-HSI via singular value decomposition. To learn the coefficients, a learning model is constructed by merging the observation models, matrix decomposition, and sparsity into a concise single formulation. For solving the proposed model, a deep framework is built by unrolling the alternating direction method of multipliers (ADMM), dubbed as ADMM-HFNet, where the involved parameters can be learned adaptively. It is worth noting that the spectral basis cannot fully represent the desired HSI. Therefore, another model is constructed here to supplement the approximation error, which can also be embedded in the deep network. After checking on three datasets, it is found that the proposed method stands out from advanced competing techniques in both quality measures and visual effects.