CasFormer: Cascaded transformers for fusion-aware computational hyperspectral imaging

Abstract

Computational hyperspectral imaging (CHI) is a cutting-edge technique, which plays a pivotal role in breaking through the quality bottleneck of hyperspectral images (HSI). Among the techniques employed in this domain, the coded aperture snapshot spectral imaging (CASSI) system holds widespread recognition. Nevertheless, the imaging capability of CASSI remains limited due to the hardware conditions and the fragility of outcomes associated with the ill-posed blind reconstruction process. To this end, we propose a novel cascaded transformer architecture, termed CasFormer , specifically crafted for fusion-aware CHI by means of a dual-imaging mechanism. CasFormer facilitates the effective enhancement of hyperspectral imaging quality by fusing RGB images , with a focus on spatial and spectral domains . As the name suggests, CasFormer is primarily composed of a series of cascade-attention blocks, enabling the fusion of high-spatial-resolution RGB images through spatial coherence alignment and the recovery of spectrally sequential information more compactly and accurately. Furthermore, CasFormer incorporates physical constraints through a decoupling-based loss function, ensuring spatial consistency and spectral fidelity in the fusion-aware CHI process. Extensive experiments conducted across multiple datasets demonstrate the superiority of CasFormer in achieving high-quality imaging results compared to SOTA CHI algorithms. Our code and benchmark datasets will be openly accessible at https://github.com/danfenghong/Information_Fusion_CasFormer .