MCFNet: Multiscale Cross-Domain Fusion Network for HSI and LiDAR Data Joint Classification

Abstract

Hyperspectral image (HSI) encompasses abundant spatial and spectral details, while Light Detection and Ranging (LiDAR) delivers precise elevation data. The amalgamation of HSI and LiDAR data significantly improves the precision of image classification. However, most methods focus solely on spatial features while neglecting frequency domain information, limiting the ability of deep models to characterize land cover. Furthermore, how to establish a sufficient interaction between different modalities is also an important issue. In this paper, we propose a novel multiscale cross-domain fusion network (MCFNet) for joint classification of HSI and LiDAR data. The main idea is that the wavelet transform can provide details at different resolutions simultaneously, supplementing spatial domain information and enriching feature representation. In addition, the multimodal fusion module (MFM) guided by HSI and the cross-domain fusion module (CDFM) strategy are developed to integrate features from diverse modalities and domains, respectively. Specifically, frequency domain features are extracted by discrete wavelet transform, and spatial domain features of the image are captured through a set of convolution operations. Then interactive fusion is performed by MFM and CDFM, and finally the integrated features are categorized using a classification module. Extensive experiments on three widely-used HSI and LiDAR datasets indicate that MCFNet outperforms the SOTA methods. The code will be available at https://github.com/MSFLabX/MCFNet.