Multiscale and Multidirection Feature Extraction Network for Hyperspectral and LiDAR Classification

Abstract

Deep learning (DL) plays an increasingly important role in earth observation by multi-source remote sensing. However, the current DL-based methods do not make fully use of the complementary information among multi-source remote sensing data, such as hyperspectral image (HSI) and light detection and ranging (LiDAR) data, and lack the consideration of multi-scale, directional and fine-grained features. To address these issues, a multi-scale and multi-direction feature extraction network is proposed in this article. Specifically, multi-scale spatial feature (MSSpaF) module is designed to extract the multi-scale spatial features, and then these features are fused by feature concatenation operation. In addition, multi-direction spatial feature (MDSpaF) module is designed to further extract multi-direction and frequency information, employing cross-layer connection and multi-scale feature fusion strategy to improve fineness of the proposed network. Moreover, spectral feature (SpeF) module is employed to provide detailed spectral information for enhancing the expression ability of multi-scale features. Experimental results on three different datasets demonstrate the superior classification performance of the proposed framework. The source code of this method can be found at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/lyywowo/MSMD-Net</uri> .