Hashing-Based Deep Metric Learning for the Classification of Hyperspectral and LiDAR Data

Abstract

Multisource remote sensing data provide abundant and complementary information for land cover classification. Existing classification methods mainly focus on designing a multi-stream deep network to extract separate features of each single-source data, then adopting a fusing strategy to combine these extracted features for final classification. However, this kind of method neglects the sample correlation of single-source and cross-source data, which may deliver an unsatisfactory classification result when dealing with high intraclass-variability and low interclass-variability samples. To this end, a novel hashing-based deep metric learning (HDML) method is proposed for hyperspectral images (HSIs) and light detection and ranging (LiDAR) data classification in this paper. First, a two-stream deep network is built to extract the spectral-spatial features of HSI and the elevation features of LiDAR, respectively. To fully use the complementary and correlated information of HSI and LiDAR data, we adopt attention-based feature fusion (AFF) modules to deliver a high-discrimination fused feature both for cross-source and single-source feature fusion. Then, the extracted features are fed into fully connected layers to generate class probabilities, respectively. Different from most existing methods that only utilize semantic information of samples, we elaborately designed a loss function to simultaneously consider the label-based semantic loss and hashing-based metric loss. Finally, a decision-level fusion strategy is adopted to further improve the classification results. Extensive experiments on three public HSI and LiDAR data sets demonstrate the effectiveness of the proposed method over some state-of-the-art approaches.