A Multiscale Deep Middle-level Feature Fusion Network for Hyperspectral Classification

Abstract

Recently, networks consider spectral-spatial information in multiscale inputs less, even though there are some networks that consider this factor, however these networks cannot guarantee to get optimal features, which are extracted from each scale input. Furthermore, these networks do not consider the complementary and related information among different scale features. To address these issues, a multiscale deep middle-level feature fusion network (MMFN) is proposed in this paper for hyperspectral classification. In MMFN, the network fully fuses the strong complementary and related information among different scale features to extract more discriminative features. The training of network contains two stages: the first stage obtains the optimal models corresponding to different scale inputs and extracts the middle-level features under the corresponding scale model. It can guarantee the multiscale middle-level features are optimal. The second stage fuses the optimal multiscale middle-level features in the convolutional layer, and the subsequent residual blocks can learn the complementary and related information among different scale middle-level features. Moreover, the idea of identity mapping in residual learning can help the network obtain a higher accuracy when the network is deeper. The effectiveness of our method is proved on four HSI data sets and the experimental results show that our method outperforms the other state-of-the-art methods especially with small training samples.