Self-Supervised Feature Learning Based on Spectral Masking for Hyperspectral Image Classification

Abstract

Deep learning has emerged as a powerful method for hyperspectral image (HSI) classification. However, a significant prerequisite for HSI classification using deep learning is enough labeled samples, which is both time-consuming and labor-intensive. Yet, labeled samples are essential for training deep learning models. This paper proposes an HSI classification method based on the self-supervised learning of spectral masking (SSLSM). The method mainly includes two steps: self-supervised pre-training and fine-tuning. First, considering the rich spectral information of HSI, we propose masked spectral reconstruction as the pretext task. The unmasked data is input into the encoder and decoder sequentially, which are composed of a multi-layer transformer, for feature learning for masked spectral reconstruction. Second, we use reference samples to fine-tune the network, and the encoder and decoder are innovatively cascaded for deep semantic feature extraction, which can further improve the ability of feature extraction in the downstream classification tasks. Experiment results show that, compared with other methods, the SSLSM obtains the highest classification accuracy of 96.52%, 97.03%, and 96.70% on the Indian Pines dataset, Pavia University dataset, and Yancheng Wetlands dataset, respectively. Our method can also be applied to other HSI datasets, and the codes will be available from https://github.com/CIRSM-GRoup/2023-TGRS-SSLSM.