Global–Local 3-D Convolutional Transformer Network for Hyperspectral Image Classification

Wenchao Qi, Changping Huang, Yibo Wang, Xia Zhang, Weiwei Sun, Lifu Zhang

IEEE Transactions on Geoscience and Remote Sensing · 2023 · 59 citations · 53 references

Concepts

Abstract

Benefiting from powerful feature extraction capabilities, convolutional neural networks (CNNs) have gained prominence in hyperspectral image (HSI) classification. Nevertheless, with restricted receptive fields of convolution kernels, CNN-based methods fail to learn complex characteristics of long-range sequences. Meanwhile, vision transformer allows us to learn long-range dependencies in a global view, but local region features are ignored. To overcome these limitations, we propose a novel method entitled global-local three-dimensional convolutional transformer network (GTCT), where 3-D convolution is embedded in a dual-branch transformer to simultaneously capture global-local associations in both spectral and spatial domains. In particular, the global-local spectral convolutional transformer (GECT) is designed to exploit global spectral sequence signatures and local spectral relationships between bands. Symmetrically, the global-local spatial convolutional transformer (GACT) is devised to exploit local spatial context features and global interactions among different pixels. In addition, multiscale global-local spectral-spatial information is adaptively fused with trainable weights by the weighted multiscale spectral-spatial feature interaction (WMSFI) module. It is worth noting that a spectral-spatial global attention mechanism (SSGAM) is incorporated into multi-head convolutional attention to further integrate discriminative spectral-spatial information. Extensive experiments on four HSI datasets, including GF-5 and ZY1-02D satellite hyperspectral images, demonstrate the superiority of the proposed GTCT method over other state-of-the-art algorithms with fewer parameters and lower floating-point operations (FLOPs) in practical applications.

References

53