Mapping Coastal Wetlands Using Transformer in Transformer Deep Network on China ZY1-02D Hyperspectral Satellite Images

Abstract

Coastal wetlands mapping is a big challenge in remote sensing fields because of similar spectrum of different ground objects and their severe fragmentation and spatial heterogeneity. In this paper, we propose a Hyperspectral Image Transformer iN Transformer (HSI-TNT) method for mapping coastal wetlands on ZY1-02D hyperspectral images, which uses two transformer deep networks to fuse local and global features. First, we put forward the idea that each hyperspectral pixel can be considered as a superpixel in spectral dimension, and subsequent position encodings are employed aiming to retain spatial information. After that, in each HSI-TNT block, the local information between pixels is extracted by Inner T-Block, and added to the patch space by linear transformation to extract the global information by Outer T-Block. Finally, the stacked HSI-TNT block, also known as HSI-TNT framework, is used for classification and mapping. Experimental results show that HSI-TNT achieves the best results on both Yancheng and Yellow River Delta wetlands data, with OCA of 95.57% and 93.69% respectively. The HSI-TNT combined with ZY1-02D satellite hyperspectral data has huge potentials in mapping coastal wetlands.