MSTNet-KD: Multilevel Transfer Networks Using Knowledge Distillation for the Dense Prediction of Remote-Sensing Images

Abstract

Recently, methods based on convolutional neural networks have achieved good results in the dense prediction of remote-sensing images, particularly when employing normalized digital surface models. However, most existing methods use multiscale convolution and attention methods to mine multimodal feature information without considering the differences and complementarities between the two features. Moreover, previous studies have prioritized model segmentation performance and ignored parametric issues, which makes it difficult to deploy the model in practical applications. To address this challenge, we designed a multilevel semantic transfer network (MSTNet) for the dense prediction of remote-sensing images using a knowledge-distillation approach to adaptively select useful semantic information for the transfer network. We designed a multilevel semantic knowledge alignment distillation framework (MSKA) to enable a compact student model to learn the semantic information extracted from a complex model. The MSKA framework comprises three main components: cross-layer semantic alignment, dynamic semantic aggregation, and softening learning for semantic information transfer and predictive label softening. Experiments on the Vaihingen and Potsdam datasets showed that the student network employing the MSKA framework achieved excellent segmentation performance with only 8.88M parameters and 2.09 gigaFLOPs in terms of computational costs compared with current state-of-the-art methods. The source code and results are available at https://github.com/LYZ00918/MSKANet.