A Multisensor Data Fusion Model for Semantic Segmentation in Aerial Images

Abstract

Semantic segmentation in high-resolution aerial images is a fundamental and challenging task with a wide range of applications. Although many segmentation methods with convolutional neural networks have achieved inspiring results, it is still difficult to distinguish regions with similar spectral features only using high-resolution data. Besides, the traditional data-independent upsampling methods may lead to sub-optimal results. This letter proposes a multisensor data fusion model (MSDFM). Following the classical encoder-decoder structure, MSDFM regards colored digital surface models (colored-DSM) data as a complementary input for further detailed feature extraction. A data-dependent upsampling (DUpsampling) method is adopted in the decoder stage instead of the common up-sampling approaches to improve the classification accuracy of pixels of the small objects. Extensive experiments on Vaihingen and Potsdam datasets demonstrate that our proposed MSDFM outperforms most related models. Significantly, segmentation performance for the car category surpasses state-of-the-art methods over the ISPRS Vaihingen dataset.