Edge and Skeleton Guidance Network for Salient Object Detection in Optical Remote Sensing Images

Abstract

The salient object detection of optical remote sensing images (ORSI-SOD) is an important research direction in ORSI processing, which has achieved promising results in the last few years. Many recent works heavily rely on feature learning of regions for the improvement of the detection accuracy, while neglecting the concrete role of edge and skeleton information in the calculation. In this work, we propose a two-stage edge and skeleton guidance network (ESGNet) for ORSI-SOD in a coarse-to-fine way, and further demonstrate that the fused features of edge and skeleton are essential for ORSI-SOD. In the first stage, we construct the spatial graph attention (SGA) module for saliency features to generate an initial saliency map, and apply the spatial self-optimization (SSO) to enhance edge and skeleton features. The multi-level interactive fusion (MIF) module is used for the adequate integration of edge and skeleton features into saliency features. In the second stage, with the aim to accomplish better prediction of salient object localization and shape, the feature enhancement integration operation is introduced to recover object details from the learned edge and skeleton features. Extensive experiments on three public ORSI-SOD datasets demonstrate that our ESGNet achieves competitive performance with the state-of-the-art methods and also confirms the importance of edge and skeleton information for ORSI-SOD. Meanwhile, generalizability experiments on natural image datasets show that our method is competent for many types of SOD tasks. The code and results of our method are available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/aoao0206/ESGNet</uri> .