Remote Sensing Scene Classification Based on Semantic-Aware Fusion Network

Abstract

The remote sensing scene classification (RSSC) based on convolutional neural networks (CNNs) are generally limited by the complex background interference and the difficulty of identifying key targets in image. Thus, this letter proposes a semantic-aware fusion network for RSSC, abbreviated as SAF-Net, to better construct discriminative features and effectively fuse features for classification. The proposed SAF-Net, which employs the ResNet50 pretrained on the ImageNet dataset as the backbone network, mainly contains the semantic-aware module and the multilayer feature fusion module (MFFM). The semantic-aware module utilizes a spatial enhanced module (SEM) and a covariance channel attention module (CCAM) to accurately capture the discriminative semantic features. It can precisely identify and extract the essential semantic elements in image, such as distinct object types and their spatial distributions. Then, the MFFM uses the features learned by the semantic-aware module to guide other layers for effective feature fusion through a self-attention mechanism. It can not only enriches the feature representation of SAF-Net but also ensure the effective fusion of the semantic information. Extensive comparisons and ablation experiments on remote sensing datasets demonstrate the effectiveness of the proposed SAF-Net, and verify that it can greatly improve the classification performance.