Transferring Deep Convolutional Neural Networks for the Scene Classification of High-Resolution Remote Sensing Imagery

TLDR

Learning efficient image representations is central to remote sensing scene classification, yet existing hand‑engineered or unsupervised methods produce only limited mid‑level features, while deep CNNs have excelled in object recognition but their application to high‑resolution remote sensing remains unclear. This study investigates how to transfer features from pre‑trained CNNs to high‑resolution remote sensing scene classification. We propose two feature extraction scenarios—using fully‑connected layer activations as image features, and extracting multi‑scale dense features from the last convolutional layer and encoding them with conventional coding methods—plus a tentative fusion of features from different CNN models. Experiments on two public datasets show that these transferred features, even with a simple linear classifier, achieve remarkable performance, surpassing state‑of‑the‑art and demonstrating that pre‑trained CNN features generalize well and are more expressive than low‑ or mid‑level alternatives.

Abstract

Learning efficient image representations is at the core of the scene classification task of remote sensing imagery. The existing methods for solving the scene classification task, based on either feature coding approaches with low-level hand-engineered features or unsupervised feature learning, can only generate mid-level image features with limited representative ability, which essentially prevents them from achieving better performance. Recently, the deep convolutional neural networks (CNNs), which are hierarchical architectures trained on large-scale datasets, have shown astounding performance in object recognition and detection. However, it is still not clear how to use these deep convolutional neural networks for high-resolution remote sensing (HRRS) scene classification. In this paper, we investigate how to transfer features from these successfully pre-trained CNNs for HRRS scene classification. We propose two scenarios for generating image features via extracting CNN features from different layers. In the first scenario, the activation vectors extracted from fully-connected layers are regarded as the final image features; in the second scenario, we extract dense features from the last convolutional layer at multiple scales and then encode the dense features into global image features through commonly used feature coding approaches. Extensive experiments on two public scene classification datasets demonstrate that the image features obtained by the two proposed scenarios, even with a simple linear classifier, can result in remarkable performance and improve the state-of-the-art by a significant margin. The results reveal that the features from pre-trained CNNs generalize well to HRRS datasets and are more expressive than the low- and mid-level features. Moreover, we tentatively combine features extracted from different CNN models for better performance.