A Human–Computer Fusion Framework for Aircraft Recognition in Remote Sensing Images

Abstract

Existing aircraft recognition methods usually regard recognition as an isolated classification problem, supposing that aircraft detection has finished. These methods use image slices each containing single aircraft as input, which is often not the case in practice. In order to recognize aircraft in remote sensing images that contain multiple objects and background, we propose a human-computer fusion framework that combines the advantages of human and computer. First, we propose candidate aircraft using human eye tracking, making use of the efficient and accurate search ability of human. Then, we propose a two-step recognition method, which simulates the recognition process of image analysts, to identify the types of candidate aircraft. In the first step, we recognize aircraft using fully connected features of a fine-tuned convolutional neural network (CNN). If it does not work, we go to the second step. In the second step, we use convolutional features extracted from the fine-tuned CNN for recognition. To improve the representational ability of convolutional features, we introduce the Fisher Vector encoding. Thorough experiments demonstrate that the proposed framework is effective, surpassing state-of-the-art methods on recognition performance.