CNN-Based UAV Detection and Classification Using Sensor Fusion

Abstract

This paper proposes a detection and classification method for unmanned aerial vehicles, commonly called drones, using sensor fusion schemes. Datasets for drone detection and classification are collected by field measurements of actual drones using the optical camera, radar, and audio microphone as well as obtained from open online databases. In the first stage of the proposed method, drone detection and classification are conducted using the convolutional neural network (CNN) models separately trained by the optical images, radar range-Doppler maps, and audio spectrograms. Then, the CNN output probabilities are combined by the multinomial logistic regression to improve the drone surveillance accuracy through the fusion of the optical, radar, and audio sensors. Numerical simulations are performed with the experimental data and the open datasets. From the results, it is verified that the proposed sensor fusion method can improve the drone detection accuracy by up to 15.6% and can enhance the drone classification accuracy by up to 28.1% in terms of the F-score, compared to individual sensing schemes.