Deep Learning-based Visual Tracking of UAVs using a PTZ Camera System

Abstract

The visual tracking problem of Unmanned Aerial Vehicles (UAVs) with a Pan-Tilt-Zoom (PTZ) camera system is the subject of this article. Given the background of an image acquired by a PTZ-camera system, a border encompassing a moving object is computed relying on optical flow and the histogram of oriented gradients. Deep Learning (DL) algorithms are trained off-line to decide on the existence of a UAV within this border. Particularly, the ResNet-50 model was trained using a collected data set with more than 50,000 registered positive images. Having identified a UAV, a visual servoing scheme is employed to adjust the PTZ-parameters in order for the border of a detected UAV to span as large as possible the cameras Field of View. The advocated servoing scheme is robust enough against the UAVs rapid maneuvers. Experimental studies are offered to highlight the efficiency of the suggested scheme.