Deep-Reinforcement-Learning-Based Optimal Transmission Policies for Opportunistic UAV-Aided Wireless Sensor Network

Abstract

When there are unmanned aerial vehicles (UAVs) performing their specifically assigned tasks in the air, some of them still have available resources to access different ground communication networks to improve their communication performance, especially for the wireless sensor network. Technically, when they execute their own given missions with predetermined trajectories, they can also provide opportunistic assistance for terrestrial networks at the same time. In this article, we solve an opportunistic UAV-assisted data transmission problem in a wireless sensor network from a novel perspective. In consideration of UAVs dynamic behaviors, varying transmission tasks, and real-time matching between UAVs and sensor clusters, we propose to jointly optimize UAV scheduling and power control aiming to obtain optimal policies to maximize the network data transmission in a long run under the opportunistic access mode. We reformulate this optimization problem as a Markov decision process (MDP) and take deep reinforcement learning (DRL) as our tool to obtain solutions. We develop a DQN-based and a deep deterministic policy gradient (DDPG)-based optimization approaches to adjust the power allocation of cluster heads, and the scheduling and bandwidth allocation of UAVs during their missions over the covered area to improve the whole network data transmission performance. Simulation results demonstrate the validity and superiority of our proposed approaches compared with other benchmark policies in different perspectives.