Learning to Rest: A Q-Learning Approach to Flying Base Station Trajectory Design with Landing Spots

Abstract

We consider the problem of trajectory optimization for an autonomous UAV-mounted base station that provides communication services to ground users with the aid of landing spots (LSs). Recently, the concept of LSs was introduced to alleviate the problem of short mission durations arising from the limited on-board battery budget of the UAV, which severely limits network performance. In this work, using Q-learning, a model-free reinforcement learning (RL) technique, we train a neural network (NN) to make movement decisions for the UAV that maximize the data collected from the ground users while minimizing power consumption by exploiting the landing spots. We show that the system intelligently integrates landing spots into the trajectory to extend flying time and is able to learn the topology of the network over several flying epochs without any explicit information about the environment.