Joint Power Allocation and 3D Deployment for UAV-BSs: A Game Theory Based Deep Reinforcement Learning Approach

Abstract

Ultra-dense unmanned aerial vehicle (UAV) plays an important role in the field of communications due to its flexibility and low-cost feature. Ultra-dense unnamed aerial vehicle base station (UAV-BS) can improve communication quality by providing temporary and cost-effective wireless communication services for hotspots. In this paper, a multiple UAV-BSs assisted downlink network is investigated to maximize the system throughput. It is still a challenging problem to jointly optimize the power allocation and the 3D deployment of multiple UAV-BSs. Therefore, in this paper, for effective interference management, the power allocation problem is first formulated as a non-cooperative game with a pricing mechanism to imitate the interactions among users served by UAV-BSs. Then, based on the combination of deep reinforcement learning (DRL) and the game theory, the power allocation and the 3D deployment of UAV-BSs are transformed into a Markov decision problem. Finally, a novel price-based proximal policy optimization (3PO) algorithm is proposed to explore the optimal policy to maximize the system throughput. Simulation results reveal that the proposed 3PO algorithm can significantly improve system throughput and energy efficiency compared to other baselines by jointly optimizing power allocation and 3D deployment for UAV-BSs.