Maneuver Decision of UAV in Short-Range Air Combat Based on Deep Reinforcement Learning

TLDR

UAVs are increasingly deployed in air combat, yet autonomous maneuver decision remains a bottleneck, especially because high‑dimensional state and action spaces make traditional DQN training computationally demanding. The study proposes a reinforcement‑learning–based autonomous maneuver decision model for UAV short‑range air combat. The model integrates an aircraft motion model, a one‑to‑one combat evaluation module, and a DQN decision policy, and employs a phased “basic‑confrontation” training strategy to reduce training time, followed by simulations of one‑to‑one combats under varied target maneuvers. Simulations demonstrate that the proposed model and training method enable UAVs to autonomously decide maneuvers and effectively defeat opponents.

Abstract

With the development of artificial intelligence and integrated sensor technologies, unmanned aerial vehicles (UAVs) are more and more applied in the air combats. A bottleneck that constrains the capability of UAVs against manned vehicles is the autonomous maneuver decision, which is a very challenging problem in the short-range air combat undergoing highly dynamic and uncertain maneuvers of enemies. In this paper, an autonomous maneuver decision model is proposed for the UAV short-range air combat based on reinforcement learning, which mainly includes the aircraft motion model, one-to-one short-range air combat evaluation model and the maneuver decision model based on deep Q network (DQN). However, such model includes a high dimensional state and action space which requires huge computation load for DQN training using traditional methods. Then, a phased training method, called "basic-confrontation", which is based on the idea that human beings gradually learn from simple to complex is proposed to help reduce the training time while getting suboptimal but efficient results. Finally, one-to-one short-range air combats are simulated under different target maneuver policies. Simulation results show that the proposed maneuver decision model and training method can help the UAV achieve autonomous decision in the air combats and obtain an effective decision policy to defeat the opponent.