IEEE Access · 2021 · 84 citations · 13 references
ComputingAerospace EngineeringComputer ScienceArtificial IntelligenceMachine LearningMotion PlanningAir WarfareDeep LearningManeuveringMission PlanningPolicy LearningReinforcement LearningBattlefield EnvironmentDeep Reinforcement LearningHuman-agent InteractionLong Short-term Memory NetworksManeuver PlanningReward Function DesignState Space FormulationTraining
Beyond-visual-range (BVR) engagement becomes more and more popular in the modern air battlefield. The key and difficulty for pilots in the fight is maneuver planning, which reflects the tactical decision-making capacity of the both sides and determinates success or failure. In this paper, we propose an intelligent maneuver planning method for BVR combat with using an improved deep Q network (DQN). First, a basic combat environment builds, which mainly includes flight motion model, relative motion model and missile attack model. Then, we create a maneuver decision framework for agent interaction with the environment. Basic perceptive variables are constructed for agents to form continuous state space. Also, considering the threat of each side missile and the constraint of airfield, the reward function is designed for agents to training. Later, we introduce a training algorithm and propose perceptional situation layers and value fitting layers to replace policy network in DQN. Based on long short-term memory (LSTM) cell, the perceptional situation layer can convert basic state to high-dimensional perception situation. The fitting layer does well in mapping action. Finally, three combat scenarios are designed for agent training and testing. Simulation result shows the agent can avoid the threat of enemy and gather own advantages to threat the target. It also proves the models and methods of agents are valid and intelligent air combat can be realized.
13
Mastering the game of Go with deep neural networks and tree search
David Silver, Aja Huang, Chris J. Maddison et al. · Nature · 2016 · 15.8K citations
Mastering the game of Go without human knowledge
David Silver, Julian Schrittwieser, Karen Simonyan et al. · Nature · 2017 · 9.2K citations
Engineering, Educational Psychology, Cognitive Psychology +7
Maneuver Decision of UAV in Short-Range Air Combat Based on Deep Reinforcement Learning
Qiming Yang, Jiandong Zhang, Guoqing Shi et al. · IEEE Access · 2019 · 180 citations · Full text