International Journal of Aerospace Engineering · 2021 · 54 citations · 17 references
ComputingAutonomous SystemsComputer ScienceArtificial IntelligenceMarkov Decision ProcessesUnmanned Aircraft SystemsHuman-swarm InteractionIntelligent AgentsMulti-agent SystemsPolicy LearningReinforcement LearningReward DesignAction Space FormulationDecision Making In Multi-agent SystemsMarkov Decision ModelsMulti-agent Decision MakingMulti-agent LearningMulti-agent Reinforcement LearningMultiagent ControlState-space Structure
With the development of unmanned aerial vehicle (UAV) technology, UAV swarm confrontation has attracted many researchers’ attention. However, the situation faced by the UAV swarm has substantial uncertainty and dynamic variability. The state space and action space increase exponentially with the number of UAVs, so that autonomous decision-making becomes a difficult problem in the confrontation environment. In this paper, a multiagent reinforcement learning method with macro action and human expertise is proposed for autonomous decision-making of UAVs. In the proposed approach, UAV swarm is modeled as a large multiagent system (MAS) with an individual UAV as an agent, and the sequential decision-making problem in swarm confrontation is modeled as a Markov decision process. Agents in the proposed method are trained based on the macro actions, where sparse and delayed rewards, large state space, and action space are effectively overcome. The key to the success of this method is the generation of the macro actions that allow the high-level policy to find a near-optimal solution. In this paper, we further leverage human expertise to design a set of good macro actions. Extensive empirical experiments in our constructed swarm confrontation environment show that our method performs better than the other algorithms.
17
Human-level control through deep reinforcement learning
Volodymyr Mnih, Koray Kavukcuoglu, David Silver et al. · Nature · 2015 · 30K citations
Mastering the game of Go without human knowledge
David Silver, Julian Schrittwieser, Karen Simonyan et al. · Nature · 2017 · 9.2K citations
Engineering, Educational Psychology, Cognitive Psychology +7
Continuous control with deep reinforcement learning
Timothy Lillicrap, Jonathan J. Hunt, Alexander Pritzel et al. · arXiv (Cornell University) · 2016 · 6.8K citations · Full text
Computing, Computer Science, Physics +17
Value-Decomposition Networks For Cooperative Multi-Agent Learning Based On Team Reward
Peter Sunehag, Guy Lever, Audrūnas Gruslys et al. · 2018 · 540 citations