IEEE Access · 2022 · 28 citations · 26 references
EngineeringAerospace EngineeringComputer ScienceAerial RoboticsArtificial IntelligenceSwarm RoboticsExploration-exploitation TradeoffMission PlanningMulti-agent SystemsPolicy LearningReinforcement LearningSwarm CoordinationAgent CooperationAir Defense SuppressionHierarchical Reinforcement LearningMetropolis AlgorithmMulti-agent ExplorationMulti-agent Reinforcement LearningPositionSwarm Decision-making
Unmanned aerial vehicle (UAV) swarm cooperative decision-making has attracted increasing attentions because of its low-cost, reusable, and distributed characteristics. However, existing non-learning-based methods rely on small-scale, known scenarios, and cannot solve complex multi-agent cooperation problem in large-scale, uncertain scenarios. This paper proposes a hierarchical multi-agent reinforcement learning (HMARL) method to solve the heterogeneous UAV swarm cooperative decision-making problem for the typical suppression of enemy air defense (SEAD) mission, which is decoupled into two sub-problems, i.e., the higher-level target allocation (TA) sub-problem and the lower-level cooperative attacking (CA) sub-problem. We establish a HMARL agent model, consisting of a multi-agent deep Q network (MADQN) based TA agent and multiple independent asynchronous proximal policy optimization (IAPPO) based CA agents. MADQN-TA agent can dynamically adjust the TA schemes according to the relative position. To encourage exploration and promote learning efficiency, the Metropolis criterion and inter-agent information exchange techniques are introduced. IAPPO-CA agent adopts independent learning paradigm, which can easily scale with the number of agents. Comparative simulation results validate the effectiveness, robustness, and scalability of the proposed method.
26
Adam: A Method for Stochastic Optimization
Diederik P. Kingma, Jimmy Ba · UvA-DARE (University of Amsterdam) · 2014 · 84.8K citations · Full text
Human-level control through deep reinforcement learning
Volodymyr Mnih, Koray Kavukcuoglu, David Silver et al. · Nature · 2015 · 30K citations
Reinforcement Learning: An Introduction
Richard S. Sutton, Andy Barto · IEEE Transactions on Neural Networks · 1998 · 27.1K citations
Grandmaster level in StarCraft II using multi-agent reinforcement learning
Oriol Vinyals, I. Babuschkin, Wojciech Marian Czarnecki et al. · Nature · 2019 · 3.4K citations