Proceedings of the AAAI Conference on Artificial Intelligence · 2019 · 54 citations · 12 references
ComputingCommunicationComputer ScienceArtificial IntelligenceMachine LearningMarkov Decision ProcessesDeep LearningMulti-agent SystemsPolicy LearningReinforcement LearningCentralized TrainingCentralized Training With Decentralized ExecutionDecentralized ExecutionDeep Reinforcement LearningLearning In Multi-agent SystemsMulti-agent LearningMulti-agent Reinforcement LearningTraining
In this paper, we propose a new learning technique named message-dropout to improve the performance for multi-agent deep reinforcement learning under two application scenarios: 1) classical multi-agent reinforcement learning with direct message communication among agents and 2) centralized training with decentralized execution. In the first application scenario of multi-agent systems in which direct message communication among agents is allowed, the messagedropout technique drops out the received messages from other agents in a block-wise manner with a certain probability in the training phase and compensates for this effect by multiplying the weights of the dropped-out block units with a correction probability. The applied message-dropout technique effectively handles the increased input dimension in multi-agent reinforcement learning with communication and makes learning robust against communication errors in the execution phase. In the second application scenario of centralized training with decentralized execution, we particularly consider the application of the proposed messagedropout to Multi-Agent Deep Deterministic Policy Gradient (MADDPG), which uses a centralized critic to train a decentralized actor for each agent. We evaluate the proposed message-dropout technique for several games, and numerical results show that the proposed message-dropout technique with proper dropout rate improves the reinforcement learning performance significantly in terms of the training speed and the steady-state performance in the execution phase.
12
Dropout: a simple way to prevent neural networks from overfitting
Nitish Srivastava, Geoffrey E. Hinton, Alex Krizhevsky et al. · 2014 · 34.3K citations
Deep Reinforcement Learning with Double Q-Learning
Hado van Hasselt, Arthur Guez, David Silver · 2016 · 6K citations · Full text
Learning to Communicate with Deep Multi-Agent Reinforcement Learning
Jakob Foerster, Yannis Assael, Nando de Freitas et al. · arXiv (Cornell University) · 2016 · 870 citations · Full text