Benchmarking Multi-Agent Deep Reinforcement Learning Algorithms in\n Cooperative Tasks

Abstract

Multi-agent deep reinforcement learning (MARL) suffers from a lack of\ncommonly-used evaluation tasks and criteria, making comparisons between\napproaches difficult. In this work, we provide a systematic evaluation and\ncomparison of three different classes of MARL algorithms (independent learning,\ncentralised multi-agent policy gradient, value decomposition) in a diverse\nrange of cooperative multi-agent learning tasks. Our experiments serve as a\nreference for the expected performance of algorithms across different learning\ntasks, and we provide insights regarding the effectiveness of different\nlearning approaches. We open-source EPyMARL, which extends the PyMARL codebase\nto include additional algorithms and allow for flexible configuration of\nalgorithm implementation details such as parameter sharing. Finally, we\nopen-source two environments for multi-agent research which focus on\ncoordination under sparse rewards.\n