Diverse Auto-Curriculum is Critical for Successful Real-World Multiagent\n Learning Systems

Abstract

Multiagent reinforcement learning (MARL) has achieved a remarkable amount of\nsuccess in solving various types of video games. A cornerstone of this success\nis the auto-curriculum framework, which shapes the learning process by\ncontinually creating new challenging tasks for agents to adapt to, thereby\nfacilitating the acquisition of new skills. In order to extend MARL methods to\nreal-world domains outside of video games, we envision in this blue sky paper\nthat maintaining a diversity-aware auto-curriculum is critical for successful\nMARL applications. Specifically, we argue that \\emph{behavioural diversity} is\na pivotal, yet under-explored, component for real-world multiagent learning\nsystems, and that significant work remains in understanding how to design a\ndiversity-aware auto-curriculum. We list four open challenges for\nauto-curriculum techniques, which we believe deserve more attention from this\ncommunity. Towards validating our vision, we recommend modelling realistic\ninteractive behaviours in autonomous driving as an important test bed, and\nrecommend the SMARTS/ULTRA benchmark.\n