Human-level performance in 3D multiplayer games with population-based reinforcement learning

TLDR

Reinforcement learning has succeeded in complex single-agent and two-player games, but real-world scenarios involve many agents learning independently to cooperate and compete. The authors employed a two-tier optimization where a population of independent RL agents is trained concurrently from thousands of parallel matches on randomly generated environments. Using only pixels and game points, the agents reached human-level performance in Quake III Arena Capture the Flag, learning internal reward signals and rich world representations, underscoring the promise of multiagent RL.

Abstract

Reinforcement learning (RL) has shown great success in increasingly complex single-agent environments and two-player turn-based games. However, the real world contains multiple agents, each learning and acting independently to cooperate and compete with other agents. We used a tournament-style evaluation to demonstrate that an agent can achieve human-level performance in a three-dimensional multiplayer first-person video game, Quake III Arena in Capture the Flag mode, using only pixels and game points scored as input. We used a two-tier optimization process in which a population of independent RL agents are trained concurrently from thousands of parallel matches on randomly generated environments. Each agent learns its own internal reward signal and rich representation of the world. These results indicate the great potential of multiagent reinforcement learning for artificial intelligence research.