Unity: A General Platform for Intelligent Agents

TLDR

Recent AI progress relies on realistic simulated environments, yet many current platforms suffer from unrealistic visuals, inaccurate physics, limited task complexity, restricted agent perspective, and inflexible configuration. The authors propose a taxonomy of simulation platforms and identify a high-level class of general platforms that support rich visual, physical, task, and social complexity. They argue that modern game engines, exemplified by Unity and its open-source ML‑Agents Toolkit, are uniquely suited as such general platforms. A survey of research using Unity and the ML‑Agents Toolkit demonstrates that a flexible, interactive, and easily configurable general platform enables diverse studies in AI and robotics.

Abstract

Recent advances in artificial intelligence have been driven by the presence of increasingly realistic and complex simulated environments. However, many of the existing environments provide either unrealistic visuals, inaccurate physics, low task complexity, restricted agent perspective, or a limited capacity for interaction among artificial agents. Furthermore, many platforms lack the ability to flexibly configure the simulation, making the simulated environment a black-box from the perspective of the learning system. In this work, we propose a novel taxonomy of existing simulation platforms and discuss the highest level class of general platforms which enable the development of learning environments that are rich in visual, physical, task, and social complexity. We argue that modern game engines are uniquely suited to act as general platforms and as a case study examine the Unity engine and open source Unity ML-Agents Toolkit. We then survey the research enabled by Unity and the Unity ML-Agents Toolkit, discussing the kinds of research a flexible, interactive and easily configurable general platform can facilitate.