Evolving dota 2 shadow fiend bots using genetic programming with external memory

Abstract

The capacity of genetic programming (GP) to evolve a 'hero' character in the Dota 2 video game is investigated. A reinforcement learning context is assumed in which the only input is a 320-dimensional state vector and performance is expressed in terms of kills and net worth. Minimal assumptions are made to initialize the GP game playing agents - evolution from a tabula rasa starting point - implying that: 1) the instruction set is not task specific; 2) end of game performance feedback reflects quantitive properties a player experiences; 3) no attempt is made to impart game specific knowledge into GP, such as heuristics for improving navigation, minimizing partial observability, improving team work or prioritizing the protection of specific strategically important structures. In short, GP has to actively develop its own strategies for all aspects of the game. We are able to demonstrate competitive play with the built in game opponents assuming 1-on-1 competitions using the 'Shadow Fiend' hero. The single most important contributing factor to this result is the provision of external memory to GP. Without this, the resulting Dota 2 bots are not able to identify strategies that match those of the built-in game bot.