Dyna, an integrated architecture for learning, planning, and reacting

TLDR

Dyna is an AI architecture that integrates learning, planning, and reactive execution, relying on machine learning methods for learning from examples while remaining agnostic to any specific technique. The paper introduces Dyna and evaluates its strengths and weaknesses relative to other architectures. Dyna employs learning to compile planning results and update action‑effect models, uses incremental planning that can operate on probabilistic and sometimes incorrect world models, and executes actions reactively without intervening planning.

Abstract

Dyna is an AI architecture that integrates learning, planning, and reactive execution. Learning methods are used in Dyna both for compiling planning results and for updating a model of the effects of the agent's actions on the world. Planning is incremental and can use the probabilistic and ofttimes incorrect world models generated by learning processes. Execution is fully reactive in the sense that no planning intervenes between perception and action. Dyna relies on machine learning methods for learning from examples---these are among the basic building blocks making up the architecture---yet is not tied to any particular method. This paper briefly introduces Dyna and discusses its strengths and weaknesses with respect to other architectures.