Optimal and Autonomous Control Using Reinforcement Learning: A Survey

TLDR

Reinforcement learning methods learn optimal control and game solutions online from measured system trajectories. This survey reviews RL‑based feedback control for optimal regulation and tracking of single and multi‑agent systems and explores off‑policy RL for continuous‑ and discrete‑time settings. The authors examine existing RL approaches, including Q‑learning and integral RL algorithms for discrete‑time and continuous‑time systems, and discuss their application across several domains.

Abstract

This paper reviews the current state of the art on reinforcement learning (RL)-based feedback control solutions to optimal regulation and tracking of single and multiagent systems. Existing RL solutions to both optimal and control problems, as well as graphical games, will be reviewed. RL methods learn the solution to optimal control and game problems online and using measured data along the system trajectories. We discuss Q-learning and the integral RL algorithm as core algorithms for discrete-time (DT) and continuous-time (CT) systems, respectively. Moreover, we discuss a new direction of off-policy RL for both CT and DT systems. Finally, we review several applications.