Accelerated Reinforcement Learning via Dynamic Mode Decomposition

Abstract

This work applies the decomposition principle to discrete-time reinforcement learning (RL) to solve the optimal control problems for a network of subsystems. The control design is defined as a linear quadratic regulator graphical problem, where the performance function couples the subsystems' dynamics. We first present a model-free discrete-time RL algorithm based on online behaviors without using system dynamics. This could become a prohibitively long learning process for larger networks. To remedy this issue, we develop an efficient model-free RL algorithm based on dynamic mode decomposition. This decomposition method reduces the size of the measured data while the dynamic information of the original network is still retained. This algorithm is then implemented online. The proposed methodology is validated using examples of a consensus network and a power system network.