IEEE Transactions on Neural Networks and Learning Systems · 2015 · 72 citations · 21 references
Natural SciencesMathematicsDynamical SystemsGame TheoryReal-time SystemsAdaptive ControlControl TheoryNash EquilibriumNonlinear DynamicsOnline AlgorithmsZero-sum GamesAdaptive Control Of Uncertain SystemsContinuous Zero-sum GamesContinuous-time Dynamical SystemsHamilton - Jacobi - Isaacs EquationsNonlinear Continuous SystemsNonlinear SystemsTwo-player GamesZero-sum Differential Games
Regarding two-player zero-sum games of continuous-time nonlinear systems with completely unknown dynamics, this paper presents an online adaptive algorithm for learning the Nash equilibrium solution, i.e., the optimal policy pair. First, for known systems, the simultaneous policy updating algorithm (SPUA) is reviewed. A new analytical method to prove the convergence is presented. Then, based on the SPUA, without using a priori knowledge of any system dynamics, an online algorithm is proposed to simultaneously learn in real time either the minimal nonnegative solution of the Hamilton-Jacobi-Isaacs (HJI) equation or the generalized algebraic Riccati equation for linear systems as a special case, along with the optimal policy pair. The approximate solution to the HJI equation and the admissible policy pair is reexpressed by the approximation theorem. The unknown constants or weights of each are identified simultaneously by resorting to the recursive least square method. The convergence of the online algorithm to the optimal solutions is provided. A practical online algorithm is also developed. Simulation results illustrate the effectiveness of the proposed method.
21