Optimal Control Applications and Methods · 2022 · 28 citations · 54 references
Natural SciencesComputer ScienceMathematicsArtificial IntelligenceDynamical SystemsGame TheoryControl Of Dynamical SystemsControl TheoryNeural NetworksOptimal ControlPolicy IterationReinforcement LearningApproximate Nash EquilibriumGradient DescentHamilton-jacobi EquationsHamilton-jacobi TheoryNash Equilibrium ExistenceNash Equilibrium ProblemsNonlinear Optimal ControlOff-policy Reinforcement Learning
Abstract This article studies the optimal tracking control problem of a class of multi‐input nonlinear system with unknown dynamics based on reinforcement learning (RL) and nonzero‐sum game theory. First of all, an augmented system composed of the tracking error dynamics and the command generator dynamics is constructed. Then, a tracking coupled Hamilton–Jacobi (HJ) equations associated with discounted cost function is derived, which gives the Nash equilibrium solution. The existence of Nash equilibrium is proved. To approximate the Nash equilibrium solution of tracking coupled HJ equations, we give two model‐based policy iteration (PI) algorithms, and analyze their equivalence and convergence. Further, to get rid of the prior knowledge of system dynamics, an off‐policy integral reinforcement learning (OP‐IRL) algorithm implemented by neural networks (NNs) is proposed. The weights of critic NNs and actor NNs are updated simultaneously by the gradient descent method. The convergence of the NNs weights and the stability of the closed‐loop error systems are proved. Finally, numerical simulation results are provided to demonstrate the effectiveness of the proposed OP‐IRL method.
54
Reinforcement Learning: An Introduction
Jeffrey D. Johnson, Jinghong Li, Zengshi Chen · Neurocomputing · 2000 · 8.7K citations
Dynamic noncooperative game theory
Harri Ehtamo · Journal of Economic Dynamics and Control · 1997 · 927 citations
Nonzero-sum differential games
A. W. Starr, Y. C. Ho · Journal of Optimization Theory and Applications · 1969 · 760 citations