2010 · 27 citations · 23 references
Natural SciencesMathematicsArtificial IntelligenceDynamical SystemsReal-time SystemsControl SystemsControl TheoryOptimal ControlPolicy IterationReal-time ControlRobust ControlContinuous-time Control SystemsContinuous-time Dynamical SystemsGradient DescentInfinite-horizon OptimizationModel-free AlgorithmsNeural Network ControlNonlinear Control SystemsOnline Policy Iteration
An online model-free solution is developed for the infinite-horizon optimal control problem for continuous-time nonlinear systems. A novel actor-critic-identifier (ACI) structure is used to implement the Policy Iteration algorithm, wherein two neural network structures are used - a robust dynamic neural network (DNN) to asymptotically identify the uncertain system with additive disturbances, and a critic NN to approximate the value function. The weight update laws for the critic NN are generated using a gradient-descent method based on a modified temporal difference error, which is independent of the system dynamics. The optimal control law (or the actor) is computed using the critic NN and the identifier DNN. Uniformly ultimately bounded (UUB) stability of the closed-loop system is guaranteed. The actor, critic and identifier structures are implemented in real-time, continuously and simultaneously.
23
Reinforcement Learning: An Introduction
Richard S. Sutton, Andy Barto · IEEE Transactions on Neural Networks · 1998 · 27.1K citations
Learning to Predict by the Methods of Temporal Differences
Richard S. Sutton · Machine Learning · 1988 · 4K citations · Full text
Learning to predict by the methods of temporal differences
Richard S. Sutton · Machine Learning · 1988 · 2.8K citations · Full text