Action-dependent adaptive critic designs

Abstract

We study a class of action-dependent adaptive critic designs. Conventional adaptive critic designs contain three basic modules: critic, model and action. Each of the three modules can be implemented using a neural network. By combining the critic network and the model network to form a new critic network, we propose a form of action-dependent adaptive critic designs where the critic network implicitly includes a model network in it. An important feature of the present design is that the proposed action-dependent adaptive critic designs can be applied to online learning control applications. We also provide details about the training of the neural networks used in the present design. The training approach described makes it possible the use of many readily available neural network training algorithms and tools without modifications. We employ the pole balancing problem in our simulation study to show the applicability of the present results.