Adaptive Dynamic Programming: An Introduction

TLDR

ADP algorithms are typically classified into two classes: those requiring an initial stable policy and those that do not, with the latter offering lower computational cost at the expense of stability guarantees, and recent studies have focused on convergence analysis of these schemes. This article surveys recent research trends in adaptive/approximate dynamic programming, outlining structural variations, algorithmic developments, and applications, and identifies topics for future investigation. The authors review recent ADP research by describing variations in ADP scheme structures, the development of new algorithms, and their practical applications.

Abstract

In this article, we introduce some recent research trends within the field of adaptive/approximate dynamic programming (ADP), including the variations on the structure of ADP schemes, the development of ADP algorithms and applications of ADP schemes. For ADP algorithms, the point of focus is that iterative algorithms of ADP can be sorted into two classes: one class is the iterative algorithm with initial stable policy; the other is the one without the requirement of initial stable policy. It is generally believed that the latter one has less computation at the cost of missing the guarantee of system stability during iteration process. In addition, many recent papers have provided convergence analysis associated with the algorithms developed. Furthermore, we point out some topics for future studies.