Adaptive control with the stochastic approximation algorithm: Geometry and convergence

Abstract

New geometric properties possessed by the sequence of parameter estimates are exhibited, which yield valuable insight into the behavior of the stochastic approximation based algorithm as it is used in minimum variance adaptive control. In particular, these geometric properties, together with certain probabilistic arguments, prove that if the system does not have a reduced-order minimum variance controller, then the parameter estimates converge to a random multiple of the true parameter. An explicit expression for the limiting parameter estimate is also available. With strictly positive probability, the limiting parameter estimate is not the true parameter, and in some cases differs from the true parameter with probability one. If the system possesses reduced-order minimum variance controllers, then convergence to a minimum variance controller in a Cesaro sense is shown. The geometry of the limit set is described. Sufficient conditions are also given for some of these results to hold for parameter estimation schemes other than stochastic approximation.