Averaging Methods for the Asymptotic Analysis of Learning and Adaptive Systems, with Small Adjustment Rate

Abstract

Recently proven theorems concerning weak convergence of nonMarkovian processes to diffusions, together with an averaging and a stability method, are applied to two (learning or adaptive) processes of current interest: (1) an automata model for route selection in telephone traffic routing; (2) an adaptive quantizer for use in the transmission of random signals in communication theory. The models are chosen because they are prototypes of a large class to which the methods can be applied. The technique of application of the basic theorems to such processes is developed. Suitably interpolated and normalized “learning or adaptive” processes converge weakly to a diffusion, as the “learning or adaptation” rate goes to zero. For small learning rates, the qualitative properties (e.g., asymptotic (large-time) variances and parametric dependence) of the processes can be determined from the properties of the limit.