Stochastic Automata Models with Applications to Learning Systems

Abstract

The performance of variable-structure stochastic automata in stationary random environments has been extensively studied for the case when the environment's response is 0 or 1 (P model). A method is suggested for extending the updating schemes known for the P model to the S model, where the environment's output can lie in the interval [0,1], and a class of optimal nonlinear schemes for the S model is derived. Computer simulations reveal the superior performance of the S model in multimodal search even when the bounds on the performance function are unknown.