Stochastic Approximations with Constant Step Size and Differential Inclusions

Abstract

We consider stochastic approximation processes with constant step size whose associated deterministic system is an upper semicontinuous differential inclusion. We prove that over any finite time span, the sample paths of the stochastic process are closely approximated by a solution of the differential inclusion with high probability. We then analyze infinite horizon behavior, showing that if the process is Markov, its stationary measures must become concentrated on the Birkhoff center of the deterministic system. Our results extend those of Benaïm for settings in which the deterministic system is Lipschitz continuous and build on the work of Benaïm, Hofbauer, and Sorin for the case of decreasing step sizes. We apply our results to models of population dynamics in games, obtaining new conclusions about the medium and long run behavior of myopic optimizing agents.