A Hybrid Bellman Equation for systems with regional dynamics

Abstract

In this paper, we study hybrid systems with regional dynamics, i.e., systems where transitions between different dynamical regimes occur as the continuous state of the system reaches given switching surfaces. In particular, we focus our attention on the optimal control problem associated with such systems, and we present a Hybrid Bellman Equation for such systems that provide a characterization of global optimality, given an upper bound on the number of switches. Not surprisingly, the solution will be hybrid in nature in that it will depend on not only the continuous control signals, but also on discrete decisions as to what domains the system should go through in the first place. A number of examples are presented to highlight the operation of the proposed approach.