Stability of Markovian processes II: continuous-time processes and sampled chains

TLDR

The paper extends Meyn and Tweedie’s discrete‑time stability results to continuous‑time Markov processes on topological spaces, aiming to link stability concepts to the space’s topology and probabilistic recurrence. The authors analyze continuous‑time Markov processes by studying stability concepts relative to the topology of the state space and proving connections to standard probabilistic recurrence notions. They demonstrate that the stability framework extends to continuous‑time Markov processes with continuous components, proving that petite sets serve as stability tests, establishing new ergodic theorems for processes with irreducible or countably reducible skeleton chains, and showing that failure of these conditions leads to decomposition into an uncountable orbit of skeleton chains.

Abstract

In this paper we extend the results of Meyn and Tweedie (1992b) from discrete-time parameter to continuous-parameter Markovian processes Φ evolving on a topological space. We consider a number of stability concepts for such processes in terms of the topology of the space, and prove connections between these and standard probabilistic recurrence concepts. We show that these structural results hold for a major class of processes (processes with continuous components) in a manner analogous to discrete-time results, and that complex operations research models such as storage models with state-dependent release rules, or diffusion models such as those with hypoelliptic generators, have this property. Also analogous to discrete time, ‘petite sets', which are known to provide test sets for stability, are here also shown to provide conditions for continuous components to exist. New ergodic theorems for processes with irreducible and countably reducible skeleton chains are derived, and we show that when these conditions do not hold, then the process may be decomposed into an uncountable orbit of skeleton chains.