Stability of Markovian processes III: Foster–Lyapunov criteria for continuous-time processes

TLDR

The authors previously established Foster–Lyapunov criteria for stability in discrete chains (Part I) and extended these concepts to continuous‑time processes (Part II). This paper develops Foster–Lyapunov criteria for stability of continuous‑parameter Markovian processes on general state spaces and strengthens exponential ergodic theorems for such processes. The criteria are derived via Foster–Lyapunov inequalities for the extended generator, proved using Dynkin’s formula, and applied to linear stochastic systems, work‑modulated queues, release‑storage and risk processes. The test‑function criteria establish non‑explosivity, non‑evanescence, Harris recurrence, positive Harris recurrence, f‑norm convergence, and provide bounds for exponential ergodicity.

Abstract

In Part I we developed stability concepts for discrete chains, together with Foster–Lyapunov criteria for them to hold. Part II was devoted to developing related stability concepts for continuous-time processes. In this paper we develop criteria for these forms of stability for continuous-parameter Markovian processes on general state spaces, based on Foster-Lyapunov inequalities for the extended generator. Such test function criteria are found for non-explosivity, non-evanescence, Harris recurrence, and positive Harris recurrence. These results are proved by systematic application of Dynkin's formula. We also strengthen known ergodic theorems, and especially exponential ergodic results, for continuous-time processes. In particular we are able to show that the test function approach provides a criterion for f -norm convergence, and bounding constants for such convergence in the exponential ergodic case. We apply the criteria to several specific processes, including linear stochastic systems under non-linear feedback, work-modulated queues, general release storage processes and risk processes.