Occupation measures for controlled Markov processes: characterization and optimality

Abhay G. Bhatt, Vivek S. Borkar

The Annals of Probability · 1996 · 132 citations · 14 references

DOIFull text

Open access

Concepts

Abstract

For controlled Markov processes taking values in a Polish space, control problems with ergodic cost, infinite-horizon discounted cost and finite-horizon cost are studied. Each is posed as a convex optimization problem wherein one tries to minimize a linear functional on a closed convex set of appropriately defined occupation measures for the problem. These are characterized as solutions of a linear equation asssociated with the problem. This characterization is used to establish the existence of optimal Markov controls. The dual convex optimization problem is also studied.

References

14