The Annals of Probability · 1996 · 132 citations · 14 references
EngineeringDecision SciencesMarkov Decision ProcessesOptimizationStochastic ProcessesMarkov ChainsMarkov ProcessesOptimal ControlOptimization In EconomicsStochastic ControlErgodic MeasuresErgodic ProcessesErgodic PropertiesFinite Horizon ControlOccupancy Measure FormulationOptimal Control Of Stochastic SystemsOptimal Policy ExistenceReward-cost Tradeoff
For controlled Markov processes taking values in a Polish space, control problems with ergodic cost, infinite-horizon discounted cost and finite-horizon cost are studied. Each is posed as a convex optimization problem wherein one tries to minimize a linear functional on a closed convex set of appropriately defined occupation measures for the problem. These are characterized as solutions of a linear equation asssociated with the problem. This characterization is used to establish the existence of optimal Markov controls. The dual convex optimization problem is also studied.
14
Markov Processes: Characterization and Convergence.
Andris Abakuks, S. N. Ethier, Thomas G. Kurtz · Biometrics · 1987 · 4.5K citations
Linear Programming and Sequential Decisions
Alan S. Manne · Management Science · 1960 · 435 citations