Operations Research · 1973 · 227 citations · 10 references
BusinessDecision SciencesMathematicsDecision TheoryMarkov Decision ProcessesStochastic ModelingBayesian Decision TheoryDecision Under UncertaintyDecision-making Under UncertaintyMarkov ProcessesPolicy IterationStochastic ControlDiscounted Markov Decision ProcessesLower BoundsOptimal PoliciesState Transition ProbabilitiesStationary Discounted PoliciesTransition Probabilities
The study focuses on finite‑state, discrete‑time, stationary Markov decision processes with time‑discounted rewards. The authors examine Markov decision processes with uncertain transition probabilities and compare game‑theoretic and Bayesian formulations. They develop policy‑iteration variants for the max‑min and max‑max cases in the game‑theoretic model and an implicit enumeration algorithm that bounds expected discounted return in the Bayesian model. They show that asymptotically Bayes‑optimal policies can be achieved.
This paper examines Markovian decision processes in which the transition probabilities corresponding to alternative decisions are not known with certainty. The processes are assumed to be finite-state, discrete-time, and stationary. The rewards axe time discounted. Both a game-theoretic and the Bayesian formulation are considered. In the game-theoretic formulation, variants of a policy-iteration algorithm are provided for both the max-min and the max-max cases. An implicit enumeration algorithm is discussed for the Bayesian formulation where upper and lower bounds on the total expected discounted return are provided by the max-max and max-min optimal policies. Finally, the paper discusses asymptotically Bayes-optimal policies.
10
Lloyd S. Shapley · Proceedings of the National Academy of Sciences · 1953 · 2.4K citations · Full text
Applied Statistical Decision Theory
D. V. Lindley, Howard Raiffa, Robert Schlaifer · Econometrica · 1962 · 2.2K citations
Branch-and-Bound Methods: A Survey
Eugene L. Lawler, Derek Wood · Operations Research · 1966 · 2K citations