Non-Discounted Denumerable Markovian Decision Models

Abstract

Abstract : Countable state, finite action Markovian decision processes are studied under the average cost criterion. The problem is studied by using the known results for the discounted-cost problem. Sufficient conditions are given for the existence of an optimal rule which is of the stationary deterministic type. This rule is shown to be, in some sense, a limit point of the optimal discounted-cost rules. Sufficient conditions are also given for the optimal discounted-cost rules to be epsilon-optimal with respect to the average cost criterion. It is shown that if there is a replacement action then there exists an optimal rule but it may not be of the stationary deterministic type. It is also shown how, in a special case, the average cost criterion can be reduced to the discounted cost criterion. Lastly, an example is given of a process for which there exists an optimal nonstationary rule which is better than any stationary rule.