Numerical Methods for Stochastic Singular Control Problems

Abstract

The paper develops a powerful class of numerical methods for stochastic singular control problems. The basic models used are diffusion or reflected diffusions, but the method is of general applicability. The central idea is that of the Markov chain approximation method, where an approximation to the control problem is found for which an optimal solution is computable, and which is an arbitrarily good approximation to the original problem and its optimal value function. The methods are convenient to program and use (and they have been used with success), and they cover a wide variety of problems. In fact, for the singular problem, they seem to be the only ones currently available. Owing to problems in proving tightness of certain processes that occur in the convergence proofs, the methods of proof used for the nonsingular problems need modifications. Examples of useful approximations, the algorithms, and the convergence proofs are given. To illustrate the power of the methods, two classes of problems are dealt with: the first is a class of discounted problems, and the second is an average-cost-per-unit time problem subject to some constraints, which arises in the study of multicustomer class queueing networks under conditions of heavy traffic. The method is applicable to the more standard singular control and ergodic problems with greater ease.