On the role of dynamic programming in statistical communication theory

Abstract

In this paper we wish to show that the fundamental problem of determining the utility of a communication channel in conveying information can be interpreted as a problem within the framework of multistage decision processes of stochastic type, and as such may be treated by means of the theory of dynamic programming. We shall begin by formulating some aspects of the general problem in terms of multistage decision processes, with brief descriptions of stochastic allocation processes and learning processes. Following this, as a simple example of the applicability of the techniques of dynamic programming, we shall discuss in detail a problem posed recently by Kelly. In this paper, it is shown by Kelly that under certain conditions, the rate of transmission, as defined by Shannon, can be obtained from a certain multistage decision process with an economic criterion. Here we shall complete Kelly's analysis in some essential points, using functional equation techniques, and considerably extend his results.