Estimating a Markov Transition Matrix from Observational Data

Abstract

Markov chains are frequently used in Operational Research to describe how a system changes over time, its behaviour being governed by its transition matrix. This paper describes a technique for finding a maximum likelihood estimate for such a transition matrix when a system is observed at infrequent time intervals. The technique is called the EM Algorithm which, for this kind of problem, has distinct advantages over other methods of optimization.