Statistical Inference about Markov Chains

TLDR

Maximum likelihood estimates and their asymptotic distribution for transition probabilities in Markov chains of arbitrary order are derived, and the relationship between likelihood ratio criteria and chi‑square tests is discussed. The study also considers tests of several additional hypotheses. The authors develop likelihood ratio and chi‑square tests for hypotheses on constant transition probabilities, specified values, Markov chain order, independence versus first‑order dependence, and for the single‑observation long‑chain case.

Abstract

Maximum likelihood estimates and their asymptotic distribution are obtained for the transition probabilities in a Markov chain of arbitrary order when there are repeated observations of the chain. Likelihood ratio tests and $\chi^2$-tests of the form used in contingency tables are obtained for testing the following hypotheses: (a) that the transition probabilities of a first order chain are constant, (b) that in case the transition probabilities are constant, they are specified numbers, and (c) that the process is a $u$th order Markov chain against the alternative it is $r$th but not $u$th order. In case $u = 0$ and $r = 1$, case (c) results in tests of the null hypothesis that observations at successive time points are statistically independent against the alternate hypothesis that observations are from a first order Markov chain. Tests of several other hypotheses are also considered. The statistical analysis in the case of a single observation of a long chain is also discussed. There is some discussion of the relation between likelihood ratio criteria and $\chi^2$-tests of the form used in contingency tables.