Hidden Markov mixtures of experts for prediction of non-stationary dynamics

Abstract

The prediction of non-stationary dynamical systems may be performed by identifying appropriate sub-dynamics and an early detection of mode changes. We present a framework which unifies the mixtures of experts approach and a generalized hidden Markov model with an input-dependent transition matrix: the hidden Markov mixtures of experts (HMME). The gating procedure incorporates state memory, information about the current location in phase space, and the previous prediction performance. The experts and the hidden Markov gating model are simultaneously trained by an EM algorithm that maximizes the likelihood during an annealing procedure. The HMME architecture allows for a fast online detection of mode changes: change points are detected as soon as the incoming input data stream contains sufficient information to indicate a change in the dynamics.