Coupled hidden Markov models for complex action recognition

TLDR

HMMs are a widely used framework for modeling dynamic behaviors, but their single‑process, limited‑memory assumptions constrain performance in vision and speech tasks. The authors present algorithms for coupling and training hidden Markov models to model interacting processes, aiming to improve classification of two‑handed actions. Coupled HMMs outperform conventional HMMs in classifying two‑handed actions, offering faster training, higher likelihoods, and greater robustness to initial conditions.

Abstract

We present algorithms for coupling and training hidden Markov models (HMMs) to model interacting processes, and demonstrate their superiority to conventional HMMs in a vision task classifying two-handed actions. HMMs are perhaps the most successful framework in perceptual computing for modeling and classifying dynamic behaviors, popular because they offer dynamic time warping, a training algorithm and a clear Bayesian semantics. However the Markovian framework makes strong restrictive assumptions about the system generating the signal-that it is a single process having a small number of states and an extremely limited state memory. The single-process model is often inappropriate for vision (and speech) applications, resulting in low ceilings on model performance. Coupled HMMs provide an efficient way to resolve many of these problems, and offer superior training speeds, model likelihoods, and robustness to initial conditions.