Parametric hidden Markov models for gesture recognition

TLDR

Parameterized gestures involve systematic spatial variation, such as a point gesture defined by two‑dimensional direction. The paper proposes a method for representing, recognizing, and interpreting parameterized gestures. The authors extend the standard hidden Markov model by adding a global parametric variation in state output probabilities, training with an EM algorithm for linear dependencies and a generalized EM for nonlinear ones, and testing by simultaneously maximizing likelihood and estimating parameters. Results show that the parametric HMM outperforms standard HMMs in recognition accuracy, is more robust to noisy input, and that the nonlinear extension enables natural spherical coordinate parameterization for pointing gestures.

Abstract

A method for the representation, recognition, and interpretation of parameterized gesture is presented. By parameterized gesture we mean gestures that exhibit a systematic spatial variation; one example is a point gesture where the relevant parameter is the two-dimensional direction. Our approach is to extend the standard hidden Markov model method of gesture recognition by including a global parametric variation in the output probabilities of the HMM states. Using a linear model of dependence, we formulate an expectation-maximization (EM) method for training the parametric HMM. During testing, a similar EM algorithm simultaneously maximizes the output likelihood of the PHMM for the given sequence and estimates the quantifying parameters. Using visually derived and directly measured three-dimensional hand position measurements as input, we present results that demonstrate the recognition superiority of the PHMM over standard HMM techniques, as well as greater robustness in parameter estimation with respect to noise in the input features. Finally, we extend the PHMM to handle arbitrary smooth (nonlinear) dependencies. The nonlinear formulation requires the use of a generalized expectation-maximization (GEM) algorithm for both training and the simultaneous recognition of the gesture and estimation of the value of the parameter. We present results on a pointing gesture, where the nonlinear approach permits the natural spherical coordinate parameterization of pointing direction.