Hidden Conditional Random Fields for Gesture Recognition

TLDR

Gesture sequences exhibit complex underlying structure, and while hidden‑state models such as HMMs have been used, they rely on conditional independence and generative likelihood maximization, which are suboptimal for discriminating gestures. The study introduces a discriminative hidden‑state model for recognizing human gestures and evaluates its effectiveness in both detection and multi‑way classification. The authors derive a discriminative sequence model with hidden states, test it on human arm and head gesture recognition, and compare its performance to generative hidden‑state and fully‑observable discriminative models.

Abstract

We introduce a discriminative hidden-state approach for the recognition of human gestures. Gesture sequences often have a complex underlying structure, and models that can incorporate hidden structures have proven to be advantageous for recognition tasks. Most existing approaches to gesture recognition with hidden states employ a Hidden Markov Model or suitable variant (e.g., a factored or coupled state model) to model gesture streams; a significant limitation of these models is the requirement of conditional independence of observations. In addition, hidden states in a generative model are selected to maximize the likelihood of generating all the examples of a given gesture class, which is not necessarily optimal for discriminating the gesture class against other gestures. Previous discriminative approaches to gesture sequence recognition have shown promising results, but have not incorporated hidden states nor addressed the problem of predicting the label of an entire sequence. In this paper, we derive a discriminative sequence model with a hidden state structure, and demonstrate its utility both in a detection and in a multi-way classification formulation. We evaluate our method on the task of recognizing human arm and head gestures, and compare the performance of our method to both generative hidden state and discriminative fully-observable models.