Recognizing abruptly changing facial expressions from time-sequential face images

Abstract

This paper proposes a method that can spot and recognize each facial expression. From time-sequential images that contain multiple facial expressions that could abruptly change from one expression to another expression. Previously, the authors have proposed an HMM (Hidden Markov Models) based method for recognizing a spotted facial expression. In this paper to HMM, we add states corresponding to the simultaneous motion of two different facial expressions: i.e. a muscle relaxation for one expression and a muscle contraction for another expression. Then, the added states are each lined from the HMM apex state of one expression and are linked to that of another expression. Experimental results showed that for most pairs of expressions the change in expression can be recognized accurately. In addition, recognition rate for very fast change of expressions improved significantly. The proposed method was applied to regenerate facial expressions on a synthesized character to show the method's effectiveness in obtaining facial motion information.