Coding, analysis, interpretation, and recognition of facial expressions

TLDR

Previous facial expression analysis relied on the FACS, a system designed for human psychologists to code expressions from static images. The study aims to develop a computer vision system that probabilistically characterizes facial motion and muscle activation, creating a more accurate representation of expressions called FACS+. The system employs optimal estimation optical flow combined with geometric, physical, and motion-based dynamic models to observe facial motion and probabilistically characterize muscle activation. The method yields a reliable parametric representation of independent muscle action groups, accurately estimates facial motion, and introduces FACS+, enabling coding, analysis, interpretation, and recognition of expressions.

Abstract

We describe a computer vision system for observing facial motion by using an optimal estimation optical flow method coupled with geometric, physical and motion-based dynamic models describing the facial structure. Our method produces a reliable parametric representation of the face's independent muscle action groups, as well as an accurate estimate of facial motion. Previous efforts at analysis of facial expression have been based on the facial action coding system (FACS), a representation developed in order to allow human psychologists to code expression from static pictures. To avoid use of this heuristic coding scheme, we have used our computer vision system to probabilistically characterize facial motion and muscle activation in an experimental population, thus deriving a new, more accurate, representation of human facial expressions that we call FACS+. Finally, we show how this method can be used for coding, analysis, interpretation, and recognition of facial expressions.