Nonlinear Shape and Appearance Models for Facial Expression Analysis and Synthesis

Abstract

Facial expression passes through nonlinear shape and appearance deformations with variations in different people and expressions. We present nonlinear shape and appearance models for facial expression analysis and synthesis using nonlinear generative models for different facial expressions in different people. To achieve accurate shape normalized appearance models, we utilize nonlinear warping using thin plate spline (TPS). A novel nonlinear generative model using conceptual manifold embedding and empirical kernel maps for facial expressions provides facial shape and appearance samples according to the configuration, personal style, and expression parameters. We can recognize facial expressions based on estimated facial expression parameters after iterative estimations of facial expression and style. In addition, the model provides accurate synthesis of facial expression sequences even with high nonlinear deformations of shape and appearance during facial expressions.