Analysis and synthesis of facial expressions with hand-generated muscle actuation basis

Abstract

We present a performance-driven facial animation system for analyzing captured expressions to find muscle actuation and synthesizing expressions with the actuation values. A significantly different approach of our work is that we let artists sculpt the initial draft of the actuation basis: the basic facial shapes corresponding to the isolated actuation of individual muscles, instead of calculating skin surface deformation entirely, relying on mathematical models such as finite element methods. We synthesize expressions by linear combinations of the basis elements, and analyze expressions by finding the weights for the combinations. Even though the hand-generated actuation basis represents the essence of the subject's characteristic expressions, it is not accurate enough to be used in the subsequent computational procedures. We also describe an iterative algorithm to increase the accuracy of the actuation basis. The experimental results suggest that our artist-in-the-loop method produces a more predictable and controllable outcome than pure mathematical models, and thus can be a quite useful tool in animation productions.