An extensible framework for interactive facial animation with facial expressions, lip synchronization and eye behavior

Abstract

In this article we describe our approach to generating convincing and empathetic facial animation. Our goal is to develop a robust facial animation platform that is usable and can be easily extended. We also want to facilitate the integration of research in the area and to directly incorporate the characters in interactive applications such as embodied conversational agents and games. We have developed a framework capable of easily animating MPEG-4 parameterized faces through high-level description of facial actions and behaviors. The animations can be generated in real time for interactive applications. We present some case studies that integrate computer vision techniques in order to provide interaction between the user and a character that interacts with different facial actions according to events in the application.