Animated conversation

TLDR

The paper presents a system that automatically generates and animates conversations among multiple human‑like agents with synchronized speech, intonation, facial expressions, and hand gestures. The system uses a dialogue planner to produce text and intonation, which together with speaker/listener roles drive facial expressions, lip movements, eye gaze, head motion, and coordinated arm, wrist, and hand gestures to create semantically meaningful animations. The authors illustrate the system’s capabilities with examples of fully synthesized, animated conversations.

Abstract

We describe an implemented system which automatically generates and animates conversations between multiple human-like agents with appropriate and synchronized speech, intonation, facial expressions, and hand gestures. Conversation is created by a dialogue planner that produces the text as well as the intonation of the utterances. The speaker/listener relationship, the text, and the intonation in turn drive facial expressions, lip motions, eye gaze, head motion, and arm gestures generators. Coordinated arm, wrist, and hand motions are invoked to create semantically meaningful gestures. Throughout we will use examples from an actual synthesized, fully animated conversation.