Publication | Open Access
The power of a nod and a glance: Envelope vs. emotional feedback in animated conversational agents
383
Citations
0
References
1999
Year
CommunicationSocial SciencesEmbodied AgentEmotional FeedbackAnimated Conversational AgentsAffective ComputingConversation AnalysisVerbal InteractionEmotional ExpressionCognitive ScienceHuman Agent InteractionMultimodal BehaviorAutonomous AgentsSpeech CommunicationEnvelope FeedbackInterpersonal CommunicationHuman InteractionHuman-computer InteractionParalinguisticsArtsEmotionNonverbal Communication
The study hypothesizes that envelope feedback is more critical than emotional feedback for embodied conversational agents. The experiment involved subjects interacting with three multimodal agents differing in feedback type—content-only, content plus envelope, and content plus emotional—while evaluations were gathered through questionnaires and behavioral coding. Results confirm that envelope feedback is more important than emotional feedback and that it enhances dialog flow, with users rating the agents highly for lifelikeness and interaction fluidity.
In this article we describe results froman experiment of user interaction with autonomous , human - like ( humanoid ) conversational agents . We hypothesize that for embodied conversational agents , nonverbal behaviors related to the process of conversation , what we call envelope feedback, is much more important than other feedback , such as emotional expression . We test this hypothesis by having subjects interact with three autonomous agents , all capable of full - duplex multimodal interaction: able to generate and recognize speech , intonation , facial displays , and gesture . Each agent , however , gave a different kind of feedback: ( 1 ) content - related only , ( 2 ) content + envelope feedback , and ( 3 ) content + emotional . Content-related feedback includes answering questions and executing commands; envelope feedback includes behaviors such as gaze , manual beat gesture , and head movements; emotional feedback includes smiles and looks of puzzlement . Subjects' evaluations of the systemwere collected with a questionnaire , and videotapes of their speech patterns and behaviors were scored according to how often the users repeated themselves , how often they hesitated , and how often they got frustrated . The results confirmour hypothesis that envelope feedback is more important in interaction than emotional feedback and that envelope feedback plays a crucial role in supporting the process of dialog . A secondary result fromthis study shows that users give our multimodal conversational humanoids very high ratings of lifelikeness and fluidity of interaction when the agents are capable of giving such feedback .