Emotion recognition in human-computer interaction

TLDR

Human interaction comprises an explicit channel for messages and an implicit channel that conveys speakers’ emotions, the latter of which is less understood. The article investigates how to develop signal‑processing and analytical methods for emotion recognition, aiming to create a hybrid face‑and‑voice system under the PKYSTA project. The authors propose combining signal‑processing techniques with psychological and linguistic analyses to build a hybrid system that fuses facial and vocal cues for emotion recognition.

Abstract

Two channels have been distinguished in human interaction: one transmits explicit messages, which may be about anything or nothing; the other transmits implicit messages about the speakers themselves. Both linguistics and technology have invested enormous efforts in understanding the first, explicit channel, but the second is not as well understood. Understanding the other party's emotions is one of the key tasks associated with the second, implicit channel. To tackle that task, signal processing and analysis techniques have to be developed, while, at the same time, consolidating psychological and linguistic analyses of emotion. This article examines basic issues in those areas. It is motivated by the PKYSTA project, in which we aim to develop a hybrid system capable of using information from faces and voices to recognize people's emotions.