Eye gaze patterns in conversations

TLDR

In multi‑agent, multi‑user settings, participants need a way to determine who is speaking to whom. The study evaluates whether users’ gaze direction can identify conversational partners. Eye‑tracking data were collected during four‑person conversations, and the results were incorporated into FRED, a system that uses gaze to infer the target agent. Gaze direction predicts conversational attention with 88 % accuracy for listeners and 77 % for speakers, making it a reliable cue for conversational systems.

Abstract

In multi-agent, multi-user environments, users as well as agents should have a means of establishing who is talking to whom. In this paper, we present an experiment aimed at evaluating whether gaze directional cues of users could be used for this purpose. Using an eye tracker, we measured subject gaze at the faces of conversational partners during four-person conversations. Results indicate that when someone is listening or speaking to individuals, there is indeed a high probability that the person looked at is the person listened (p=88%) or spoken to (p=77%). We conclude that gaze is an excellent predictor of conversational attention in multiparty conversations. As such, it may form a reliable source of input for conversational systems that need to establish whom the user is speaking or listening to. We implemented our findings in FRED, a multi-agent conversational system that uses eye input to gauge which agent the user is listening or speaking to.