Automatic analysis of multimodal group actions in meetings

TLDR

The study aims to recognize group actions during meetings. The authors model group actions with HMMs that fuse audiovisual features of individual participants to capture interaction dynamics. Experiments show that accounting for participant interactions improves group action modeling and that visual cues add value even for audio‑dominant events, supporting a multimodal analysis.

Abstract

This paper investigates the recognition of group actions in meetings. A framework is employed in which group actions result from the interactions of the individual participants. The group actions are modeled using different HMM-based approaches, where the observations are provided by a set of audiovisual features monitoring the actions of individuals. Experiments demonstrate the importance of taking interactions into account in modeling the group actions. It is also shown that the visual modality contains useful information, even for predominantly audio-based events, motivating a multimodal approach to meeting analysis.