Social roles in hierarchical models for human activity recognition

TLDR

The study introduces a hierarchical model for recognizing human activity in multi‑person scenes. The model represents actions from low‑level to high‑level events, incorporates social role expectations and inter‑person interactions, and is trained using a discriminative max‑margin framework. Experiments show the model improves performance across all levels of detail on two challenging datasets.

Abstract

We present a hierarchical model for human activity recognition in entire multi-person scenes. Our model describes human behaviour at multiple levels of detail, ranging from low-level actions through to high-level events. We also include a model of social roles, the expected behaviours of certain people, or groups of people, in a scene. The hierarchical model includes these varied representations, and various forms of interactions between people present in a scene. The model is trained in a discriminative max-margin framework. Experimental results demonstrate that this model can improve performance at all considered levels of detail, on two challenging datasets.