Beyond Actions: Discriminative Models for Contextual Group Activities

Abstract

In this dissertation, we go beyond recognizing individual person actions and focus on group activities instead. This motivates from the observation that human actions are rarely performed in isolation -- the contextual information of nearby humans provides useful cues for understanding the high-level activities. We propose a discriminative model for recognizing group activities. Our model jointly captures the group activity, the individual person actions, and the interactions among them. Two new types of contextual information, group-person interaction and person-person interaction, are explored in a latent variable framework. In particular, the person-person interaction is modeled in two ways: 1) the structure of the interaction is modeled as a latent variable and implicitly inferred during learning and inference. 2) The interaction is explored in feature level and a new contextual feature descriptor is introduced. Our experimental results demonstrate the benefit of using contextual information for disambiguating group activities.