Grounding the Lexical Semantics of Verbs in Visual Perception using Force Dynamics and Event Logic

TLDR

The study implements a system that recognizes spatial‑motion verb events in short image sequences. It models verb semantics with event‑logic expressions over force‑dynamic relations, employs a compact interval representation for liquid events, and infers compound events from primitives. The system demonstrates greater robustness than prior motion‑profile‑based approaches.

Abstract

This paper presents an implemented system for recognizing the occurrence of events described by simple spatial-motion verbs in short image sequences. The semantics of these verbs is specified with event-logic expressions that describe changes in the state of force-dynamic relations between the participants of the event. An efficient finite representation is introduced for the infinite sets of intervals that occur when describing liquid and semi-liquid events. Additionally, an efficient procedure using this representation is presented for inferring occurrences of compound events, described with event-logic expressions, from occurrences of primitive events. Using force dynamics and event logic to specify the lexical semantics of events allows the system to be more robust than prior systems based on motion profile.