arXiv (Cornell University) · 2018 · 95 citations · 0 references
Open access
ComputingComputer ScienceComputer VisionMachine LearningVisual AttentionAction RecognitionAttention NetworksDeep LearningObject DetectionObject RecognitionScene UnderstandingHuman Interactions
Recent progress in object instance detection has highlighted the need for computers to understand human–object interactions to fully interpret scene context. This work addresses the challenging problem of detecting human–object interactions (HOI) in images. We introduce an instance‑centric attention module that dynamically highlights image regions based on each instance’s appearance, enabling selective feature aggregation for HOI recognition. Experiments on Verb in COCO and HICO‑DET demonstrate that the proposed network outperforms state‑of‑the‑art methods.
Recent years have witnessed rapid progress in detecting and recognizing individual object instances. To understand the situation in a scene, however, computers need to recognize how humans interact with surrounding objects. In this paper, we tackle the challenging task of detecting human-object interactions (HOI). Our core idea is that the appearance of a person or an object instance contains informative cues on which relevant parts of an image to attend to for facilitating interaction prediction. To exploit these cues, we propose an instance-centric attention module that learns to dynamically highlight regions in an image conditioned on the appearance of each instance. Such an attention-based network allows us to selectively aggregate features relevant for recognizing HOIs. We validate the efficacy of the proposed network on the Verb in COCO and HICO-DET datasets and show that our approach compares favorably with the state-of-the-arts.