2023 · 19 citations · 45 references
ComputingAutonomous SystemsComputer ScienceComputer VisionMachine LearningRobot LearningDeep LearningEmbodied LearningEnvironment RepresentationRobot NavigationScene UnderstandingSelf-supervised LearningContrastive LearningGraph Contrastive LearningNavigationRepresentation Learning For RoboticsScene Graph AnalysisScene Graph RepresentationScene GraphsSelf-supervised Navigation
Training effective embodied AI agents often involves expert imitation, specialized components such as maps, or leveraging additional sensors for depth and localization. Another approach is to use neural architectures alongside self-supervised objectives which encourage better representation learning. However, in practice, there are few guarantees that these self-supervised objectives encode task-relevant information. We propose the Scene Graph Contrastive (SGC) loss, which uses scene graphs as training-only supervisory signals. The SGC loss does away with explicit graph decoding and instead uses contrastive learning to align an agent’s representation with a rich graphical encoding of its environment. The SGC loss is simple to implement and encourages representations that encode objects’ semantics, relationships, and history. By using the SGC loss, we attain gains on three embodied tasks: Object Navigation, Multi-Object Navigation, and Arm Point Navigation. Finally, we present studies and analyses which demonstrate the ability of our trained representation to encode semantic cues about the environment.
45
Guillem Cucurull, Arantxa Casanova, Adriana Romero et al. · arXiv (Cornell University) · 2017 · 8.3K citations · Full text
A Simple Framework for Contrastive Learning of Visual Representations
Ting Chen, Simon Kornblith, Mohammad Norouzi et al. · arXiv (Cornell University) · 2020 · 7.3K citations · Full text
Scene Graph Generation by Iterative Message Passing
Danfei Xu, Yuke Zhu, Christopher Choy et al. · 2017 · 1.2K citations
Scene Graph Generation from Objects, Phrases and Region Captions
Yikang Li, Wanli Ouyang, Bolei Zhou et al. · 2017 · 542 citations