2023 · 51 citations · 72 references
EngineeringAutonomous SystemsComputer ScienceComputer VisionRobot PerceptionRobot VisionEnvironment RepresentationMulti-view GeometryRobotic VisionScene ReconstructionScene UnderstandingVisual Navigation3D Navigation3D Scene UnderstandingBird's-eye PerspectiveScene Graph AnalysisScene Graph GenerationScene Graph ModelingScene Graph RepresentationScene Graphs
Vision-language navigation (VLN), which entails an agent to navigate 3D environments following human instructions, has shown great advances. However, current agents are built upon panoramic observations, which hinders their ability to perceive 3D scene geometry and easily leads to ambiguous selection of panoramic view. To address these limitations, we present a BEV Scene Graph (BSG), which leverages multi-step BEV representations to encode scene layouts and geometric cues of indoor environment under the supervision of 3D detection. During navigation, BSG builds a local BEV representation at each step and maintains a BEV-based global scene map, which stores and organizes all the online collected local BEV representations according to their topological relations. Based on BSG, the agent predicts a local BEV grid-level decision score and a global graph-level decision score, combined with a subview selection score on panoramic views, for more accurate action prediction. Our approach significantly outperforms state-of-the-art methods on REVERIE, R2R, and R4R, showing the potential of BEV perception in VLN.
72
Focal Loss for Dense Object Detection
Tsung-Yi Lin, Priya Goyal, Ross Girshick et al. · 2017 · 25.7K citations
Are we ready for autonomous driving? The KITTI vision benchmark suite
Andreas Geiger, P Lenz, R. Urtasun · 2012 · 14.4K citations
PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
Raffaelli Charles, Hao Su, Kaichun Mo et al. · 2017 · 9.8K citations
Sebastian Thrun · Communications of the ACM · 2002 · 8K citations
Dynamic Graph CNN for Learning on Point Clouds
Yue Wang, Yongbin Sun, Ziwei Liu et al. · ACM Transactions on Graphics · 2019 · 6.7K citations · Full text