arXiv (Cornell University) · 2020 · 21 citations · 34 references
EngineeringAutonomous SystemsComputer ScienceComputer VisionRobot LearningRoboticsAutonomous NavigationNavigation LearningRobot NavigationRobotic VisionSim-to-real TransferVisual NavigationAction Space FormulationDomain RandomizationNavigationRobot SimulationSimulated EnvironmentsTask Success RateTransfer Learning For RoboticsVision-based Navigation
We study the challenging problem of releasing a robot in a previously unseen environment, and having it follow unconstrained natural language navigation instructions. Recent work on the task of Vision-and-Language Navigation (VLN) has achieved significant progress in simulation. To assess the implications of this work for robotics, we transfer a VLN agent trained in simulation to a physical robot. To bridge the gap between the high-level discrete action space learned by the VLN agent, and the robot's low-level continuous action space, we propose a subgoal model to identify nearby waypoints, and use domain randomization to mitigate visual domain differences. For accurate sim and real comparisons in parallel environments, we annotate a 325m2 office space with 1.3km of navigation instructions, and create a digitized replica in simulation. We find that sim-to-real transfer to an environment not seen in training is successful if an occupancy map and navigation graph can be collected and annotated in advance (success rate of 46.8% vs. 55.9% in sim), but much more challenging in the hardest setting with no prior mapping at all (success rate of 22.5%).
34
Deep Residual Learning for Image Recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren et al. · 2016 · 222.1K citations · Full text
Automatic differentiation in PyTorch
Adam Paszke, Sam Gross, Soumith Chintala et al. · 2017 · 11.1K citations
Domain randomization for transferring deep neural networks from simulation to the real world
Josh Tobin, Rachel Fong, Alex Ray et al. · 2017 · 2.8K citations
Sinkhorn Distances: Lightspeed Computation of Optimal Transport
Marco Cuturi · 2013 · 2.2K citations