2022 International Conference on Robotics and Automation (ICRA) · 2022 · 71 citations · 35 references
ComputingAutonomous SystemsComputer ScienceAutonomous VehiclesComputer VisionSensor FusionAgent-based SimulationComputational ModelingSim-to-real TransferVehicle ModelingVehicle PerceptionAutonomous Vehicle ControlBenchmark DatasetsCamerasHigh Fidelity Vehicle SimulationMultimodal SensingRobustnessSensingSensor Simulation
Simulation has the potential to transform the development of robust algorithms for mobile agents deployed in safety-critical scenarios. However, the poor photorealism and lack of diverse sensor modalities of existing simulation engines remain key hurdles towards realizing this potential. Here, we present VISTA <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">†</sup> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">†</sup> Full code release for the VISTA data-driven simulation engine is available here: vista.csail.mit.edu., an open source, data-driven simulator that integrates multiple types of sensors for autonomous vehicles. Using high fidelity, real-world datasets, VISTA represents and simulates RGB cameras, 3D LiDAR, and event-based cameras, enabling the rapid generation of novel viewpoints in simulation and thereby enriching the data available for policy learning with corner cases that are difficult to capture in the physical world. Using VISTA, we demonstrate the ability to train and test perception-to-control policies across each of the sensor types and showcase the power of this approach via deployment on a full scale autonomous vehicle. The policies learned in VISTA exhibit sim-to-real transfer without modification and greater robustness than those trained exclusively on real-world data.
35
MuJoCo: A physics engine for model-based control
Emanuel Todorov, Tom Erez, Yuval Tassa · 2012 · 4.4K citations
Engineering, Mechanical Engineering, Robotics Engineering +17
End to End Learning for Self-Driving Cars
Mariusz Bojarski, Davide Testa, Daniel Dworakowski et al. · arXiv (Cornell University) · 2016 · 3.1K citations · Full text
Deep Ordinal Regression Network for Monocular Depth Estimation
Huan Fu, Mingming Gong, Chaohui Wang et al. · 2018 · 1.9K citations · Full text