ACM Journal on Autonomous Transportation Systems · 2023 · 14 citations · 23 references
EngineeringAutonomous SystemsRobotics EngineeringAutonomous VehiclesVehicle TestingVehicle ValidationPerformance ValidationSim-to-real TransferSimulation-based AssessmentSimulation-based ValidationAutonomous Vehicle ValidationControlled Environment TestingDomain AdaptationDriving Simulator AssessmentRobot SimulationSimulated EnvironmentsSimulated-to-real BenchmarkingSimulation-based Testing
Modern autonomous systems require extensive testing to ensure reliability and build trust in ground vehicles. However, testing these systems in the real-world is challenging due to the lack of large and diverse datasets, especially in edge cases. Therefore, simulations are necessary for their development and evaluation. However, existing open-source simulators often exhibit a significant gap between synthetic and real-world domains, leading to deteriorated mobility performance and reduced platform reliability when using simulation data. To address this issue, our Scoping Autonomous Vehicle Simulation (SAVeS) platform benchmarks the performance of simulated environments for autonomous ground vehicle testing between synthetic and real-world domains. Our platform aims to quantify the domain gap and enable researchers to develop and test autonomous systems in a controlled environment. Additionally, we propose using domain adaptation technologies to address the domain gap between synthetic and real-world data with our SAVeS + extension. Our results demonstrate that SAVeS + is effective in helping to close the gap between synthetic and real-world domains and yields comparable performance for models trained with processed synthetic datasets to those trained on real-world datasets of same scale. Finally, we introduce two new autonomy driving datasets with complex scenes, essential sensor data, ground truth and improved imagery. The data is generated using both open-source and commercial simulators and processed through our SAVeS + domain adaptation pipeline. This paper highlights our efforts to quantify and address the domain gap between synthetic and real-world data for autonomy simulation. By enabling researchers to develop and test autonomous systems in a controlled environment, we hope to bring autonomy simulation one step closer to realization. 1
23
Are we ready for autonomous driving? The KITTI vision benchmark suite
Andreas Geiger, P Lenz, R. Urtasun · 2012 · 14.4K citations
Deep Ordinal Regression Network for Monocular Depth Estimation
Huan Fu, Mingming Gong, Chaohui Wang et al. · 2018 · 1.9K citations · Full text
Synthetic Data for Text Localisation in Natural Images
Ankush Gupta, Andrea Vedaldi, Andrew Zisserman · 2016 · 1.5K citations