Structured Domain Randomization: Bridging the Reality Gap by Context-Aware Synthetic Data

TLDR

Domain randomization typically places objects and distractors uniformly at random, whereas structured domain randomization samples them from problem‑specific probability distributions to reflect scene structure. This work introduces structured domain randomization to embed contextual information into synthetic training data. SDR generates images by sampling object and distractor placements from context‑aware distributions, enabling neural networks to learn from surrounding scene cues during detection. On the KITTI benchmark, SDR-trained models achieve competitive real‑data performance, outperform other synthetic generation methods and cross‑domain real data, and further improve when combined with real KITTI data.

Abstract

We present structured domain randomization (SDR), a variant of domain randomization (DR) that takes into account the structure of the scene in order to add context to the generated data. In contrast to DR, which places objects and distractors randomly according to a uniform probability distribution, SDR places objects and distractors randomly according to probability distributions that arise from the specific problem at hand. In this manner, SDR-generated imagery enables the neural network to take the context around an object into consideration during detection. We demonstrate the power of SDR for the problem of 2D bounding box car detection, achieving competitive results on real data after training only on synthetic data. On the KITTI easy, moderate, and hard tasks, we show that SDR outperforms other approaches to generating synthetic data (VKITTI, Sim 200k, or DR), as well as real data collected in a different domain (BDD100K). Moreover, synthetic SDR data combined with real KITTI data outperforms real KITTI data alone.