Scalability in Perception for Autonomous Driving: Waymo Open Dataset

TLDR

Autonomous driving research is growing, yet existing datasets are limited in scale and environmental diversity, hindering generalization across regions. The authors present a large‑scale, high‑quality, diverse dataset to better match real‑world self‑driving challenges. The dataset comprises 1,150 20‑second scenes of synchronized, calibrated LiDAR and camera data from diverse urban and suburban locations, fully annotated with consistent 2D and 3D bounding boxes, and is used to examine how dataset size and geographic diversity influence 3D detection performance. The dataset is 15× more diverse than the largest existing camera‑LiDAR set, and baseline experiments demonstrate its utility for 2D and 3D detection and tracking while revealing the impact of dataset size and geographic variation on performance. Data, code, and updates are available at http://www.waymo.com/open.

Abstract

The research community has increasing interest in autonomous driving research, despite the resource intensity of obtaining representative real world data. Existing self-driving datasets are limited in the scale and variation of the environments they capture, even though generalization within and between operating regions is crucial to the over-all viability of the technology. In an effort to help align the research community's contributions with real-world self-driving problems, we introduce a new large scale, high quality, diverse dataset. Our new dataset consists of 1150 scenes that each span 20 seconds, consisting of well synchronized and calibrated high quality LiDAR and camera data captured across a range of urban and suburban geographies. It is 15x more diverse than the largest camera+LiDAR dataset available based on our proposed diversity metric. We exhaustively annotated this data with 2D (camera image) and 3D (LiDAR) bounding boxes, with consistent identifiers across frames. Finally, we provide strong baselines for 2D as well as 3D detection and tracking tasks. We further study the effects of dataset size and generalization across geographies on 3D detection methods. Find data, code and more up-to-date information at http://www.waymo.com/open.