BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning

TLDR

Existing driving datasets lack visual diversity and multitask support, limiting progress in autonomous driving vision. The authors create BDD100K, a 100K‑video dataset with 10 tasks, to benchmark heterogeneous multitask learning for autonomous driving. The dataset’s 100K videos span diverse geography, environments, and weather, and a benchmark is built to evaluate heterogeneous multitask learning. Experiments demonstrate that existing models need specialized training to perform heterogeneous tasks, and BDD100K facilitates future studies.

Abstract

Datasets drive vision progress, yet existing driving datasets are impoverished in terms of visual content and supported tasks to study multitask learning for autonomous driving. Researchers are usually constrained to study a small set of problems on one dataset, while real-world computer vision applications require performing tasks of various complexities. We construct BDD100K, the largest driving video dataset with 100K videos and 10 tasks to evaluate the exciting progress of image recognition algorithms on autonomous driving. The dataset possesses geographic, environmental, and weather diversity, which is useful for training models that are less likely to be surprised by new conditions. Based on this diverse dataset, we build a benchmark for heterogeneous multitask learning and study how to solve the tasks together. Our experiments show that special training strategies are needed for existing models to perform such heterogeneous tasks. BDD100K opens the door for future studies in this important venue.