Toward Robust Robot 3-D Perception in Urban Environments: The UT Campus Object Dataset

Abstract

We introduce the UT Campus Object Dataset (CODa), a mobile robot egocentric perception dataset collected on the University of Texas Austin Campus. Our dataset contains 8.5 hours of multimodal sensor data from 3D LiDAR, stereo RGB and RGBD cameras, and a 9-DOF IMU. CODa contains 58 minutes of ground-truth annotations containing 1.3 million 3D bounding boxes with instance IDs for 53 semantic classes, 5000 frames of 3D semantic annotations for urban terrain, and pseudo-ground truth localization. We repeatedly traverse identical geographic regions for diverse indoor and outdoor areas, weather conditions, and times of the day. Using CODa, we empirically demonstrate that: 1) 3D object detection performance improves in urban settings when trained using CODa compared to existing datasets, 2) sensor-specific fine-tuning increases 3D object detection accuracy and 3) pretraining on CODa improves cross-dataset 3D object detection performance in urban settings compared to pretraining on AV datasets. We release benchmarks for 3D object detection and 3D semantic segmentation, with future plans for additional tasks. We publicly release CODa on the Texas Data Repository [1], pre-trained models, dataset development package, and interactive dataset viewer1. We expect CODa to be a valuable dataset for egocentric perception and planning for navigation in urban environments.