A dataset for developing and benchmarking active vision

TLDR

The authors introduce a publicly available dataset designed to simulate robotic vision in indoor scenes and validate it for active vision tasks, including next‑best‑move prediction via reinforcement learning. The dataset comprises over 20,000 RGB‑D images and 50,000 2D bounding boxes across nine scenes, and the authors train a fast object‑category detector and a deep‑network system for next‑best‑move prediction. Results demonstrate that state‑of‑the‑art object detection remains sensitive to object scale, occlusion, and viewing direction, and the dataset is freely downloadable at cs.unc.edu/~ammirato/active_vision_dataset_website/.

Abstract

We present a new public dataset with a focus on simulating robotic vision tasks in everyday indoor environments using real imagery. The dataset includes 20,000+ RGB-D images and 50,000+ 2D bounding boxes of object instances densely captured in 9 unique scenes. We train a fast object category detector for instance detection on our data. Using the dataset we show that, although increasingly accurate and fast, the state of the art for object detection is still severely impacted by object scale, occlusion, and viewing direction all of which matter for robotics applications. We next validate the dataset for simulating active vision, and use the dataset to develop and evaluate a deep-network-based system for next best move prediction for object classification using reinforcement learning. Our dataset is available for download at cs.unc.edu/~ammirato/active_vision_dataset_website/.