Deep Object-Centric Policies for Autonomous Driving

TLDR

Learning visuomotor skills end‑to‑end is appealing yet deep networks are often uninterpretable and fail unexpectedly, while explicit object representations can improve robustness and interpretability in autonomous driving. The paper proposes a taxonomy of object‑centric models that combine object instances with end‑to‑end learning. These models integrate explicit object detection with end‑to‑end neural learning to form object‑centric policies. In both simulated and real‑world tests, object‑centric models beat object‑agnostic baselines, especially in scenes with other vehicles and pedestrians and in low‑data regimes.

Abstract

While learning visuomotor skills in an end-to-end manner is appealing, deep neural networks are often uninterpretable and fail in surprising ways. For robotics tasks, such as autonomous driving, models that explicitly represent objects may be more robust to new scenes and provide intuitive visualizations. We describe a taxonomy of “object-centric” models which leverage both object instances and end-to-end learning. In the Grand Theft Auto V simulator, we show that object-centric models outperform object-agnostic methods in scenes with other vehicles and pedestrians, even with an imperfect detector. We also demonstrate that our architectures perform well on real-world environments by evaluating on the Berkeley DeepDrive Video dataset, where an object-centric model outperforms object-agnostic models in the low-data regimes.