Pseudo-LiDAR++: Accurate Depth for 3D Object Detection in Autonomous\n Driving

Abstract

Detecting objects such as cars and pedestrians in 3D plays an indispensable\nrole in autonomous driving. Existing approaches largely rely on expensive LiDAR\nsensors for accurate depth information. While recently pseudo-LiDAR has been\nintroduced as a promising alternative, at a much lower cost based solely on\nstereo images, there is still a notable performance gap. In this paper we\nprovide substantial advances to the pseudo-LiDAR framework through improvements\nin stereo depth estimation. Concretely, we adapt the stereo network\narchitecture and loss function to be more aligned with accurate depth\nestimation of faraway objects --- currently the primary weakness of\npseudo-LiDAR. Further, we explore the idea to leverage cheaper but extremely\nsparse LiDAR sensors, which alone provide insufficient information for 3D\ndetection, to de-bias our depth estimation. We propose a depth-propagation\nalgorithm, guided by the initial depth estimates, to diffuse these few exact\nmeasurements across the entire depth map. We show on the KITTI object detection\nbenchmark that our combined approach yields substantial improvements in depth\nestimation and stereo-based 3D object detection --- outperforming the previous\nstate-of-the-art detection accuracy for faraway objects by 40%. Our code is\navailable at https://github.com/mileyan/Pseudo_Lidar_V2.\n