2023 · 55 citations · 39 references
ComputingGeoinformaticsComputer VisionMachine VisionSensor Fusion3D ReconstructionDepth EstimationImagingPoint Cloud ProcessingScene ReconstructionSurface ReconstructionDense Depth EstimationDepth Map EstimationDepth Map ReconstructionGated FusionMean Absolute ErrorPoint Cloud FusionRadar FusionSparse-to-dense Depth Estimation
We present a method for inferring dense depth from a camera image and a sparse noisy radar point cloud. We first describe the mechanics behind mmWave radar point cloud formation and the challenges that it poses, i.e. ambiguous elevation and noisy depth and azimuth components that yields incorrect positions when projected onto the image, and how existing works have overlooked these nuances in camera-radar fusion. Our approach is motivated by these mechanics, leading to the design of a network that maps each radar point to the possible surfaces that it may project onto in the image plane. Unlike existing works, we do not process the raw radar point cloud as an erroneous depth map, but query each raw point independently to associate it with likely pixels in the image – yielding a semi-dense radar depth map. To fuse radar depth with an image, we propose a gated fusion scheme that accounts for the confidence scores of the correspondence so that we selectively combine radar and camera embeddings to yield a dense depth map. We test our method on the NuScenes benchmark and show a 10.3% improvement in mean absolute error and a 9.1% improvement in root-mean-square error over the best method. Code: https://github.com/nesl/radar-camera-fusion-depth.
39
Deep Residual Learning for Image Recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren et al. · 2016 · 222.1K citations · Full text
Adam: A Method for Stochastic Optimization
Diederik P. Kingma, Jimmy Ba · UvA-DARE (University of Amsterdam) · 2014 · 84.8K citations · Full text
Kaiming He, Georgia Gkioxari, Piotr Dollár et al. · 2017 · 28.8K citations
Edge Computing: Vision and Challenges
Weisong Shi, Jie Cao, Quan Zhang et al. · IEEE Internet of Things Journal · 2016 · 7.9K citations