2018 · 694 citations · 37 references
ComputingComputer ScienceComputer VisionRobot LearningRobot VisionDepth EstimationRobotic VisionScene ReconstructionScene UnderstandingStereo VisionVisual OdometryCamera Motion EstimationFeature-based Visual OdometryMonocular Depth EstimationOdometryPhotometric Error MinimizationScene Depth ReconstructionStereo Visual OdometryWarping
Learning-based methods have shown promise for single-view depth estimation and visual odometry, yet most rely on supervised training. This work investigates unsupervised learning of depth and visual odometry using stereo sequences and deep-feature warping. The authors employ stereo sequences to compute spatial and temporal photometric warp errors, constrain depth and motion to real-world scale, and incorporate deep-feature warping to train a model that predicts monocular depth and two-view odometry. Experiments on the KITTI dataset demonstrate that joint depth–odometry training improves depth accuracy, deep-feature warping outperforms simple photometric loss, and the method surpasses existing learning-based approaches on both tasks. Source code is available at https://github.com/Huangying-Zhan/Depth-VO-Feat.
Despite learning based methods showing promising results in single view depth estimation and visual odometry, most existing approaches treat the tasks in a supervised manner. Recent approaches to single view depth estimation explore the possibility of learning without full supervision via minimizing photometric error. In this paper, we explore the use of stereo sequences for learning depth and visual odometry. The use of stereo sequences enables the use of both spatial (between left-right pairs) and temporal (forward backward) photometric warp error, and constrains the scene depth and camera motion to be in a common, real-world scale. At test time our framework is able to estimate single view depth and two-view odometry from a monocular sequence. We also show how we can improve on a standard photometric warp loss by considering a warp of deep features. We show through extensive experiments that: (i) jointly training for single view depth and visual odometry improves depth prediction because of the additional constraint imposed on depths and achieves competitive results for visual odometry; (ii) deep feature-based warping loss improves upon simple photometric warp loss for both single view depth estimation and visual odometry. Our method outperforms existing learning based methods on the KITTI driving dataset in both tasks. The source code is available at https://github.com/Huangying-Zhan/Depth-VO-Feat.
37
Deep Residual Learning for Image Recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren et al. · 2016 · 222.1K citations · Full text
Adam: A Method for Stochastic Optimization
Diederik P. Kingma, Jimmy Ba · UvA-DARE (University of Amsterdam) · 2014 · 84.8K citations · Full text
Image quality assessment: from error visibility to structural similarity
Zhou Wang, Alan C. Bovik, Hamid R. Sheikh et al. · IEEE Transactions on Image Processing · 2004 · 56K citations
ImageNet Large Scale Visual Recognition Challenge
Olga Russakovsky, Jia Deng, Hao Su et al. · International Journal of Computer Vision · 2015 · 40.2K citations