2019 · 13 citations · 32 references
ComputingComputer ScienceComputer VisionRobot PerceptionRobot Vision3D ReconstructionMulti-view GeometryPose EstimationScene ReconstructionScene UnderstandingSimultaneous Localization3D Scene UnderstandingConvolutional Neural NetworksGeometric ReconstructionLoop DetectionRelocalizationSemantic PredictionSlam-based ReconstructionTrajectory Estimation
We present a real-time dense mapping system which uses the predicted 2D semantic labels for optimizing the geometric quality of reconstruction. With a combination of Convolutional Neural Networks (CNNs) for 2D labeling and a Simultaneous Localization and Mapping (SLAM) system for camera trajectory estimation, recent approaches have succeeded in incrementally fusing and labeling 3D scenes. However, the geometric quality of the reconstruction can be further improved by incorporating such semantic prediction results, which is not sufficiently exploited by existing methods. In this paper, we propose to use semantic information to improve two crucial modules in the reconstruction pipeline, namely tracking and loop detection, for obtaining mutual benefits in geometric reconstruction and semantic recognition. Specifically for tracking, we use a novel probabilistic projective association approach to efficiently pick out candidate correspondences, where the confidence of these correspondences is quantified concerning similarities on all available short-term invariant features. For the loop detection, we incorporate these semantic labels into the original encoding through Randomized Ferns to generate a more comprehensive representation for retrieving candidate loop frames. Evaluations on a publicly available synthetic dataset have shown the effectiveness of our approach that considers such semantic hints as a reliable feature for achieving higher geometric quality.
32
Kaiming He, Georgia Gkioxari, Piotr Dollár et al. · 2017 · 28.8K citations
Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos et al. · IEEE Transactions on Pattern Analysis and Machine Intelligence · 2017 · 22.1K citations
Learning Deconvolution Network for Semantic Segmentation
Hyeonwoo Noh, Seunghoon Hong, Bohyung Han · 2015 · 4K citations
KinectFusion: Real-time dense surface mapping and tracking
Richard A. Newcombe, Andrew Fitzgibbon, Shahram Izadi et al. · 2011 · 3.9K citations