2019 · 239 citations · 38 references
ComputingComputer Science3D ModelingComputational GeometryComputer Vision3D ReconstructionGeometry ProcessingMulti-view GeometryScene ReconstructionScene Understanding3D Shape ReconstructionHandcrafted FeaturesIndoor Scene ReconstructionJoint EmbeddingJoint Embedding LearningJoint Feature EmbeddingModel AlignmentScan-to-cad Reconstruction
We present Scan2CAD, a novel data-driven method that learns to align clean 3D CAD models from a shape database to the noisy and incomplete geometry of a commodity RGB-D scan. For a 3D reconstruction of an indoor scene, our method takes as input a set of CAD models, and predicts a 9DoF pose that aligns each model to the underlying scan geometry. To tackle this problem, we create a new scan-to-CAD alignment dataset based on 1506 ScanNet scans with 97607 annotated keypoint pairs between 14225 CAD models from ShapeNet and their counterpart objects in the scans. Our method selects a set of representative keypoints in a 3D scan for which we find correspondences to the CAD geometry. To this end, we design a novel 3D CNN architecture that learns a joint embedding between real and synthetic objects, and from this predicts a correspondence heatmap. Based on these correspondence heatmaps, we formulate a variational energy minimization that aligns a given set of CAD models to the reconstruction. We evaluate our approach on our newly introduced Scan2CAD benchmark where we outperform both handcrafted feature descriptor as well as state-of-the-art CNN based methods by 21.39%.
38
ScanNet: Richly-Annotated 3D Reconstructions of Indoor Scenes
Angela Dai, Manolis Savva, Maciej Halber et al. · 2017 · 4K citations
KinectFusion: Real-time dense surface mapping and tracking
Richard A. Newcombe, Andrew Fitzgibbon, Shahram Izadi et al. · 2011 · 3.9K citations
Fast Point Feature Histograms (FPFH) for 3D registration
Radu Bogdan Rusu, Nico Blodow, Michael Beetz · 2009 · 3.8K citations
A volumetric method for building complex models from range images
Brian Curless, Marc Levoy · 1996 · 3.1K citations