2021 · 26 citations · 50 references
ComputingComputer ScienceComputer VisionMachine LearningRendering3D Reconstruction3D RenderingDeep LearningObject DetectionObject RecognitionScene Understanding3D Scene UnderstandingAdaptive ExplorationExplorationMonte Carlo Tree SearchRoom LayoutsScene ExplorationScene Understanding OptimizationSolution Space Exploration
We explore how a general AI algorithm can be used for 3D scene understanding to reduce the need for training data. More exactly, we propose a modification of the Monte Carlo Tree Search (MCTS) algorithm to retrieve objects and room layouts from noisy RGB-D scans. While MCTS was developed as a game-playing algorithm, we show it can also be used for complex perception problems. Our adapted MCTS algorithm has few easy-to-tune hyperparameters and can optimise general losses. We use it to optimise the posterior probability of objects and room layout hypotheses given the RGB-D data. This results in an analysis-by-synthesis approach that explores the solution space by rendering the current solution and comparing it to the RGB-D observations. To perform this exploration even more efficiently, we propose simple changes to the standard MCTS’ tree construction and exploration policy. We demonstrate our approach on the ScanNet dataset. Our method often retrieves configurations that are better than some manual annotations, especially on layouts.
50
ScanNet: Richly-Annotated 3D Reconstructions of Indoor Scenes
Angela Dai, Manolis Savva, Maciej Halber et al. · 2017 · 4K citations
A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play
David Silver, Thomas Hubert, Julian Schrittwieser et al. · Science · 2018 · 3.5K citations · Full text
ShapeNet: An Information-Rich 3D Model Repository
Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan et al. · arXiv (Cornell University) · 2015 · 2.4K citations · Full text