RGB-D mapping: Using Kinect-style depth cameras for dense 3D modeling of indoor environments

TLDR

RGB‑D cameras, like the Microsoft Kinect, capture RGB images with per‑pixel depth, enabling dense 3D maps useful for robot navigation, manipulation, semantic mapping, and telepresence. The study investigates using RGB‑D cameras to build dense 3D maps of indoor environments. RGB‑D Mapping fuses visual features with shape‑based alignment through a joint optimization algorithm, incorporates view‑based loop‑closure detection, and performs pose optimization to produce globally consistent maps. Evaluation on two large indoor environments demonstrates that RGB‑D Mapping effectively combines visual and shape information to generate accurate dense 3D maps.

Abstract

RGB-D cameras (such as the Microsoft Kinect) are novel sensing systems that capture RGB images along with per-pixel depth information. In this paper we investigate how such cameras can be used for building dense 3D maps of indoor environments. Such maps have applications in robot navigation, manipulation, semantic mapping, and telepresence. We present RGB-D Mapping, a full 3D mapping system that utilizes a novel joint optimization algorithm combining visual features and shape-based alignment. Visual and depth information are also combined for view-based loop-closure detection, followed by pose optimization to achieve globally consistent maps. We evaluate RGB-D Mapping on two large indoor environments, and show that it effectively combines the visual and shape information available from RGB-D cameras.