Real-time 3D reconstruction at scale using voxel hashing

TLDR

Online 3D reconstruction is gaining interest due to real‑time consumer depth cameras, yet fusing live overlapping depth maps into a single model remains challenging when real‑time performance is required without sacrificing quality or scale. The paper presents an online system for large‑scale, fine‑resolution volumetric reconstruction that uses a memory‑ and speed‑efficient data structure. It employs spatial hashing to compress space, stores surface data only where measurements exist, streams data efficiently, and processes depth‑map pre‑processing, pose estimation, fusion, and rendering at real‑time rates on commodity GPUs. The system achieves interactive reconstructions of diverse scenes, capturing fine details and large‑scale environments, and outperforms current state‑of‑the‑art online systems in both performance and reconstruction quality.

Abstract

Online 3D reconstruction is gaining newfound interest due to the availability of real-time consumer depth cameras. The basic problem takes live overlapping depth maps as input and incrementally fuses these into a single 3D model. This is challenging particularly when real-time performance is desired without trading quality or scale. We contribute an online system for large and fine scale volumetric reconstruction based on a memory and speed efficient data structure. Our system uses a simple spatial hashing scheme that compresses space, and allows for real-time access and updates of implicit surface data, without the need for a regular or hierarchical grid data structure. Surface data is only stored densely where measurements are observed. Additionally, data can be streamed efficiently in or out of the hash table, allowing for further scalability during sensor motion. We show interactive reconstructions of a variety of scenes, reconstructing both fine-grained details and large scale environments. We illustrate how all parts of our pipeline from depth map pre-processing, camera pose estimation, depth map fusion, and surface rendering are performed at real-time rates on commodity graphics hardware. We conclude with a comparison to current state-of-the-art online systems, illustrating improved performance and reconstruction quality.