Local Implicit Grid Representations for 3D Scenes

TLDR

Shape priors learned from data are widely used for 3D reconstruction, yet indoor scenes lack such priors because typical autoencoders cannot manage their scale, complexity, or diversity, and most surfaces share geometric details at intermediate scales. The paper introduces Local Implicit Grid Representations, a scalable and general 3D shape representation. An autoencoder learns embeddings of local shape crops, and a decoder is used in a shape‑optimization process that assigns latent codes on a regular grid of overlapping crops so that interpolated decoded shapes fit partial or noisy observations. The method yields significantly better 3D surface reconstruction from sparse point observations than alternative approaches.

Abstract

Shape priors learned from data are commonly used to reconstruct 3D objects from partial or noisy data. Yet no such shape priors are available for indoor scenes, since typical 3D autoencoders cannot handle their scale, complexity, or diversity. In this paper, we introduce Local Implicit Grid Representations, a new 3D shape representation designed for scalability and generality. The motivating idea is that most 3D surfaces share geometric details at some scale - i.e., at a scale smaller than an entire object and larger than a small patch. We train an autoencoder to learn an embedding of local crops of 3D shapes at that size. Then, we use the decoder as a component in a shape optimization that solves for a set of latent codes on a regular grid of overlapping crops such that an interpolation of the decoded local shapes matches a partial or noisy observation. We demonstrate the value of this proposed approach for 3D surface reconstruction from sparse point observations, showing significantly better results than alternative approaches.