Image-Based 3D Object Reconstruction: State-of-the-Art and Trends in the Deep Learning Era

TLDR

3D reconstruction is an ill‑posed problem that has been studied for decades, and since 2015 CNN‑based image‑based methods have gained traction and shown impressive performance. This article surveys recent developments in image‑based 3D reconstruction using deep learning. The survey focuses on deep‑learning approaches that estimate generic object shapes from one or more RGB images, organizing the literature by shape representation, network architecture, and training strategy, while also covering class‑specific methods for humans and faces. It analyzes and compares key papers, highlights open problems, and outlines promising future research directions.

Abstract

3D reconstruction is a longstanding ill-posed problem, which has been explored for decades by the computer vision, computer graphics, and machine learning communities. Since 2015, image-based 3D reconstruction using convolutional neural networks (CNN) has attracted increasing interest and demonstrated an impressive performance. Given this new era of rapid evolution, this article provides a comprehensive survey of the recent developments in this field. We focus on the works which use deep learning techniques to estimate the 3D shape of generic objects either from a single or multiple RGB images. We organize the literature based on the shape representations, the network architectures, and the training mechanisms they use. While this survey is intended for methods which reconstruct generic objects, we also review some of the recent works which focus on specific object classes such as human body shapes and faces. We provide an analysis and comparison of the performance of some key papers, summarize some of the open problems in this field, and discuss promising directions for future research.