Depth Map Prediction from a Single Image using a Multi-Scale Deep Network

TLDR

Depth estimation from a single image is challenging because it requires integrating global and local cues and suffers from inherent ambiguity and scale uncertainty. The paper proposes a two‑stack deep network that first predicts a coarse global depth map and then refines it locally. The method employs two deep network stacks—one for coarse global prediction and one for local refinement—and applies a scale‑invariant error metric to evaluate depth relations. The approach attains state‑of‑the‑art performance on NYU Depth and KITTI datasets, accurately capturing depth boundaries without superpixelation.

Abstract

Predicting depth is an essential component in understanding the 3D geometry of a scene. While for stereo images local correspondence suffices for estimation, finding depth relations from a single image is less straightforward, requiring integration of both global and local information from various cues. Moreover, the task is inherently ambiguous, with a large source of uncertainty coming from the overall scale. In this paper, we present a new method that addresses this task by employing two deep network stacks: one that makes a coarse global prediction based on the entire image, and another that refines this prediction locally. We also apply a scale-invariant error to help measure depth relations rather than scale. By leveraging the raw datasets as large sources of training data, our method achieves state-of-the-art results on both NYU Depth and KITTI, and matches detailed depth boundaries without the need for superpixelation.