Indoor scene segmentation using a structured light sensor

TLDR

The paper explores using a structured light depth sensor (Microsoft Kinect) to aid indoor scene segmentation and introduces a new challenging dataset with accurate depth maps and dense labels. The authors employ a CRF-based model to evaluate various depth representations and propose a novel prior on 3D location. The combined depth and intensity images yield dramatic performance gains over intensity alone, demonstrating the utility of structured light sensors for scene understanding.

Abstract

In this paper we explore how a structured light depth sensor, in the form of the Microsoft Kinect, can assist with indoor scene segmentation. We use a CRF-based model to evaluate a range of different representations for depth information and propose a novel prior on 3D location. We introduce a new and challenging indoor scene dataset, complete with accurate depth maps and dense label coverage. Evaluating our model on this dataset reveals that the combination of depth and intensity images gives dramatic performance gains over intensity images alone. Our results clearly demonstrate the utility of structured light sensors for scene understanding.