Spatiotemporal inpainting for recovering texture maps of partially occluded building facades

Abstract

We present a technique for constructing a "clean" texture map of a partially occluded building facade from a series of images taken from a moving camera. Building regions blocked by trees, signs, people, and other foreground objects in a minority of views can be recovered via temporal median filtering on a registered image mosaic of the planar facade. However, when such areas are occluded in the majority of camera views, appearance information from other visible portions of the facade provides a critical cue to correctly complete the mosaic. In this paper we apply a robust measure of spread to infer whether a particular mosaic pixel is occluded in a majority of views, and introduce a novel spatiotemporal timeline-based inpainting algorithm that uses an additional motion cue in order to fill the texture map in majority-occluded regions.