Depth-aided image inpainting for novel view synthesis

Abstract

Depth Image Based Rendering (DIBR) technique has been recognized as a promising tool for supporting advanced 3D video services required in MultiView Video (MVV) systems. However, an inherent problem with DIBR is to fill newly exposed areas (holes) caused by disocclusions. This paper addresses the disocclusion problem. To deal with small disocclusions, hole-filling strategies have been designed by the state-of-the-art through pre-processing techniques of the depth video. For larger disocclusions, where depth pre-processing has some limitations, we propose an inpainting approach to retrieve missing pixels. Specifically, we propose in the texture and structure propagation process to take into account the depth information by distinguishing foreground and background parts of the scene. Experimental results illustrate the efficiency of the proposed method.