Asymmetric bidirectional view synthesis for free viewpoint and three-dimensional video

Abstract

Virtual view synthesis is a key technology for future free viewpoint video and three-dimensional video applications. A novel asymmetric bidirectional view synthesis method is proposed for multi-view video plus depth based systems. An intermediate virtual view is synthesized based on two adjacent reference views, which are termed as the main view and the auxiliary view. The main view contains two-dimensional color images plus per-pixel depth information. The auxiliary view only contains two-dimensional color images without depth information. The basic depth-image-based rendering technique is firstly optimized with a distance- and- depth weighted algorithm to solve the visibility and resampling problems and then applied in the following four-step view synthesis process. First, extract the occluded color data from the auxiliary view. Second, generate the corresponding occluded depth data. Then, generate two candidate virtual view images based on the main view information and the occluded information. Last, asymmetrically synthesize the two candidate virtual view images to generate the final virtual view image. Theoretical analysis and experimental results show that the proposed method can generate intermediate virtual views with high image quality.