Free-Viewpoint TV

TLDR

Free‑viewpoint television (FTV) enables users to view a 3‑D scene from any viewpoint, offering infinite views and positioning it as the ultimate 3‑DTV; it is a key technology for immersive communication and is the focus of MPEG standardization efforts. This work proposes the FTV concept, builds the world’s first real‑time system covering capture to display, and reviews related technologies. The real‑time system employs a ray‑space representation, runs on a single PC and mobile player, and incorporates multiple ray‑capture interfaces including a 360° capture/display setup.

Abstract

Free-viewpoint television (FTV) is an innovative visual media that enables us to view a three-dimensional (3-D) scene by freely changing our viewpoints. We proposed the concept of FTV and constructed the world???s first real-time system including the complete chain of operation from image capture to display. We also carried out the FTV on a single personal computer (PC) and a mobile player. FTV is based on the ray-space method that represents one ray in real space with one point in the ray-space. We have developed several types of ray capture systems and interfaces such as a 360° capture/ray-reproducing display. FTV is regarded as the ultimate 3DTV, since it can generate infinite number of views. Thus, FTV is the key to immersive communication. Regarding FTV as the most challenging 3-D media, the Motion Picture Experts Group (MPEG) has been conducting its international standardization activities. This article reviews FTV and its related technologies.