Holoportation

TLDR

Audio‑visual communication with remote users is increasingly approaching face‑to‑face interaction. The authors present Holoportation, an end‑to‑end augmented and virtual reality telepresence system, and describe its interactive capabilities, application scenarios, and an initial qualitative study. The system uses depth cameras to reconstruct full‑scene 3D models and streams them in real time to remote users, enabling interactive telepresence. The system achieves high‑quality, real‑time 3D reconstructions of entire spaces and allows users wearing VR/AR displays to see, hear, and interact with remote participants as if they were physically present.

Abstract

We present an end-to-end system for augmented and virtual reality telepresence, called Holoportation. Our system demonstrates high-quality, real-time 3D reconstructions of an entire space, including people, furniture and objects, using a set of new depth cameras. These 3D models can also be transmitted in real-time to remote users. This allows users wearing virtual or augmented reality displays to see, hear and interact with remote participants in 3D, almost as if they were present in the same physical space. From an audio-visual perspective, communicating and interacting with remote users edges closer to face-to-face communication. This paper describes the Holoportation technical system in full, its key interactive capabilities, the application scenarios it enables, and an initial qualitative study of using this new communication medium.