Improving co-presence with augmented visual communication cues for sharing experience through video conference

TLDR

Video conferencing is increasingly used to share real‑world experiences with remote friends or family beyond face‑to‑face interaction. This study investigates whether adding augmented visual communication cues can enhance the experience of sharing a remote task space and collaborating together. The authors built a prototype that streams live video of the task space from an HMD or handheld display and allows voice, pointer, or annotation communication, and evaluated it in a user study comparing voice only, voice + pointer, and voice + annotation conditions. Results show that augmented visual cues significantly improve the sense of togetherness during remote puzzle‑solving, with the pointer being the most preferred cue and revealing distinct user behavior states.

Abstract

Video conferencing is becoming more widely used in areas other than face-to-face conversation, such as sharing real world experience with remote friends or family. In this paper we explore how adding augmented visual communication cues can improve the experience of sharing remote task space and collaborating together. We developed a prototype system that allows users to share live video view of their task space taken on a Head Mounted Display (HMD) or Handheld Display (HHD), and communicate through not only voice but also using augmented pointer or annotations drawn on the shared view. To explore the effect of having such an interface for remote collaboration, we conducted a user study comparing three video-conferencing conditions with different combination of communication cues: (1) voice only, (2) voice + pointer, and (3) voice + annotation. The participants used our remote collaboration system to share a parallel experience of puzzle solving in the user study, and we found that adding augmented visual cues significantly improved the sense of being together. The pointer was the most preferred additional cue by users for parallel experience, and there were different states of the users' behavior found in remote collaboration.