A User Study on Mixed Reality Remote Collaboration with Eye Gaze and Hand Gesture Sharing

TLDR

Supporting natural communication cues is critical for remote and face‑to‑face collaboration. The paper presents an MR remote collaboration system that lets a local worker share a live 3D panorama of their surroundings with a remote expert. The system enables the remote expert to provide visual task instructions and augmented gaze and gesture cues to the local worker, and a user study examined their impact on collaboration performance and experience. Combining gaze and gesture cues significantly increased co‑presence for both users and improved ease of conveying spatial actions compared to gaze alone.

Abstract

Supporting natural communication cues is critical for people to work together remotely and face-to-face. In this paper we present a Mixed Reality (MR) remote collaboration system that enables a local worker to share a live 3D panorama of his/her surroundings with a remote expert. The remote expert can also share task instructions back to the local worker using visual cues in addition to verbal communication. We conducted a user study to investigate how sharing augmented gaze and gesture cues from the remote expert to the local worker could affect the overall collaboration performance and user experience. We found that by combing gaze and gesture cues, our remote collaboration system could provide a significantly stronger sense of co-presence for both the local and remote users than using the gaze cue alone. The combined cues were also rated significantly higher than the gaze in terms of ease of conveying spatial actions.