Exploring Augmented Reality Approaches to Real-Time Captioning

Abstract

We explore an augmented reality (AR) approach to real-time captioning for people who are deaf and hard of hearing. In contrast to traditional captioning, which uses an external, fixed display (e.g., laptop or large screen), our approach allows users to manipulate the shape, number and placement of captions in 3D space. We discuss design factors, describe two early prototypes, and report on an autoethnographic evaluation of the prototypes. Preliminary findings suggest that, compared to traditional laptop-based captions, HMD captioning may increase glanceability, improve visual contact with speakers, and support access to other visual information (e.g., slides).