Combining Brain-Computer Interface and Eye Tracking for High-Speed Text Entry in Virtual Reality

Abstract

Gaze interaction provides an efficient way for users to communicate and control in virtual reality (VR) presented by head-mounted displays. In gaze-based text-entry systems, eye tracking and brain-computer interface (BCI) are the two most commonly used approaches. This paper presents a hybrid BCI system for text entry in VR by combining steady-state visual evoked potentials (SSVEP) and eye tracking. The user interface in VR designed a 40-target virtual keyboard using a joint frequency-phase modulation method for SSVEP. Eye position was measured by an eye-tracking accessory in the VR headset. Target-related gaze direction was detected by combining simultaneously recorded SSVEP and eye position data. Offline and online experiments indicate that the proposed system can type at a speed around 10 words per minute, leading to an information transfer rate (ITR) of 270 bits per minute. The results further demonstrate the superiority of the hybrid method over single-modality methods for VR applications.