An assistive EyeWear prototype that interactively converts 3D object locations into spatial audio

Abstract

We present an end-to-end prototype for an assistive EyeWear system aimed at Vision Impaired users. The system uses computer vision to detect objects on planar surfaces and sonifies their 3D locations using spatial audio. A key novelty of the system is that it operates in real time (15Hz), allowing the user to interactively affect the audio feedback by actively moving a headworn sensor. A quantitative user study was conducted on 12 blindfolded subjects performing an object localisation and placement task using our system. This detailed study of near field interactive spatial audio for users operating at around arm's length departs from existing studies focused on far-field audio and non-interactive systems. The object localisation accuracy achieved on naive users suggests that the EyeWear prototype has a lot of potential as a real world assistive device. User feedback collected from exit surveys and mathematical modelling of user errors provide several promising avenues to further improve system performance.