A vision system for providing 3D perception of the environment via transcutaneous electro-neural stimulation

Abstract

The development of effective user interfaces, appropriate sensors, and information processing techniques for enabling the blind to achieve additional perception of the environment is a relentless challenge confronting HCI and sensor researchers. To address this challenge we have developed a novel 3D vision system that can enable the 3D structure of the immediate environment to be perceived via head mounted stereo video cameras and electro-tactile data gloves without requiring any use of the eyes. The electro-neural vision system (ENVS) works by extracting a depth map from the camera images by measuring the disparity between the stereo images. This range data is then delivered to the fingers via electro-neural stimulation to indicate to the user the range of objects being viewed by the cameras. To interpret this information, the user only has to imagine that the hands are held in the direction viewed by the cameras, with fingers extended, and the amount of stimulation felt by each finger indicates the range of objects in the direction pointed at by each finger. This intuitive means of perceiving the 3D structure of the environment in real time effectively enables the user to navigate the environment without use of the eyes or other blind aids. Experimental results are provided demonstrating the potential that this form of 3D environment perception has at enabling the user to achieve localisation and obstacle avoidance skills without using the eyes.