Object Localization and Navigation Assistant for the Visually challenged

Abstract

Locating an item in a room or navigating to it is remaining as a significant bottleneck for visually impaired people. In this paper, we have devised a methodology to detect the presence of a target object and evaluate its relative depth from the user to return navigation directions. The user will be led to the target and issued warnings about the presence of obstacles using a text to speech engine. The proposed system employs a pipeline consisting of real-time multiple object detection, depth calculation, and text to speech components. For object detection, SSDLite MobilenetV2 model has been trained with the ms coco dataset using TensorFlow. The information is obtained using triangulation with stereo vision. Stereo vision is simulated using two Logitech c270 HD web cameras mounted on a T-shaped bracket. The position and size of the bounding boxes of object detection and depth data are used to obtain the 3D coordinates of both the target and obstacles. This prototype has been tested with objects belonging to the ms coco dataset and successfully leads the user to everyday items like cups and handbags. The current algorithm can be improved with respect to the number of object classes that can be detected and the accuracy can also be increased while detecting far away or fast-moving objects due to lighting issues and computational complexity.