Real-time system for virtually touching objects in the real world using modality transformation from images to haptic information

Abstract

A system was developed that supports virtually touching objects in a real environment by real-time transformation of visual data into haptic data. With the proposed system, local visual data of real objects are acquired using active vision; the visual data are then subjected to high-speed parallel image processing, and transformed into virtual reactive forces from objects. A virtual tactile sensation is generated by exerting a reactive force on the operator's fingertip with a force display device. By using an original high-speed image processing system, the loop from visual data sensing through haptic display is operated at about 200 Hz. The present paper presents the basic concept of modality transformation, the system configuration, and results of experiments with objects of various shape carried out to verify the system's operation. © 1999 Scripta Technica, Syst Comp Jpn, 30(9): 17–24, 1999