What you can see is what you can feel-development of a visual/haptic interface to virtual environment

Abstract

We propose a new concept of visual/haptic interfaces called WYSIWYF display. The proposed concept provides correct visual/haptic registration using a vision based object tracking technique and a video keying technique so that what the user can see via a visual interface is consistent with what he/she can feel through a haptic interface. Using Chroma Keying, a live video image of the user's hand is extracted and blended with the graphic scene of the virtual environment. The user's hand "encounters" the haptic device exactly when his/her hand touches a virtual object in the blended scene. The first prototype has been built and the proposed concept was demonstrated.