IEEE Transactions on Visualization and Computer Graphics · 2012 · 38 citations · 51 references
ComputingVisualization3D ModelingGraspingVisual PerceptionHapticsMixed RealityReal-time Rendering For Virtual RealityVirtual RealityColor SegmentationHaptic DevicesHaptic InteractionHaptic InterfacesMixed Reality EnvironmentsMixed Reality InteractionMixed Reality InterfacesSegmentation By Color
Visuo-haptic mixed reality consists of adding to a real scene the ability to see and touch virtual objects. It requires the use of see-through display technology for visually mixing real and virtual objects, and haptic devices for adding haptic interaction with the virtual objects. Unfortunately, the use of commodity haptic devices poses obstruction and misalignment issues that complicate the correct integration of a virtual tool and the user's real hand in the mixed reality scene. In this work, we propose a novel mixed reality paradigm where it is possible to touch and see virtual objects in combination with a real scene, using commodity haptic devices, and with a visually consistent integration of the user's hand and the virtual tool. We discuss the visual obstruction and misalignment issues introduced by commodity haptic devices, and then propose a solution that relies on four simple technical steps: color-based segmentation of the hand, tracking-based segmentation of the haptic device, background repainting using image-based models, and misalignment-free compositing of the user's hand. We have developed a successful proof-of-concept implementation, where a user can touch virtual objects and interact with them in the context of a real scene, and we have evaluated the impact on user performance of obstruction and misalignment correction.
51
Parallel Tracking and Mapping for Small AR Workspaces
Georg Klein, David W. Murray · 2007 · 4.2K citations
Unstructured lumigraph rendering
Chris Buehler, Michael Bosse, Leonard McMillan et al. · 2001 · 889 citations · Full text