Augmented Virtuality Using Touch-Sensitive 3D-Printed Objects

Abstract

Virtual reality (VR) technologies have become more and more affordable and popular in the last five years thanks to hardware and software advancements. A critical issue for these technologies is finding paradigms that allow user interactions in ways that are as similar as possible to the real world, bringing physicality into the experience. Current literature has shown, with different experiments, that the mapping of real objects in virtual reality alongside haptic feedback significantly increases the realism of the experience and user engagement, leading to augmented virtuality. In this paper, we present a system to improve engagement in a VR experience using inexpensive, physical, and sensorized copies of real artefacts made with cheap 3D fabrication technologies. Based on a combination of hardware and software components, the proposed system gives the user the possibility to interact with the physical replica in the virtual environment and to see the appearance of the original cultural heritage artefact. In this way, we overcome one of the main limitations of mainstream 3D fabrication technologies: a faithful appearance reproduction. Using a consumer device for the real-time hand tracking and a custom electronic controller for the capacitive touch sensing, the system permits the creation of augmented experiences where the user with their hands can change the virtual appearance of the real replica object using a set of personalization actions selectable from a physical 3D-printed palette.