Gesture-driven Interactions on a Virtual Hologram in Mixed Reality

Abstract

This paper describes a framework using the Microsoft Kinect 2 and the Microsoft HoloLens that can assist users in analyzing complex datasets. The system allows for groups of people to view a topological map as a virtual hologram in order to assist them in understanding complex datasets. In addition, the gestures that are built into the system were created with the idea of usability in mind. By allowing the user to resize, rotate and reposition the map, it opens up a much wider range of understanding the data that they have received. Custom gestures are also possible depending on the situation, such as raising or lowering the water level in a potential flood hot spot, or viewing graphs and charts associated with a specific data point.