A combined mixed reality and networked home approach to improving user interaction with consumer electronics

Abstract

User interaction in a modern networked home can be unintuitive when utilizing a smartphone for control. Different layers of menus and the lack of a tangible real-world interaction metaphor complicates the everyday usage of those devices in a smartphone. We propose a solution focusing on mixed reality interaction, by combining computer vision and ubiquitous computing techniques, to allow the user to simply point to the device of interest, to be controlled, via the smartphone camera. This adds value to the consumer electronics ecosystem by improving networked home user interaction experiences.