"Move the couch where?" : developing an augmented reality multimodal interface

Sylvia Irawati, Scott Green, Mark Billinghurst, Andreas Duenser, Heedong Ko

2006 · 52 citations · 22 references

Concepts

Abstract

This paper describes an augmented reality (AR) multimodal interface that uses speech and paddle gestures for interaction. The application allows users to intuitively arrange virtual furniture in a virtual room using a combination of speech and gestures from a real paddle. Unlike other multimodal AR applications, the multimodal fusion is based on the combination of time-based and semantic techniques to disambiguate a users speech and gesture input. We describe our AR multimodal interface architecture and discuss how the multimodal inputs are semantically integrated into a single interpretation by considering the input time stamps, the object properties, and the user context.

References

22