Multimodal integration

Abstract

Advances in multimedia, virtual reality, and immersive environments have expanded human computer interaction beyond text and vision to include touch, gestures, voice, and 3D sound. Although well developed single modalities for communication already exist, we do not really understand the general problem of designing integrated multimodal systems. We explore this issue and the diverse approaches to it, with emphasis on a generic platform to support multimodal interaction.