Middleware support for implementing context-aware multimodal user interfaces

Abstract

In context-aware pervasive networking (Capnet) environments the user interface of a service application has to be adapted to whatever type of a device the user is using, taking into account the situation at hand, i.e. the user's current context, as well as user's personal preferences. In addition, new and more natural interaction styles have to be provided for the user. When designing and implementing user interfaces, or generating them automatically at runtime, some a priori knowledge of the capabilities of user's device and her personal preferences are required. Furthermore, runtime adaptation calls for mechanisms for recognizing context and delivering it. This paper presents the support that could be provided in middleware level for implementing context-aware multimodal user interfaces and the Capnet middleware user interface architecture model for providing that support. Adaptation algorithms are not discussed. The Capnet middleware backbone provides support for context-awareness via context recognition and automatic delivery of context data through a notification mechanism. For multimodal input and output third party components are required. The proposed architecture model is partly implemented and tested with software prototypes.