Modality is the message

Abstract

New media interfaces offer a wide variety of modalities for interacting with systems. While typing and clicking remain the staple of most interfaces, several other modalities have emerged in recent years, enabling users to perform a range of other actions, such as dragging, sliding, zooming-in/out, mousing-over and flipping through a revolving carousel of images (as in cover flow). While each modality offers a unique way of interacting with information, it is not clear whether it brings unique psychological advantages. Does a drag engender greater user engagement? Is the mouse-over likely to enhance user's perceptual bandwidth? A scientific assessment of such effects is impossible with existing interfaces given the confounded nature of modality combinations and information provided by them. Therefore, we designed six Web interface prototypes with identical content, differing only in modality, for experimentally isolating the effects of each, using a between-subjects design. Ongoing data collection involves both physiological and psychological measures of perceptual bandwidth and engagement.