Apperception as a multisensory process in material experience

Abstract

Visual perspective has dominated experience research in human-technology interaction for decades now. The neglect of other sensory modalities is gradually being addressed by scholars and designers, who investigate user experience based on touch, smell, taste, sound and even expressive bodily interactions. In cognitive and affective processes, user experience is always multi-modal, not just regarding perceived multi-sensory information, but also while perceiving through one modality we mentally construct information relevant to the other senses. This article reports the results of an experiment, where participants (N = 52) appraised materials either only by touching them or only by seeing. The results indicate that with certain affects, the logic of the appraisal depends on the modality. These results are discussed within the theoretical framework of mental content, apperception, and appraisal. Further, we discuss the relevance of the findings for material design, especially in the context of multimodal interaction.