A Human-Centered Taxonomy of Interaction Modalities and Devices

Abstract

In the field of human–computer interaction, taxonomies are used to classify and describe interaction (i.e. input and output) modalities, methods, technologies and devices. However, so far, most of these taxonomies and classification schemes consider only a subset of modalities and related methods, often reducing them to vision, audition and touch. Additionally, they are usually either technology- or task- instead of human-centered and thus vulnerable to rapid outdating with technological advancement and the advent of novel sensor and actor technologies. To tackle both problems, we propose a novel taxonomy that has been designed around the human and the human capabilities to sense output from and provide input to computer systems. We argue that although knowledge about the human sensory system might be up to changes as well, this process is considerably slower compared to technological advancement. Further, we reduce the taxonomy to what humans can actively and consciously sense or produce which is why novel findings related to human perception might not immediately compromise its validity. This article motivates the need for a novel taxonomy in the light of how computers and humans are able to perceive each other. It discusses existing taxonomies and introduces the new one that is intended to be (i) centered around the human and (ii) as holistic and timeless as possible. Further, the new taxonomy was evaluated with six human–computer interaction experts with regards to its practical use for researchers and different application scenarios. We present and discuss a new taxonomy for input/output modalities and devices that is based on humans’ capabilities to provide input to and perceive output from computer systems. The taxonomy is rooted in the physiological and cognitive characteristics of the human body. The taxonomy can be used by researchers but also practitioners As a human-centered and (nearly) holistic overview of interaction modalities and categories of devices For the classification of interaction modalities For the description of interaction devices The taxonomy has been qualitatively evaluated with six HCI experts.