How are icons processed by the brain? Neuroimaging measures of four types of visual stimuli used in information systems

Abstract

We sought to understand how users interpret meanings of symbols commonly used in information systems, especially how icons are processed by the brain. We investigated C hinese and E nglish speakers' processing of 4 types of visual stimuli: icons, pictures, C hinese characters, and E nglish words. The goal was to examine, via functional magnetic resonance imaging ( fMRI ) data, the hypothesis that people cognitively process icons as logographic words and to provide neurological evidence related to human−computer interaction ( HCI ), which has been rare in traditional information system studies. According to the neuroimaging data of 19 participants, we conclude that icons are not cognitively processed as logographical words like C hinese characters, although they both stimulate the semantic system in the brain that is needed for language processing. Instead, more similar to images and pictures, icons are not as efficient as words in conveying meanings, and brains (people) make more effort to process icons than words. We use this study to demonstrate that it is practicable to test information system constructs such as elements of graphical user interfaces ( GUIs ) with neuroscience data and that, with such data, we can better understand individual or group differences related to system usage and user−computer interactions.