From trash to treasure: Learning about brain images through multimodality

Abstract

Cognitive Neuroscientists use functional magnetic resonance imaging (fMRI) technology to generate digital images of the human brain. An fMRI image, as a final product of the scientific work, does not document movements and sounds that were present when such an image was recorded. Yet, a focus on actual moments of scientific practice reveals that such forgotten elements of practice can play important roles in understanding and knowledge acquisition. The multimodal interaction among scientists and digital screens shows how movements of the experimental subject and the scanner noise are performed to make images meaningful. Moreover, it suggests that the phenomena whose detection is crucial for a scientific reading of the brain images, such as motion artifacts, become visible as a result of coordination of various semiotic modalities (i.e., images, talk, body movements, gesture, etc.).