Dynamics of Cortico-subcortical Cross-modal Operations Involved in Audio-visual Object Detection in Humans

TLDR

Neuroimaging studies are exploring how the brain integrates sensory modalities, but differing paradigms and stimulus types make it hard to link perceptual factors to underlying physiology. The study aimed to map the cross‑modal network underlying simple audio‑visual object detection. Using event‑related potentials, the authors recorded spatio‑temporal responses to objects defined by auditory, visual, or combined features during a bimodal detection task. Bimodal stimuli were detected faster, and early (<200 µs) interactions in occipito‑parietal, superior colliculus, and right temporo‑frontal regions differed from those seen in object identification, showing that multisensory integration depends on the perceptual task.

Abstract

Very recently, a number of neuroimaging studies in humans have begun to investigate the question of how the brain integrates information from different sensory modalities to form unified percepts. Already, intermodal neural processing appears to depend on the modalities of inputs or the nature (speech/non-speech) of information to be combined. Yet, the variety of paradigms, stimuli and technics used make it difficult to understand the relationships between the factors operating at the perceptual level and the underlying physiological processes. In a previous experiment, we used event-related potentials to describe the spatio-temporal organization of audio-visual interactions during a bimodal object recognition task. Here we examined the network of cross-modal interactions involved in simple detection of the same objects. The objects were defined either by unimodal auditory or visual features alone, or by the combination of the two features. As expected, subjects detected bimodal stimuli more rapidly than either unimodal stimuli. Combined analysis of potentials, scalp current densities and dipole modeling revealed several interaction patterns within the first 200μs post-stimulus: in occipito-parietal visual areas (45–85μs), in deep brain structures, possibly the superior colliculus (105–140μs), and in right temporo-frontal regions (170–185μs). These interactions differed from those found during object identification in sensory-specific areas and possibly in the superior colliculus, indicating that the neural operations governing multisensory integration depend crucially on the nature of the perceptual processes involved.