Audiovisual Temporal Correspondence Modulates Human Multisensory Superior Temporal Sulcus Plus Primary Sensory Cortices

TLDR

The brain integrates multisensory information, and temporal patterning of inputs can signal whether stimuli are related. The study examined how temporal correspondence between auditory and visual streams influences human brain activity. Using fMRI, participants viewed jittered visual flashes presented on either side, with or without coincident, noncoincident, or isolated auditory tones, to probe audiovisual temporal correspondence. Temporal correspondence between auditory and visual streams produced stronger BOLD responses in the contralateral multisensory superior temporal sulcus and in primary visual and auditory cortices, with connectivity analyses showing mSTS driving sensory areas during correspondence and reduced activity during noncorrespondence.

Abstract

The brain should integrate related but not unrelated information from different senses. Temporal patterning of inputs to different modalities may provide critical information about whether those inputs are related or not. We studied effects of temporal correspondence between auditory and visual streams on human brain activity with functional magnetic resonance imaging (fMRI). Streams of visual flashes with irregularly jittered, arrhythmic timing could appear on right or left, with or without a stream of auditory tones that coincided perfectly when present (highly unlikely by chance), were noncoincident with vision (different erratic, arrhythmic pattern with same temporal statistics), or an auditory stream appeared alone. fMRI revealed blood oxygenation level-dependent (BOLD) increases in multisensory superior temporal sulcus (mSTS), contralateral to a visual stream when coincident with an auditory stream, and BOLD decreases for noncoincidence relative to unisensory baselines. Contralateral primary visual cortex and auditory cortex were also affected by audiovisual temporal correspondence or noncorrespondence, as confirmed in individuals. Connectivity analyses indicated enhanced influence from mSTS on primary sensory areas, rather than vice versa, during audiovisual correspondence. Temporal correspondence between auditory and visual streams affects a network of both multisensory (mSTS) and sensory-specific areas in humans, including even primary visual and auditory cortex, with stronger responses for corresponding and thus related audiovisual inputs.