Bayesian integration of visual and auditory signals for spatial localization

TLDR

Human observers localize events by integrating visual and auditory signals, with theories proposing either visual capture or weighted averaging based on signal reliability. The study evaluated two theories of spatial localization to determine how visual and auditory signals are weighted when they specify different spatial locations. The authors compared predictions of visual capture and maximum‑likelihood estimation by testing participants’ localization judgments under conditions where visual and auditory cues conflicted. Results show that relative signal reliability influences spatial judgments, yet there remains a bias toward visual cues, supporting aspects of both theories and highlighting implications for cue integration and adult neural plasticity.

Abstract

Human observers localize events in the world by using sensory signals from multiple modalities. We evaluated two theories of spatial localization that predict how visual and auditory information are weighted when these signals specify different locations in space. According to one theory (visual capture), the signal that is typically most reliable dominates in a winner-take-all competition, whereas the other theory (maximum-likelihood estimation) proposes that perceptual judgments are based on a weighted average of the sensory signals in proportion to each signal's relative reliability. Our results indicate that both theories are partially correct, in that relative signal reliability significantly altered judgments of spatial location, but these judgments were also characterized by an overall bias to rely on visual over auditory information. These results have important implications for the development of cue integration and for neural plasticity in the adult brain that enables humans to optimally integrate multimodal information.