It’s in your eyes—using gaze-contingent stimuli to create truly interactive paradigms for social cognitive and affective neuroscience

TLDR

Social neuroscience has advanced in uncovering neural mechanisms of social cognition, yet new experimental approaches are needed to study social encounters in a truly interactive, online reciprocal manner. This article introduces a real‑time eyetracking‑controlled visual stimulation method during fMRI to generate gaze‑contingent stimuli. The method uses participants’ eye‑tracking data in real time to modulate visual stimuli within the constraints of fMRI. Using this technique, two paradigms animate a virtual character that responds to being looked at, enabling participants to engage in real‑time online interaction, and the authors discuss its potential as a new tool for social cognitive and affective neuroscience.

Abstract

The field of social neuroscience has made remarkable progress in elucidating the neural mechanisms of social cognition. More recently, the need for new experimental approaches has been highlighted that allow studying social encounters in a truly interactive manner by establishing 'online’ reciprocity in social interaction. In this article, we present a newly developed adaptation of a method which uses eyetracking data obtained from participants in real time to control visual stimulation during functional magnetic resonance imaging, thus, providing an innovative tool to generate gaze-contingent stimuli in spite of the constraints of this experimental setting. We review results of two paradigms employing this technique and demonstrate how gaze data can be used to animate a virtual character whose behavior becomes 'responsive’ to being looked at allowing the participant to engage in 'online’ interaction with this virtual other in real-time. Possible applications of this setup are discussed highlighting the potential of this development as a new 'tool of the trade’ in social cognitive and affective neuroscience.