Looking just below the eyes is optimal across face recognition tasks

TLDR

People tend to look at the eyes when viewing faces, and while eye contact has social value, it is unclear whether eye fixation during face identification serves a functional role or is merely a byproduct of social gaze behavior. The study proposes that gaze behavior during identity, emotional, and gender judgments reflects an adaptive strategy that learns eye‑movement plans to optimize performance in these perceptually important tasks. By extending a foveated ideal observer model to a large database of real‑world faces, the authors demonstrate that the optimality of these eye‑movement strategies generalizes across the population. The results show that humans fixate at points that maximize perceptual performance for identity, gender, and emotion, with optimal fixation locations differing modestly across tasks and accurately predicted by the foveated ideal observer, indicating that the visual system guides eye movements just below the eyes to optimize face recognition.

Abstract

When viewing a human face, people often look toward the eyes. Maintaining good eye contact carries significant social value and allows for the extraction of information about gaze direction. When identifying faces, humans also look toward the eyes, but it is unclear whether this behavior is solely a byproduct of the socially important eye movement behavior or whether it has functional importance in basic perceptual tasks. Here, we propose that gaze behavior while determining a person’s identity, emotional state, or gender can be explained as an adaptive brain strategy to learn eye movement plans that optimize performance in these evolutionarily important perceptual tasks. We show that humans move their eyes to locations that maximize perceptual performance determining the identity, gender, and emotional state of a face. These optimal fixation points, which differ moderately across tasks, are predicted correctly by a Bayesian ideal observer that integrates information optimally across the face but is constrained by the decrease in resolution and sensitivity from the fovea toward the visual periphery (foveated ideal observer). Neither a model that disregards the foveated nature of the visual system and makes fixations on the local region with maximal information, nor a model that makes center-of-gravity fixations correctly predict human eye movements. Extension of the foveated ideal observer framework to a large database of real-world faces shows that the optimality of these strategies generalizes across the population. These results suggest that the human visual system optimizes face recognition performance through guidance of eye movements not only toward but, more precisely, just below the eyes.