Scene Construction, Visual Foraging, and Active Inference

TLDR

Active inference posits that perception and action minimize expected variational free energy, extending risk‑sensitive control and expected utility theory to incorporate epistemic value for resolving uncertainty about ambiguous cues. The study applies active inference to saccadic visual searches to illustrate how approximate Bayesian inference under mean‑field assumptions relates to functional segregation in visual cortex and aims to model empirical saccadic behavior and intersubject variability. The authors implement an active inference framework that sequentially samples visual cues via saccades, accumulating evidence for scene categorization through prediction and postdiction, and relate this to functional segregation in visual cortex.

Abstract

This paper describes an active inference scheme for visual searches and the perceptual synthesis entailed by scene construction. Active inference assumes that perception and action minimize variational free energy, where actions are selected to minimize the free energy expected in the future. This assumption generalizes risk-sensitive control and expected utility theory to include epistemic value; namely, the value (or salience) of information inherent in resolving uncertainty about the causes of ambiguous cues or outcomes. Here, we apply active inference to saccadic searches of a visual scene. We consider the (difficult) problem of categorizing a scene, based on the spatial relationship among visual objects where, crucially, visual cues are sampled myopically through a sequence of saccadic eye movements. This means that evidence for competing hypotheses about the scene has to be accumulated sequentially, calling upon both prediction (planning) and postdiction (memory). Our aim is to highlight some simple but fundamental aspects of the requisite functional anatomy; namely, the link between approximate Bayesian inference under mean field assumptions and functional segregation in the visual cortex. This link rests upon the (neurobiologically plausible) process theory that accompanies the normative formulation of active inference for Markov decision processes. In future work, we hope to use this scheme to model empirical saccadic searches and identify the prior beliefs that underwrite intersubject variability in the way people forage for information in visual scenes (e.g., in schizophrenia).