Predicting Goal-Directed Human Attention Using Inverse Reinforcement Learning

TLDR

Human gaze prediction is crucial for behavioral and computer vision, yet existing saliency models mainly address free‑viewing and fail to generalize to goal‑directed visual search. This work introduces the first inverse reinforcement learning model that learns the internal reward function and policy guiding human visual search. The model represents the viewer’s belief about object locations as dynamic contextual belief maps, learns them from the COCO‑Search18 dataset—10 participants, 18 target categories, 6202 images, ~300,000 fixations—and uses them to predict scanpaths for multiple targets. On COCO‑Search18, the IRL model surpasses baseline saliency methods in predicting human fixation scanpaths and search efficiency, and its recovered reward maps reveal target‑dependent object prioritization patterns.

Abstract

Human gaze behavior prediction is important for behavioral vision and for computer vision applications. Most models mainly focus on predicting free-viewing behavior using saliency maps, but do not generalize to goal-directed behavior, such as when a person searches for a visual target object. We propose the first inverse reinforcement learning (IRL) model to learn the internal reward function and policy used by humans during visual search. We modeled the viewer's internal belief states as dynamic contextual belief maps of object locations. These maps were learned and then used to predict behavioral scanpaths for multiple target categories. To train and evaluate our IRL model we created COCO-Search18, which is now the largest dataset of high-quality search fixations in existence. COCO-Search18 has 10 participants searching for each of 18 target-object categories in 6202 images, making about 300,000 goal-directed fixations. When trained and evaluated on COCO-Search18, the IRL model outperformed baseline models in predicting search fixation scanpaths, both in terms of similarity to human search behavior and search efficiency. Finally, reward maps recovered by the IRL model reveal distinctive target-dependent patterns of object prioritization, which we interpret as a learned object context.