Model-Based Influences on Humans' Choices and Striatal Prediction Errors

TLDR

The mesostriatal dopamine system is known to encode model‑free prediction errors in ventral striatal BOLD signals, yet evidence of model‑based planning and the interaction between these systems remains underexplored. The study aimed to disentangle model‑based and model‑free influences on human choice by designing a multistep decision task. Participants performed the task while ventral striatal BOLD activity was recorded, allowing the authors to test whether the signal reflected only model‑free or both types of predictions. The ventral striatal signal tracked both model‑free and model‑based predictions in proportions that best explained choice behavior, challenging the idea of a separate model‑free learner and supporting an integrated decision‑making architecture.

Abstract

The mesostriatal dopamine system is prominently implicated in model-free reinforcement learning, with fMRI BOLD signals in ventral striatum notably covarying with model-free prediction errors. However, latent learning and devaluation studies show that behavior also shows hallmarks of model-based planning, and the interaction between model-based and model-free values, prediction errors, and preferences is underexplored. We designed a multistep decision task in which model-based and model-free influences on human choice behavior could be distinguished. By showing that choices reflected both influences we could then test the purity of the ventral striatal BOLD signal as a model-free report. Contrary to expectations, the signal reflected both model-free and model-based predictions in proportions matching those that best explained choice behavior. These results challenge the notion of a separate model-free learner and suggest a more integrated computational architecture for high-level human decision-making.