Journal of Cognitive Neuroscience · 2023 · 15 citations · 62 references
Life SciencesNeurosciencePsychologyBehavioral NeuroscienceBiological PsychologyExperimental PsychologyArousal RegulationBehavioral Decision-makingChoice Under UncertaintyDecision-making ModelsUncertainty EstimationDirected ExplorationExploration Under UncertaintyExploration With Uncertainty EstimationGuided ExplorationLocus Coeruleus CircuitsPupil Dilation ResponseUncertainty-aware ExplorationUndirected Exploration
Exploration is an important part of decision making and is crucial to maximizing long-term rewards. Past work has shown that people use different forms of uncertainty to guide exploration. In this study, we investigate the role of the pupil-linked arousal system in uncertainty-guided exploration. We measured participants' (n = 48) pupil dilation while they performed a two-armed bandit task. Consistent with previous work, we found that people adopted a hybrid of directed, random, and undirected exploration, which are sensitive to relative uncertainty, total uncertainty, and value difference between options, respectively. We also found a positive correlation between pupil size and total uncertainty. Furthermore, augmenting the choice model with subject-specific total uncertainty estimates decoded from the pupil size improved predictions of held-out choices, suggesting that people used the uncertainty estimate encoded in pupil size to decide which option to explore. Together, the data shed light on the computations underlying uncertainty-driven exploration. Under the assumption that pupil size reflects locus coeruleus-norepinephrine neuromodulatory activity, these results also extend the theory of the locus coeruleus-norepinephrine function in exploration, highlighting its selective role in driving uncertainty-guided random exploration.
62
Cortical substrates for exploratory decisions in humans
Nathaniel D. Daw, John P. O’Doherty, Peter Dayan et al. · Nature · 2006 · 2.3K citations · Full text