Neuron · 2020 · 187 citations · 62 references
Life SciencesNeurosciencePsychologyAnimal BehaviorBehavioral NeuroscienceCognitive NeuroscienceAction SelectionCognitive ModelingDecision-making ModelsHabit FormationReinforcement LearningReward ProcessingBehavioral Neuroscience ApplicationsCalcium ImagingCalcium Imaging AnalysisCingulate Cortex FunctionHabitual ActionIntracellular Calcium ImagingReinforcementState-space Control
Behavioral control is not unitary. It comprises parallel systems, model based and model free, that respectively generate flexible and habitual behaviors. Model-based decisions use predictions of the specific consequences of actions, but how these are implemented in the brain is poorly understood. We used calcium imaging and optogenetics in a sequential decision task for mice to show that the anterior cingulate cortex (ACC) predicts the state that actions will lead to, not simply whether they are good or bad, and monitors whether outcomes match these predictions. ACC represents the complete state space of the task, with reward signals that depend strongly on the state where reward is obtained but minimally on the preceding choice. Accordingly, ACC is necessary only for updating model-based strategies, not for basic reward-driven action reinforcement. These results reveal that ACC is a critical node in model-based control, with a specific role in predicting future states given chosen actions.
62
Reinforcement Learning: An Introduction
Richard S. Sutton, Andy Barto · IEEE Transactions on Neural Networks · 1998 · 27.1K citations
Reinforcement Learning: An Introduction
IEEE Transactions on Neural Networks · 2005 · 25.8K citations
A mesoscale connectome of the mouse brain
Seung Wook Oh, Julie A. Harris, Lydia Ng et al. · Nature · 2014 · 2.9K citations · Full text