Scientific Reports · 2018 · 18 citations · 45 references
Life SciencesNeurobiologyNeuroscienceAnimal BehaviorBehavioral NeuroscienceMotor NeuroscienceAnimal Decision-makingDecision-making CircuitsExploration-exploitation TradeoffLearning In AnimalsLearning In Neural SystemsOperant ConditioningExplorationLearning From ExplorationLearning In Decision-makingMotor Cortex MicrocircuitsMotor Cortex ProjectionsPrefrontal Cortex CircuitsRodent LearningTemporal Uncertainty
Animals face the dilemma between exploiting known opportunities and exploring new ones, a decision-making process supported by cortical circuits. While different types of learning may bias exploration, the circumstances and the degree to which bias occurs is unclear. We used an instrumental lever press task in mice to examine whether learned rules generalize to exploratory situations and the cortical circuits involved. We first trained mice to press one lever for food and subsequently assessed how that learning influenced pressing of a second novel lever. Using outcome devaluation procedures we found that novel lever exploration was not dependent on the food value associated with the trained lever. Further, changes in the temporal uncertainty of when a lever press would produce food did not affect exploration. Instead, accrued experience with the instrumental contingency was strongly predictive of test lever pressing with a positive correlation between experience and trained lever exploitation, but not novel lever exploration. Chemogenetic attenuation of orbital frontal cortex (OFC) projection into secondary motor cortex (M2) biased novel lever exploration, suggesting that experience increases OFC-M2 dependent exploitation of learned associations but leaves exploration constant. Our data suggests exploitation and exploration are parallel decision-making systems that do not necessarily compete.
45
Reinforcement Learning: An Introduction
Richard S. Sutton, Andy Barto · IEEE Transactions on Neural Networks · 1998 · 27.1K citations
Reinforcement Learning: An Introduction
IEEE Transactions on Neural Networks · 2005 · 25.8K citations
A mesoscale connectome of the mouse brain
Seung Wook Oh, Julie A. Harris, Lydia Ng et al. · Nature · 2014 · 2.9K citations · Full text
Cortical substrates for exploratory decisions in humans
Nathaniel D. Daw, John P. O’Doherty, Peter Dayan et al. · Nature · 2006 · 2.3K citations · Full text
Learning the value of information in an uncertain world
Timothy E.J. Behrens, Mark W. Woolrich, Mark E. Walton et al. · Nature Neuroscience · 2007 · 2.2K citations