Frontiers in Psychology · 2014 · 32 citations · 18 references
Life SciencesNeurosciencePsychologyBehavioral NeuroscienceCognitive PsychologyExperimental PsychologyDecision-making ModelsError-based LearningLearning In Neural SystemsOperant ConditioningReinforcement LearningBehavioral Data AnalysisComputer Simulation FrameworksHybrid Model-based LearningModel Fit ComparisonReinforcement Learning ModelsReversal LearningReward Prediction Error SignalingSurprise Effects
Humans are capable of correcting their actions based on actions performed in the past, and this ability enables them to adapt to a changing environment. The computational field of reinforcement learning (RL) has provided a powerful explanation for understanding such processes. Recently, the dual learning system, modeled as a hybrid model that incorporates value update based on reward-prediction error and learning rate modulation based on the surprise signal, has gained attention as a model for explaining various neural signals. However, the functional significance of the hybrid model has not been established. In the present study, we used computer simulation in a reversal learning task to address functional significance in a probabilistic reversal learning task. The hybrid model was found to perform better than the standard RL model in a large parameter setting. These results suggest that the hybrid model is more robust against the mistuning of parameters compared with the standard RL model when decision-makers continue to learn stimulus-reward contingencies, which can create abrupt changes. The parameter fitting results also indicated that the hybrid model fit better than the standard RL model for more than 50% of the participants, which suggests that the hybrid model has more explanatory power for the behavioral data than the standard RL model.
18
Reinforcement Learning: An Introduction
Richard S. Sutton, Andy Barto · IEEE Transactions on Neural Networks · 1998 · 27.1K citations
Reinforcement Learning: An Introduction
IEEE Transactions on Neural Networks · 2005 · 25.8K citations
Christopher J. Watkins, Peter Dayan · Machine Learning · 1992 · 9K citations · Full text
Christopher J. Watkins, Peter Dayan · Machine Learning · 1992 · 3.7K citations