On Adaptation, Maximization, and Reinforcement Learning Among Cognitive Strategies.

TLDR

Binary choice behavior in iterated tasks with immediate feedback shows deviations from maximization, including a payoff‑variability effect, underweighting of rare events, and loss aversion. The study aims to model these deviations using a reinforcement learning model of cognitive strategies (RELACS). RELACS captures the three deviations, learning curves, and the influence of information on uncertainty avoidance. The model predicts probability‑matching behavior and outperforms other models in data fit and cross‑experiment prediction.

Abstract

Analysis of binary choice behavior in iterated tasks with immediate feedback reveals robust deviations from maximization that can be described as indications of 3 effects: (a) a payoff variability effect, in which high payoff variability seems to move choice behavior toward random choice; (b) underweighting of rare events, in which alternatives that yield the best payoffs most of the time are attractive even when they are associated with a lower expected return; and (c) loss aversion, in which alternatives that minimize the probability of losses can be more attractive than those that maximize expected payoffs. The results are closer to probability matching than to maximization. Best approximation is provided with a model of reinforcement learning among cognitive strategies (RELACS). This model captures the 3 deviations, the learning curves, and the effect of information on uncertainty avoidance. It outperforms other models in fitting the data and in predicting behavior in other experiments.