Social SciencesDecision SciencesGame TheoryMarkov Decision ProcessesOptimizationCooperative Game TheoryCoordination GamesNash EquilibriumReinforcement LearningReinforcement Learning FoundationsStochastic GamesConstant-sum GamesConvergenceGame-theoretic LearningLearning In GamesMarkov GamesMulti-agent LearningQ-learningReinforcement Learning In Games
Bowling named two desiderata for multiagent learning algorithms: rationality and convergence. This paper introduces co~elated-Q learning, a natural generalization of Nash-Q and FF-Q that satisfies these criteria. NashoQ satisfies rationality, but in general it does not converge. FF-Q satisfies convergence, but in general it is not rational. Correlated-Q satisfies rationality by construction. This papers demonstrates the empirical convergence of correlated-Q on a standard testbed of general-sum Markov games.
24
Reinforcement Learning: An Introduction
IEEE Transactions on Neural Networks · 2005 · 25.8K citations