Complex-Valued Reinforcement Learning

Abstract

A new reinforcement learning algorithm with complex-valued functions is proposed. The algorithm is inspired by complex-valued neural networks introducing complex numbers representing phase and amplitude into a conventional neural network. The strong advantage of using complex values in reinforcement learning is that the state-action function in a time series can be easily extended. In particular, considering the coherence of each complex value, the proposed learning algorithm can represent the context of agent behavior. This extension allows compensating for the perceptual aliasing problem and provides for the intelligent behavior of mobile robots in the real world. The complex-valued functions are applied to the conventional reinforcement learning algorithms: Q-learning and profit sharing. These algorithms are evaluated by simple maze problems and a bar-carrying task involving perceptual aliasings. Simulation experiments show that the new algorithm can efficiently solve perceptual aliasing.