Adaptive state construction for reinforcement learning and its application to robot navigation problems

Abstract

This paper applies our state construction method by ART neural network to robot navigation problems. Agents in this paper consist of ART neural network and contradiction resolution mechanism. The ART neural network serves as a mean of state recognition which maps stimulus inputs to a certain state and state construction which creates a new state when a current stimulus input cannot be categorized into any known states. On the other hand, the contradiction resolution mechanism (CRM) uses agents' state transition table to detect inconsistency among constructed states. In the proposed method, two kinds of inconsistency for the CRM are introduced: "Different results caused by the same states and the same actions" and "Contradiction due to ambiguous states." The simulation results on the robot navigation problems confirm the effectiveness of the proposed method.