Towards automatic shaping in robot navigation

Abstract

Shaping is a potentially powerful tool in reinforcement learning applications. Shaping often fails to function effectively because of a lack of understanding about its effects when applied in reinforcement learning settings and the use of inadequate algorithms in its implementation. Due to these difficulties current shaping techniques require some form of manual intervention. We examine some of the principles involved in shaping and present a new algorithm for automatic transferral of knowledge, which uses the Q-values established in a previous task to guide exploration in the learning of a new task. This algorithm is applied to two different but related robot navigation tasks.