Value Function Approximation on Non-Linear Manifolds for Robot Motor Control

Masashi Sugiyama, Hirotaka Hachiya, Christopher Towell, Sethu Vijayakumar

Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · 2007 · 15 citations · 14 references

Concepts

Abstract

The least squares approach works efficiently in value function approximation, given appropriate basis functions. Because of its smoothness, the Gaussian kernel is a popular and useful choice as a basis function. However, it does not allow for discontinuity which typically arises in real-world reinforcement learning tasks. In this paper, we propose a new basis function based on geodesic Gaussian kernels, which exploits the non-linear manifold structure induced by the Markov decision processes. The usefulness of the proposed method is successfully demonstrated in a simulated robot arm control and Khepera robot navigation.

References

14