National Conference on Artificial Intelligence · 2004 · 66 citations · 16 references
ComputingComputer ScienceRobotics EngineeringMachine LearningRobot ControlRoboticsBioinspired LocomotionBipedal LocomotionLegged Robot ControlLocomotor ControlPolicy LearningReinforcement LearningLocomotor Central Pattern GeneratorsRecurrent Neural CircuitsRecurrent Neural NetworksReinforcement Learning For LocomotionReinforcement Learning For RoboticsState-value Function
Animal's rhythmic movements such as locomotion are considered to be controlled by neural circuits called central pattern generators (CPGs). This article presents a reinforcement learning (RL) method for a CPG controller, which is inspired by the control mechanism of animals. Because the CPG controller is an instance of recurrent neural networks, a naive application of RL involves difficulties. In addition, since state and action spaces of controlled systems are very large in real problems such as robot control, the learning of the value function is also difficult. In this study, we propose a learning scheme for a CPG controller called a CPG-actor-critic model, whose learning algorithm is based on a policy gradient method. We apply our RL method to autonomous acquisition of biped locomotion by a biped robot simulator. Computer simulations show our method is able to train a CPG controller such that the learning process is stable.
16
Reinforcement Learning: An Introduction
IEEE Transactions on Neural Networks · 2005 · 25.8K citations
The development of Honda humanoid robot
Kazuo Hirai, Masato Hirose, Y. Haikawa et al. · 2002 · 1.9K citations