Poincaré-Map-Based Reinforcement Learning For Biped Walking

TLDR

The authors propose a model‑based reinforcement learning algorithm that enables a biped robot to learn how to modulate an observed walking pattern. The algorithm detects via‑points from recorded trajectories using a minimum‑jerk criterion, then modulates these via‑points as control actions guided by a learned Poincaré‑map model that predicts the next single‑support state, and is evaluated on both a simulated robot and a real biped. The approach successfully learns walking policies that allow the robot to perform stable biped locomotion.

Abstract

We propose a model-based reinforcement learning algorithm for biped walking in which the robot learns to appropriately modulate an observed walking pattern. Via-points are detected from the observed walking trajectories using the minimum jerk criterion. The learning algorithm modulates the via-points as control actions to improve walking trajectories. This decision is based on a learned model of the Poincaré map of the periodic walking pattern. The model maps from a state in the single support phase and the control actions to a state in the next single support phase. We applied this approach to both a simulated robot model and an actual biped robot. We show that successful walking policies are acquired.