Skill learning and task outcome prediction for manipulation

TLDR

Learning complex motor skills for real‑world tasks is a hard problem in robotic manipulation that often requires painstaking manual tuning and design by a human expert. The study presents a reinforcement‑learning approach that learns new motor skills from demonstration, incorporates task domain knowledge, and uses sensor feedback to predict task outcomes. The method uses reinforcement learning from demonstration, integrates task‑specific constraints, learns proprioceptive sensor feedback for online monitoring and aborting, and is demonstrated on a PR2 robot performing a pool stroke and a chopstick‑based box‑flipping task. The approach enables the robot to acquire fine manipulation skills and markedly improves success rate and skill level from a coarse demonstration.

Abstract

Learning complex motor skills for real world tasks is a hard problem in robotic manipulation that often requires painstaking manual tuning and design by a human expert. In this work, we present a Reinforcement Learning based approach to acquiring new motor skills from demonstration. Our approach allows the robot to learn fine manipulation skills and significantly improve its success rate and skill level starting from a possibly coarse demonstration. Our approach aims to incorporate task domain knowledge, where appropriate, by working in a space consistent with the constraints of a specific task. In addition, we also present an approach to using sensor feedback to learn a predictive model of the task outcome. This allows our system to learn the proprioceptive sensor feedback needed to monitor subsequent executions of the task online and abort execution in the event of predicted failure. We illustrate our approach using two example tasks executed with the PR2 dual-arm robot: a straight and accurate pool stroke and a box flipping task using two chopsticks as tools.