Deep visual foresight for planning robot motion

TLDR

Scaling robot learning to many skills requires eliminating human supervision, and model‑based reinforcement learning offers a way to predict action effects without detailed human feedback. The study develops a method that combines deep action‑conditioned video prediction with model‑predictive control using entirely unlabeled training data. The method integrates deep action‑conditioned video prediction models with model‑predictive control, trained solely on unlabeled data. The approach works without a calibrated camera, instrumented setup, or precise sensing, enabling a real robot to push objects and handle novel items unseen during training.

Abstract

A key challenge in scaling up robot learning to many skills and environments is removing the need for human supervision, so that robots can collect their own data and improve their own performance without being limited by the cost of requesting human feedback. Model-based reinforcement learning holds the promise of enabling an agent to learn to predict the effects of its actions, which could provide flexible predictive models for a wide range of tasks and environments, without detailed human supervision. We develop a method for combining deep action-conditioned video prediction models with model-predictive control that uses entirely unlabeled training data. Our approach does not require a calibrated camera, an instrumented training set-up, nor precise sensing and actuation. Our results show that our method enables a real robot to perform nonprehensile manipulation - pushing objects - and can handle novel objects not seen during training.