IEEE Robotics and Automation Letters · 2022 · 11 citations · 16 references
EngineeringComputer ScienceArtificial IntelligenceRobot LearningRoboticsData-efficient Robot LearningPolicy LearningReinforcement LearningRobot SafetySafe Robot LearningSim-to-real TransferBayesian OptimizationBayesian Optimization Under UncertaintyProbabilistic Reward LearningReinforcement Learning For RoboticsSafe LearningSafe Model-based Reinforcement LearningSafe Policy LearningSafe Sim-to-real Transfer
The framework of sim-to-real learning, i.e., training policies in simulation and transferring them to real-world systems, is one of the most promising approaches towards data-efficient learning in robotics. However, due to the inevitable reality gap between the simulation and the real world, a policy learned in the simulation may not always generate a safe behaviour on the real robot. As a result, during policy adaptation in the real world, the robot may damage itself or cause harm to its surroundings. In this work, we introduce SafeAPT, a multi-goal robot learning algorithm that leverages a diverse repertoire of policies evolved in simulation and transfers the most promising safe policy to the real robot through episodic interaction. To achieve this, SafeAPT iteratively learns probabilistic reward and safety models from real-world observations using simulated experiences as priors. Then, it performs Bayesian optimization to select the best policy from the repertoire with the reward model, while maintaining the specified safety constraint using the safety model. SafeAPT allows a robot to adapt to a wide range of goals safely with the same repertoire of policies evolved in the simulation. We compare SafeAPT with several baselines, both in simulated and real robotic experiments, and show that SafeAPT finds high-performing policies within a few minutes of real-world operation while minimizing safety violations during the interactions.
16
MuJoCo: A physics engine for model-based control
Emanuel Todorov, Tom Erez, Yuval Tassa · 2012 · 4.4K citations
Engineering, Mechanical Engineering, Robotics Engineering +17
Eric Brochu, Vlad M. Cora, Nando de Freitas · arXiv (Cornell University) · 2010 · 2.2K citations · Full text
Robots that can adapt like animals
Antoine Cully, Jeff Clune, Danesh Tarapore et al. · Nature · 2015 · 950 citations · Full text
Emergence of Locomotion Behaviours in Rich Environments
Nicolas Heess, Dhruva Tb, Sriram Srinivasan et al. · arXiv (Cornell University) · 2017 · 669 citations · Full text
Safe Reinforcement Learning via Shielding
Mohammed Alshiekh, Roderick Bloem, Rüdiger Ehlers et al. · 2018 · 620 citations