2024 · 10 citations · 19 references
EngineeringAutonomous SystemsComputer ScienceHumanoid RoboticsRobot ControlRoboticsPolicy LearningReinforcement LearningReinforcement Learning FoundationsSafe Robot LearningSim-to-real TransferValue FunctionsHigh-dimensional State SpacesHumanoid LocomotionReinforcement Learning For RoboticsReward ShapingSafe Reinforcement LearningSafe Reinforcement Learning For RobotsSafe Sim-to-real TransferScalability
Reinforcement Learning (RL) has progressed from simple control tasks to complex real-world challenges with large state spaces. While RL excels in these tasks, training time remains a limitation. Reward shaping is a popular solution, but existing methods often rely on value functions, which face scalability issues. This paper presents a novel safety-oriented reward-shaping framework inspired by barrier functions, offering simplicity and ease of implementation across various environments and tasks. To evaluate the effectiveness of the proposed reward formulations, we conduct simulation experiments on CartPole, Ant, and Humanoid environments, along with real-world deployment on the Unitree Go1 quadruped robot. Our results demonstrate that our method leads to 1.4-2.8 times faster convergence and as low as 50-60% actuation effort compared to the vanilla reward. In a sim-to-real experiment with the Go1 robot, we demonstrated better control and dynamics of the bot with our reward framework. We have open-sourced our code at https://github.com/Safe-RL-IISc/barrier_shaping.
19
Human-level control through deep reinforcement learning
Volodymyr Mnih, Koray Kavukcuoglu, David Silver et al. · Nature · 2015 · 30K citations
Training language models to follow instructions with human feedback
Long Ouyang, Jeff Wu, Xu Jiang et al. · arXiv (Cornell University) · 2022 · 4.3K citations · Full text