IEEE Transactions on Intelligent Vehicles · 2022 · 30 citations · 28 references
ComputingAutomotive EngineeringAutonomous SystemsAutonomous VehiclesMachine LearningVehicle DynamicsOptimal ControlPolicy LearningReinforcement LearningSafe Robot LearningVehicle ControlVehicle Control SystemsControl Barrier FunctionsControl Lyapunov FunctionsDeep Reinforcement LearningLyapunov ControlLyapunov FunctionsModel-free ControlSafe LearningSafe Reinforcement Learning
Deep Reinforcement Learning (DRL) has been successfully applied to learn policies for safety-critical systems with unknown model dynamics in simulation. DRL controllers though optimal in terms of reward, do not provide any safety and stability guarantees. With reliance on model information, safety conditions can be expressed as Control Barrier Functions (CBF’s) and performance objectives can be expressed as Control Lyapunov Functions (CLF’s) for real-time optimization-based controllers. In this work, we use an amalgamation of model-free RL and model-based controllers to establish safety and stability. We first design CLF, CBF Quadratic Programs (QP’s) for different driving manoeuvres on nominal vehicle dynamics. Reinforcement Learning (RL) agents are trained to learn policies for the actual vehicle with enhanced dynamics. In order to incorporate safety and stability while retaining optimal behaviour we selectively guide the RL agents using CLF, CBF QP’s. This results in both safe and stable (S <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> RL) policies. We empirically validate the proposed methodology on different driving manoeuvres.
28
Reinforcement Learning: An Introduction
Jeffrey D. Johnson, Jinghong Li, Zengshi Chen · Neurocomputing · 2000 · 8.7K citations