Safe Reinforcement Learning of Lane Change Decision Making with Risk-Fused Constraint

Abstract

Deep reinforcement learning (DRL) has become a powerful method for autonomous driving while often lacking safety guarantees. In this paper, we propose a Risk-fused Constraint Deep Reinforcement Learning (RCDRL) with D3QN network for safe decision making in lane change maneuver. The problem is formulated as a state-wise MDP (SCMDP), which embeds a rule-based risk-fused Constraint module. We map the decision action to the trajectory layer via a polynomial curve-based trajectory planner, which is combined with the predicted trajectories of surrounding vehicles to assess future risk and correct the unsafe action. Therefore, the proposed method can deal with unsafe decision actions when training the policy network. To investigate the decision performance, the trained RCDRL policy is tested and validated under different traffic densities. In particular, we implement real vehicle tests to validate the effectiveness of the proposed method. Simulation and real vehicle tests demonstrated that the proposed RCDRL method achieves better performance, especially in safe decision. In addition, the framework can be extended with other advanced DRL networks.