Driving Decision and Control for Automated Lane Change Behavior based on Deep Reinforcement Learning

Abstract

To fulfill high-level automation, an automated vehicle needs to learn to make decisions and control its movement under complex scenarios. Due to the uncertainty and complexity of the driving environment, most classical rule-based methods cannot solve the problem of complicated decision tasks. Deep reinforcement learning has demonstrated impressive achievements in many fields such as playing games and robotics. However, a direct application of reinforcement learning algorithm for automated driving still face challenges in handling complex driving tasks. In this paper, we proposed a hierarchical reinforcement learning based architecture for decision making and control of lane changing situations. We divided the decision and control process into two correlated processes: 1) when to conduct lane change maneuver and 2) how to conduct the maneuver. To be specific, we first apply Deep Q-network (DQN) to decide when to conduct the maneuver based on the consideration of safety. Subsequently, we design a Deep Q-learning framework with quadratic approximator for deciding how to complete the maneuver in longitudinal direction (e.g. adjust to the selected gap or just follow the preceding vehicle). Finally, a polynomial lane change trajectory is generated and Pure Pursuit Control is implemented for path tracking for the lane change situation. We demonstrate the effectiveness of this framework in simulation, from both the decision-making and control layers.