Hybrid DDPG Approach for Vehicle Motion Planning

Abstract

The paper presents a motion planning solution which combines classic control techniques with machine learning.For this task, a reinforcement learning environment has been created, where the quality of the fulfilment of the designed path by a classic control loop provides the reward function.System dynamics is described by a nonlinear planar single track vehicle model with dynamic wheel mode model.The goodness of the planned trajectory is evaluated by driving the vehicle along the track.The paper shows that this encapsulated problem and environment provides a one-step reinforcement learning task with continuous actions that can be handled with Deep Deterministic Policy Gradient learning agent.The solution of the problem provides a real-time neural network-based motion planner along with a tracking algorithm, and since the trained network provides a preliminary estimate on the expected reward of the current state-action pair, the system acts as a trajectory feasibility estimator as well.Highly automated and autonomous driving is expected to enhance the quality of road transportation in multiple aspects, such as increasing the level of safety while reducing fuel consumption and emissions.The development potential makes the topic one of the most intense research fields both for vehicle industry and related academic institutions.This paper deals with the problem of feasible motion planning, i.e. the design and evaluation of the trajectory that the vehicle must follow.Many different approaches have been evolved over the years to solve the motion planning problem for wheeled vehicles, all having advantages and drawbacks as well.Geometric approaches assemble the path of the vehicle from geometric curves as clothoids, circular arcs and splines.A popular choice is to define curvature as function of arc length (Li et al., 2015).They are often used in simple lowa