Driving on Highway by Using Reinforcement Learning with CNN and LSTM Networks

Abstract

This work presents a powerful and intelligent driver agent, designed to operate in a preset highway situation using Policy Gradient Reinforcement Learning (RL) agent. Our goal is to create an agent that is capable of navigating safely in changing highway traffic and successfully accomplish to get through the defined section keeping the reference speed. Meanwhile, creating a state representation that is capable of extracting information from images based on the actual highway situation. The algorithm uses Convolutional Neural Network (CNN) with Long-Short Term Memory (LSTM) layers as a function approximator for the agent with discrete action space on the control level, e.g., acceleration and lane change. Simulation of Urban MObility (SUMO), an open-source microscopic traffic simulator is chosen as our simulation environment. It is integrated with an open interface to interact with the agent in real-time. The agent can learn from numerous driving and highway situations that are created and fed to it. The representation becomes more general by randomizing and customizing the behavior of the other road users in the simulation, thus the experience of the agent can be much more diverse. The article briefly describes the modeling environment, the details on the learning agent, and the rewarding scheme. After evaluating the experiences gained from the training, some further plans and optimization ideas are briefed.