Reinforcement Learning for Joint Control of Traffic Signals in a Transportation Network

Abstract

Reinforcement learning (RL) approaches have recently been spotlighted for use in adaptive traffic-signal control on an area-wide level. Most researchers have employed multi-agent reinforcement learning (MARL) algorithms wherein each agent shares a holistic traffic state and cooperates with other agents to reach a common goal. However, MARL algorithms cannot guarantee a global optimal solution unless the actions of all agents are fully coordinated. The present study employs a RL algorithm that recognizes an entire traffic state and jointly controls all the traffic signals of multiple intersections. With this approach, a deep Q-network (DQN) that depends solely on traffic images is extended to overcome the curse of dimensionality that is associated with a large state and action space. Several front layers in a deep convolutional neural network (CNN) to approximate the true Q-function are shared by each intersection approach. Weight parameters connecting the last hidden layer to the output layer are fixed. The proposed methodology outperforms a fixed-signal operation, a fully actuated signal operation, a multi-agent RL control without coordination, and a multi-agent RL control with partial coordination.