CCGN: Centralized collaborative graphical transformer multi-agent reinforcement learning for multi-intersection signal free-corridor

Abstract

Tackling traffic signal control through multi-agent reinforcement learning is a widely-employed approach. However, current state-of-the-art models have drawbacks: intersections optimize their own local rewards and cause traffic to waste time and fuel with a start-stop mode at each intersection. They also lack information sharing among intersections and their specialized policy hinders the ability to adapt to new traffic scenarios. To overcome these limitations, This work presents a centralized collaborative graph network (CCGN) with the core objective of a signal-free corridor once the traffic flows have waited at the entry intersection of the traffic intersection network on either side, the subsequent intersection gives the open signal as the traffic flows arrive. CCGN combines local policy networks (LPN) and global policy networks, where LPN employed at each intersection predicts actions based on Transformer and Graph Convolutional Network (GCN). In contrast, GPN is based on GCN and Q-network that receives the LPN states, traffic flow and road information to manage intersections to provide a signal-free corridor. We developed the Deep Graph Convolution Q-Network (DGCQ) by combining Deep Q-Network (DQN) and GCN to achieve a signal-free corridor. DGCQ leverages GCN's intersection collaboration and DQN's information aggregation for traffic control decisions Proposed CCGN model is trained on the robust synthetic traffic network and evaluated on the real-world traffic networks that outperform the other state-of-the-art models.