Toward A Thousand Lights: Decentralized Deep Reinforcement Learning for Large-Scale Traffic Signal Control

TLDR

Traffic congestion worldwide has spurred a surge in applying reinforcement learning to traffic signal control, yet coordinating signals in large‑scale urban networks—especially beyond a thousand intersections—remains untested and fraught with scalability, coordination, and data challenges. The study aims to develop and evaluate reinforcement‑learning agents for multi‑intersection traffic signal control in large‑scale networks, guided by transportation theory. The authors design RL agents that use a pressure‑based reward to coordinate signals regionally, enabling implicit coordination with reduced dimensionality, and validate the approach through extensive simulations, including a real‑world test with 2,510 Manhattan intersections.

Abstract

Traffic congestion plagues cities around the world. Recent years have witnessed an unprecedented trend in applying reinforcement learning for traffic signal control. However, the primary challenge is to control and coordinate traffic lights in large-scale urban networks. No one has ever tested RL models on a network of more than a thousand traffic lights. In this paper, we tackle the problem of multi-intersection traffic signal control, especially for large-scale networks, based on RL techniques and transportation theories. This problem is quite difficult because there are challenges such as scalability, signal coordination, data feasibility, etc. To address these challenges, we (1) design our RL agents utilizing ‘pressure’ concept to achieve signal coordination in region-level; (2) show that implicit coordination could be achieved by individual control agents with well-crafted reward design thus reducing the dimensionality; and (3) conduct extensive experiments on multiple scenarios, including a real-world scenario with 2510 traffic lights in Manhattan, New York City 1 2.