Reinforcement Learning for <i>True</i> Adaptive Traffic Signal Control

Baher Abdulhai, Rob Pringle, Grigoris Karakoulas

Journal of Transportation Engineering · 2003 · 534 citations · 16 references

Concepts

TL;DR

Real‑time adaptive control is central to intelligent transportation systems, and reinforcement learning offers a promising alternative to conventional prespecified models by enabling agents to learn action‑effect relationships directly, potentially overcoming model accuracy limits. The study aims to apply Q‑learning to traffic signal control, demonstrating its effectiveness on an isolated intersection and outlining future work to extend the approach to linear and networked signal systems for optimal control of congested two‑dimensional road networks. The authors employ Q‑learning, a simple reinforcement learning algorithm, to model traffic signal control, first testing it on an isolated intersection and then proposing its extension to linear and networked systems with dynamic route guidance. The Q‑learning approach yields encouraging results on an isolated traffic signal, especially under variable traffic conditions.

Abstract

The ability to exert real-time, adaptive control of transportation processes is the core of many intelligent transportation systems decision support tools. Reinforcement learning, an artificial intelligence approach undergoing development in the machine-learning community, offers key advantages in this regard. The ability of a control agent to learn relationships between control actions and their effect on the environment while pursuing a goal is a distinct improvement over prespecified models of the environment. Prespecified models are a prerequisite of conventional control methods and their accuracy limits the performance of control agents. This paper contains an introduction to Q-learning, a simple yet powerful reinforcement learning algorithm, and presents a case study involving application to traffic signal control. Encouraging results of the application to an isolated traffic signal, particularly under variable traffic conditions, are presented. A broader research effort is outlined, including extension to linear and networked signal systems and integration with dynamic route guidance. The research objective involves optimal control of heavily congested traffic across a two-dimensional road network—a challenging task for conventional traffic signal control methodologies.

References

16