Leveraging the Capabilities of Connected and Autonomous Vehicles and\n Multi-Agent Reinforcement Learning to Mitigate Highway Bottleneck Congestion

Abstract

Active Traffic Management strategies are often adopted in real-time to\naddress such sudden flow breakdowns. When queuing is imminent, Speed\nHarmonization (SH), which adjusts speeds in upstream traffic to mitigate\ntraffic showckwaves downstream, can be applied. However, because SH depends on\ndriver awareness and compliance, it may not always be effective in mitigating\ncongestion. The use of multiagent reinforcement learning for collaborative\nlearning, is a promising solution to this challenge. By incorporating this\ntechnique in the control algorithms of connected and autonomous vehicle (CAV),\nit may be possible to train the CAVs to make joint decisions that can mitigate\nhighway bottleneck congestion without human driver compliance to altered speed\nlimits. In this regard, we present an RL-based multi-agent CAV control model to\noperate in mixed traffic (both CAVs and human-driven vehicles (HDVs)). The\nresults suggest that even at CAV percent share of corridor traffic as low as\n10%, CAVs can significantly mitigate bottlenecks in highway traffic. Another\nobjective was to assess the efficacy of the RL-based controller vis-\\`a-vis\nthat of the rule-based controller. In addressing this objective, we duly\nrecognize that one of the main challenges of RL-based CAV controllers is the\nvariety and complexity of inputs that exist in the real world, such as the\ninformation provided to the CAV by other connected entities and sensed\ninformation. These translate as dynamic length inputs which are difficult to\nprocess and learn from. For this reason, we propose the use of Graphical\nConvolution Networks (GCN), a specific RL technique, to preserve information\nnetwork topology and corresponding dynamic length inputs. We then use this,\ncombined with Deep Deterministic Policy Gradient (DDPG), to carry out\nmulti-agent training for congestion mitigation using the CAV controllers.\n