Reinforcement-Learning-Based Multilane Cooperative Control for On-Ramp Merging in Mixed-Autonomy Traffic

Abstract

On-ramp merging areas are typical bottlenecks in the freeway network. Vehicles merging on the ramp usually lead to reduced traffic efficiency, increased risk of collisions and fuel consumption. Most previous studies have mainly addressed single-lane merging, neglecting the complexity of multilane on-ramp entrances. To solve this problem, graph convolutional proximal policy optimization for connected and automated vehicles (GCAV-CPO) is proposed in a multilane on-ramp merging scenario in a mixed-autonomy traffic environment. This system is a distributed reinforcement learning framework that integrates vehicle graph structures with multiagent reinforcement learning (MARL) to coordinate the merging of connected and automated vehicles (CAVs) in a multilane traffic environment. An eco-friendly Markov decision process is developed, taking into account factors, such as energy consumption, travel time and safety, and incorporating a lane-changing factor into the reward function. Our method is evaluated under three different traffic pressure modes and four CAV penetration rates to thoroughly demonstrate its generalization ability. Experimental results show that the proposed GCAV-CPO significantly outperforms current baselines in terms of increasing traffic efficiency and safety, as well as in reducing energy consumption.