Cooperative multi-agent traffic signal control system using fast gradient-descent function approximation for V2I networks

Abstract

The traffic signal control is the basic method to solve the urban congestions problem coming with the accelerating urbanization. For large city, it is challenge to improve the traffic signal control flexibility to adapt the real-time traffic change while to decrease the computation complexity. This paper proposes a cooperative Q-learning with function approximation(CQFA) algorithm for vehicle to infrastructure (V2I) networks. By gathering the local intersection traffic information from V2I networks and employing cooperative behaviors with neighboring intersections, the algorithm can achieve the optimal policy without any central supervising agents. To address the curse of dimensionality effectively, the Q-learning function is approximated by using a fast gradient-descent function approximation method to pick out the optimal Q-learning action. The Q-learning with Function Approximation algorithm combining the cooperative mechanism balances the urban traffic flow and uses approximating strategy to decrease the computation dimensionality. It can improve the traffic throughout, reduce the average waiting time and avoid congestions. Simulation results verify the effectiveness of the proposed algorithm.