SoC-VRP: A Deep-Reinforcement-Learning-Based Vehicle Route Planning Mechanism for Service-Oriented Cooperative ITS

Abstract

With the rapid development of emerging information technology and its increasing integration with transportation systems, the Intelligent Transportation System (ITS) is entering a new phase, called Cooperative ITS (C-ITS). It offers promising solutions to numerous challenges in traditional transportation systems, among which the Vehicle Routing Problem (VRP) is a significant concern addressed in this work. Considering the varying urgency levels of different vehicles and their different traveling constraints in the Service-oriented Cooperative ITS (SoC-ITS) framework studied in our previous research, the Service-oriented Cooperative Vehicle Routing Problem (SoC-VRP) is firstly analyzed, in which cooperative planning and vehicle urgency degrees are two vital factors. After examining the characteristics of both VRP and SoC-VRP, a Deep Reinforcement Learning (DRL)-based prioritized route planning mechanism is proposed. Specifically, we establish a deep reinforcement learning model with Rainbow DQN and devise a prioritized successive decision-making route planning method for SoC-ITS, where vehicle urgency degrees are mapped to three priorities: High for emergency vehicles, Medium for shuttle buses, and Low for the rest. All proposed models and methods are implemented, trained using various scenarios on typical road networks, and verified with SUMO-based scenes. Experimental results demonstrate the effectiveness of this hybrid prioritized route planning mechanism.